The following question is asked in order to better understand from your answers how to think "Async" when developing on Node.js
I have the following code:
router.get('/', function(req, res, next) {
... //Definition of rules and paramsObj
//Validation that returns a promise
Indicative.validate(rules,paramsObj)
.then(function(success){
//we passed the validation. start processing the request
//ProcessRequest has async calls but when all async functions are over, it sets paramObj.someVal with a calculated value.
processRequest(paramsObj,next);
//My problem is here. at this point paramsObj.someVal is not set yet. and therefore the response to the user will be incorrect.
res.send(paramsObj.someVal);
}).catch(function(err){
console.log(err);
next(err);
}).done();
}
I wish to understand how to better think "async" while i need to wait with the response to the user until all async functions are over.
My question is how to execute res.send(paramObj.someVal) only after the paramObj.someVal is set by some async methods in processRequest(paramsObj,next);
If you need to wait on the result of processRequest for paramsObj.someVal to be set then ultimately you need to handle that callback
router.get('/', function(req, res, next) {
... //Definition of rules and paramsObj
//Validation that returns a promise
Indicative.validate(rules,paramsObj)
.then(function(success){
//we passed the validation. start processing the request
//ProcessRequest has async calls but when all async functions are over, it sets paramObj.someVal with a calculated value.
processRequest(paramsObj, function(err) {
if (!err && !paramsObj.someVal) {
// raise a custom error if the value is not set
err = new Error('Value not set');
}
if (err) {
next(err);
} else {
res.send(paramsObj.someVal);
}
});
}).catch(function(err){
console.log(err);
next(err);
}).done();
}
Assuming the second argument to processRequest() is a completion callback, you can pass your own function for that callback and do your res.send() in that custom callback like this:
router.get('/', function(req, res, next) {
... //Definition of rules and paramsObj
//Validation that returns a promise
Indicative.validate(rules,paramsObj)
.then(function(success){
//we passed the validation. start processing the request
//ProcessRequest has async calls but when all async functions are over, it sets paramObj.someVal with a calculated value.
processRequest(paramsObj,function() {
res.send(paramsObj.someVal);
});
}).catch(function(err){
console.log(err);
next(err);
}).done();
}
Since you do res.send(...), I assume you don't want to actually call next() in that code path.
Related
I have API to deal with post request as follow(simplified):
myFunc(req: express.Request, res: express.Response, next){
let err = 'err detected!';
//validateSometing() returns a boolean value, true if validation pass false otherwise
if(!validateSomething()){
res.status(500).json(err);
return next(err);
}
//more code...logic if validation pass
}
I would like to know if return next(err); or return; is required to stop the function flow after sending the status and related err back to client. In other words, does res.status(500).json(err); stops the function flow?
Thanks!
next() is a middleware function in the application’s request-response cycle. You must call next() to pass control to the next middleware function. Otherwise, the request will be left hanging.
res.json(), res.send() is a express function used to send response to the client application . In other words it used this functions used to build your HTTP Reponse.
return keyword returns from your function, thus ending its execution. This means that any lines of code after it will not be executed.
Note : Both next() and res.send() will not end your function from execution. Where adding a return will stop function execution after triggering the callback.
Use return is to ensure that the execution stops after triggering the callback. In some circumstances, you may want to use res.send and then do other stuff.
Example :
app.use((req, res, next) => {
console.log('This is a middleware')
next()
console.log('This is first-half middleware')
})
app.use((req, res, next) => {
console.log('This is second middleware')
next()
})
app.use((req, res, next) => {
console.log('This is third middleware')
next()
})
Your output will be:
This is a middleware
This is second middleware
This is third middleware
This is first-half middleware
That is, it runs the code below next() after all middleware function finished.
However, if you use return next(), it will jump out the callback immediately and the code below return next() in the callback will be unreachable.
These two middleware functions behave differently and I cannot figure out why:
Here, the error will get trapped by try/catch:
router.get('/force_async_error/0', async function (req, res, next) {
try{
await Promise.reject(new Error('my zoom 0'));
}
catch(err){
next(err);
}
});
But here, the error will not get trapped by try/catch:
router.get('/force_async_error/1', async function (req, res, next) {
await Promise.reject(new Error('my zoom 1'));
});
I thought Express wrapped all middleware functions with try/catch, so I don't see how it would behave differently?
I looked into the Express source, and the handler looks like:
Layer.prototype.handle_request = function handle(req, res, next) {
var fn = this.handle;
if (fn.length > 3) {
// not a standard request handler
return next();
}
try {
fn(req, res, next); // shouldn't this trap the async/await error?
} catch (err) {
next(err);
}
};
so why doesn't the try/catch there capture the thrown error?
I'm going to add an answer here even though you've already accepted another one because I think what's going on here can be explained better and this will help others attempting to understand this.
In your code here:
router.get('/force_async_error/1', async function (req, res, next) {
await Promise.reject(new Error('my zoom 1'));
});
Let's discuss what is going on:
First, you declared the callback as async which you had to do in order to use await in it. An async function tells the interpreter to do several important things.
1. An async function always returns a promise. The resolved value of the promise will be whatever the function returns.
2. An async function is internally wrapped with a try/catch. If any exceptions are thrown in the top level scope of the function code, then those exceptions will be caught and will automatically reject the promise that the function returns.
3. An async function allows you to use await. This is an indicator to the interpreter that it should implement and allow the await syntax inside the function. This is tied to the previous two points above which is why you can't use await in just any 'ol function. Any uncaught rejections from await will also reject the promise that the function returns.
It's important to understand that while the async/await syntax allows you to kind of program with exceptions and try/catch like synchronous code, it isn't exactly the same thing. The function is still returning a promise immediately and uncaught exceptions in the function cause that promise to get rejected at some time later. They don't cause a synchronous exception to bubble up to the caller. So, the Express try/catch won't see a synchronous exception.
But here, the error will not get trapped by try/catch
I thought Express wrapped all middleware functions with try/catch, so I don't see how it would behave differently?
so why doesn't the try/catch [in Express] there capture the thrown error?
This is for two reasons:
The rejected promise is not a synchronous throw so there's no way for Express to catch it with a try/catch. The function just returns a rejected promise.
Express is not looking at the return value of the route handler callback at all (you can see that in the Express code you show). So, the fact that your async function returns a promise which is later rejected is just completely ignored by Express. It just does this fn(req, res, next); and does not pay attention to the returned promise. Thus the rejection of the promise falls on deaf ears.
There is a somewhat Express-like framework called Koa that uses promises a lot and does pay attention to returned promises and which would see your rejected promise. But, that's not what Express does.
If you wanted some Koa-type functionality in Express, you could implement it yourself. In order to leave other functionality undisturbed so it can work normally, I'll implement a new method called getAsync that does use promises:
router.getAsync = function(...args) {
let fn = args.pop();
// replace route with our own route wrapper
args.push(function(req, res, next) {
let p = fn(req, res, next);
// if it looks like a promise was returned here
if (p && typeof p.catch === "function") {
p.catch(err => {
next(err);
});
}
});
return router.get(...args);
}
You could then do this:
router.getAsync('/force_async_error/1', async function (req, res, next) {
await Promise.reject(new Error('my zoom 1'));
});
And, it would properly call next(err) with your error.
Or, your code could even just be this:
router.getAsync('/force_async_error/1', function (req, res, next) {
return Promise.reject(new Error('my zoom 1'));
});
P.S. In a full implementation, you'd probably make async versions of a bunch of the verbs and you'd implement it for middleware and you'd put it on the router prototype. But, this example is to show you how that could work, not to do a full implementation here.
This is because the call is asynchronous, take this code :
try {
console.log('Before setTimeout')
setTimeout(() => {
throw new Error('Oups')
})
console.log('After setTimeout')
}
catch(err) {
console.log('Caught', err)
}
console.log("Point of non-return, I can't handle anything anymore")
If you run it you should see that the error is triggered after Point of non-return.
When we're at the throw line it's too late, we're outside of try/catch. At this moment if an error is thrown it'll be uncaught.
You can work around this by using async/await in the caller (doesn't matter for the callee), ie :
void async function () {
try {
console.log('Before setTimeout')
await new Promise((resolve, reject) =>
setTimeout(() => {
reject(new Error('Oups'))
})
)
console.log('After setTimeout')
}
catch(err) {
console.log('Caught', err.stack)
}
console.log("Point of non-return, I can't handle anything anymore")
}()
Finally, this means that for Express to handle async errors you would need to change the code to :
async function handle(req, res, next) {
// [...]
try {
await fn(req, res, next); // shouldn't this trap the async/await error?
} catch (err) {
next(err);
}
}
A better workaround:
Define a wrap function like this :
const wrap = fn => (...args) => Promise
.resolve(fn(...args))
.catch(args[2])
And use it like this :
app.get('/', wrap(async () => {
await Promise.reject('It crashes!')
}))
Neither of these really answer the question, which if I understand correctly is:
Since the async/await syntax lets you handle rejected "awaits" with non-async style try/catch syntax, why doesn't a failed "await" get handled by Express' try/catch at the top level and turned into a 500 for you?
I believe the answer is that whatever function in the Express internals that calls you would also have to be declared with "async" and invoke your handler with "await" to enable async-catching try/catch to work at that level.
Wonder if there's a feature request for the Express team? All they'd need to add is two keywords in two places. If success, do nothing, if exception hand off to the error handling stack.
Beware that if you don't await or return the promise, it has nothing to do with express - it just crashes the whole process.
For a general solution for detached promise rejections:
https://stackoverflow.com/a/28709667
Copied from above answer:
process.on("unhandledRejection", function(reason, p){
console.log("Unhandled", reason, p); // log all your errors, "unsuppressing" them.
//throw reason; // optional, in case you want to treat these as errors
});
I'm starting out w/ NodeJS and Express. Coming from the other popular scripting languages and C++ background, asynchronously calling DB functions is a bit foreign. I've sorted out a pattern, but I'm still curious about catching exceptions. Below is my basic pattern.
var callback = function(req, res) {
// do stuff
connection.query(queryString, function(err,result){
if (err) throw err;
// process results.
};
};
var express = require('express');
var app = express();
app.get('/', callback);
app.listen(3000,function() {
console.log('listening');
};
Generally I have a lot of endpoints and callbacks. I'm a bit lost on where I set up ta try/catch block to catch errors thrown in the callback though. I've looked around for some suggestions, but a lot of them seem to be on the web framework (if any) being used.
When you throw in an asynchronous callback, the exception just goes back to the internals of the database event handler and there is NO way for you to ever catch or handle that exception. So, basically it does no good at all. It will just cause you to abort the handling of that request and you will never send a response on that request.
Basically, you have several choices for how to handle the error. You can handle it completely right in each endpoint and send some sort of error response.
Send Response right at each point of error
app.get('/', function(req, res) {
// do stuff
connection.query(queryString, function(err,result){
if (err) return res.status(500).send(someErrorResponse);
// process results.
};
});
Forward on to centralized error handler
Or, you can forward the error on to a centralized error handler by calling next(err):
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString, function(err,result){
// if error, forward it on to our centralized error handler
if (err) return next(err);
// process results.
};
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
See Nodejs handle unsupported URLs and request types for more info on the ways to have generalized error handlers in Express.
Use promises to collect errors within each route
If you are using more involved sequences of asynchronous operations where you may have more than one async operation sequenced together, then it does get to be a pain to handle errors at every single async operation. This is where using promises with all your async operations more easily allows all the errors to percolate up to one .catch() statement at the top level of each route. You don't say what database you're using, but here's an idea what that looks like. The general idea is that you can write your code so that all promise rejections (e.g. errors) will propagate up to one central .catch() in each route handler and you can then call next(err) from that .catch(), sending the error to your centralized error handler. Here's how that looks for a recent version of Mongoose (you didn't say which database you were using) with one database operation.
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString).exec().then(function(result){
// process results.
}).catch(next);
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
And, here's what it looks like if you have more than one operation:
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString).exec().then(function(result){
// process results, then make another query
// return the promise from this second operaton so both results
// and error are chained to the first promise
return connection.query(...).exec();
}).then(function(result) {
// process chained result
}).catch(next);
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
Since ES6 built in support for promises and ES7 will add support for async/await for asynchronous operations (which is based on promises) and all significant libraries that offer asynchronous operations have added or are adding support for promises, it is clear that promises are the future of the language for managing asynchronous operations. That would be my strong recommendation.
You should never, ever throw an error like that! :) The reason is that at some point your whole node app will just stop working, because of some db query failed. This should be handled instead of just die.
And because this is a route handler - handles specific url that the user is getting (for example /), there should be some output. You can always show a page with status 500 and a nice design, if there was such an error that you cannot handle or you might have your internal state messed up.
So basically just act as nothing happened - return respones of any kind, or even render a page, but provide information that something went wrong.
Also, a common scenario is something like what Alon Oz presented. All routes in express are actually a middleware functions, that are called one after another. If the route does not match the requested one, the function just skips and the next one is called. You can manually control that. The actual pattern of the router is this:
app.get('/', function(req, res, next) {
// you can have the request
// you can send response like res.send('hello')
// OR you can skip this function using NEXT
});
The actual signature of next is next(err). So if you call it without any arguments, it will just skip to the next middleware. If you call it with an argument, it will skip all regular functions and go to the last ones in the stack, or more specifically the ones that handle errors. They are like the regular ones, but taking four arguments instead of three:
app.use(function (err, req, res, next) { });
It's very important to understand that this function will be called if you call next with an argument. Throwing an error won't do any good! Of course if none of your routes match the specific criteria (url) the final one will in the call will be called, so you can still handle the "not found" error.
This is a common scenario that you will use:
// development error handler, will print stacktrace
if (app.get('env') === 'development') {
app.use(function(err, req, res, next) {
debug('ERROR [ip: %s]:: dev env -> ', req.ip, err); // I'm using debug library - very helpful
res.status(err.status || 500);
res.render('deverr', { // I render custom template with the whole stack beautifully displayed
errMessage: err.message,
error: err
});
});
}
// production error handler, no stacktraces leaked to user
app.use(function(err, req, res, next) {
res.status(err.status || 500);
res.render('pages/error', { // custom error page with nice design and a message
errMessage: err.message,
error: {}
});
});
Hope that helps! :)
Since you are using express, it has its own way to handle exceptions,
defined like this:
function clientErrorHandler (err, req, res, next) {
if (req.xhr) {
res.status(500).send({ error: 'Something failed!' })
} else {
next(err)
}
}
app.use(clientErrorHandler)
For more info:
https://expressjs.com/en/guide/error-handling.html
There are most commonly three major types of errors that we need to take into account.
Promise failures (Any failures that come up during async/await or result of a promise in then/catch)
In order to handle promise failures, as suggested in the strong loop document or node js 2018 best practices, it's important to have a common function that can handle it.
// app.js file
app.get('/:id', async (req,res,next) => {
if(!req.params.id) {
return res.status(412).send('enter a valid user id');
}
try {
const results = await UserDAL(id);
} catch(e) {
next(e);
}
}
// common error middleware defined in middleware/error.js
module.exports = function (err,req,res,next) {
logger.error(`${err.status || 500} - ${err.message} - ${req.originalUrl} - ${req.method} - ${req.ip}`);
return res.status(500).send('something failed.');
};
Unhandled Rejections
process.on('unhandledRejection', e => {
// do something
});
Unhandled exceptions
process.on('uncaughtException', e => {
// do something
});
If you see a lot of try/ catch blocks in your express methods you can abstract that to a separate async function like below:
module.exports = function asyncMiddleWare(handler) {
return async (req,res,next) => {
try {
await handler(req,res)
} catch(e) {
next(e);
}
}
};
I would like to get some help with the following problem. I'm writing my bsc thesis, and this small part of code would be responsible for registering a user. (I'm new at nodejs actually). I'm using express and mongoose for this too.
I would like to process the request data, and check for some errors, first I would like to check if all fields exist, secondly if someone already registered with this e-mail address.
Based on the errors (or on success), I would like to send different responses. If a field is missing, then a 400 Bad request, if a user exists, then 409 Conflict, and 200 OK, if everything is ok. But I would only like to do the callback if there are no errors, but I'm kinda stuck here... I get the error Can't set headers after they are sent, which is obvious actually, because JS continues processing the code even if a response is set.
app.post('/register', function (req, res) {
var user = new User(req.body);
checkErrors(req, res, user, registerUser);
});
var registerUser = function(req, res, user){
user.save(function(err, user){
if (err) return console.log(err);
});
res.sendStatus(200);
};
var checkErrors = function(req, res, user, callback){
var properties = [ 'firstName', 'lastName', 'email', 'password', 'dateOfBirth' ];
for(var i = 0; i < properties.length; i++){
if(!req.body.hasOwnProperty(properties[i])){
res.status(400).send('field ' + properties[i] + ' not found');
}
}
var criteria = {
email: req.body.email
};
User.find(criteria).exec(function(err, user){
if(user.length > 0){
res.status(409).send('user already exists');
}
});
callback(req, res, user);
};
I think the problem is in the for loop in checkErrors. Since you call res.status(400).send() within the loop, you can end up calling it multiple times, which will trigger an error after the first call since a response will already have been sent back to the client.
Inside the loop, you can instead add missing fields to an array, then check the length of the array to see if you should respond with a 400 or continue. That way, you will only call res.status(400).send() one time.
For example:
...
var missingFields = [];
for(var i = 0; i < properties.length; i++){
if(!req.body.hasOwnProperty(properties[i])){
missingFields.push(properties[i]);
}
}
if(missingFields.length > 0) {
return res.status(400).send({"missingFields" : missingFields});
}
...
In general, I advise that you put return in front of each res.send() call, to ensure that no others are accidentally called later on.
An example of this is:
User.find(criteria).exec(function(err, user){
// We put return here in case you later add conditionals that are not
// mutually exclusive, since execution will continue past the
// res.status() call without return
if(user.length > 0){
return res.status(409).send('user already exists');
}
// Note that we also put this function call within the block of the
// User.find() callback, since it should not execute until
// User.find() completes and we can check for existing users.
return callback(req, res, user);
});
You probably noticed that I moved callback(req, res, user). If we leave callback(req, res, user) outside the body of the User.find() callback, it is possible that it will be executed before User.find() is completed. This is one of the gotchas of asynchronous programming with Node.js. Callback functions signal when a task is completed, so execution can be done "out of order" in relation to your source code if you don't wrap operations that you want to be sequential within callbacks.
On a side note, in the function registerUser, if user.save fails then the client will never know, since the function sends a 200 status code for any request. This happens for the same reason I mentioned above: because res.sendStatus(200) is not wrapped inside the user.save callback function, it may run before the save operation has completed. If an error occurs during a save, you should tell the client, probably with a 500 status code. For example:
var registerUser = function(req, res, user){
user.save(function(err, user){
if (err) {
console.error(err);
return res.status(500).send(err);
}
return res.sendStatus(201);
});
};
Your call to registerUser() is defined after the route and would be undefined since it's not a hoisted function.
Your use of scope in the closures isn't correct. For your specific error, it's because you're running res.send() in a loop when it's only supposed to be called once per request (hence already sent headers a.k.a. response already sent). You should be returning from the function directly after calling res.send() as well.
I'm using MEAN stack from meanjs and have this routes:
// Teams Routes
app.route('/teams')
.get(teams.list)
.post(users.requiresLogin, teams.create);
app.route('/teams/:teamId')
.get(teams.read)
.put(users.requiresLogin, teams.update)
.delete(users.requiresLogin, teams.delete);
app.route('/teams/:teamId/participants')
.get(teams.teamParticipants);
// Finish by binding the Team middleware
app.param('teamId', teams.teamByID);
The issue here is, whenever I'm accessing a resource with this path:
[GET]
http://localhost:3000/teams/547dd53b964b3514294d2dfe/participants
it always return a 404 status. When the request reaches the server, it's accessing
teams.teamByID
from param but wasn't been able to proceed to:
teams.teamParticipants
What I wanna know if there's something I'm doing wrong when it comes to defining my routes, and if there's any better way of defining routes.
Thank you in advance.
EDITS
#mscdex
Here's my teamByID
exports.teamByID = function(req, res, next, id) {
Team.findById(id).exec(function(err, team) {
if (err) return next(err);
if (! team) return next(new Error('Failed to load Team ' + id));
req.team = team ;
next();
});
};
I found the problem here. I dig into express' code and checked how it handle its routes.
Express handles the routes callbacks based on the number of arguments the function has.
If the function for the route has four(4), like the one I have:
exports.teamParticipants = function(req, res, next, id) {
Participant.find({team: id}, function(err, participants){
if (err) return next(err);
if (! participants) return next(new Error('Failed to load Participants from Team ' + id));
res.jsonp(participants);
next();
});
};
It would use its 'handle_error' of its Layer class, passing four arguments: error, req, res, and next.
If the route has less than 4 arguments, it would use 'handle_request' method of it Layer class, passing 3 main arguments: req, res, next. So correcting my 'teamParticipants' method, I should have this kind of implementation for it to work:
exports.teamParticipants = function(req, res) {
Participant.find({team: req.team._id}, function(err, participants){
if (err){
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else {
res.jsonp(participants);
}
});
};
So far, the issue here was Express handles param and route differently. I thought that param and route passed the same arguments.
param handler has this signature: param(req, res, callback, value, key)
which is different from routes
route's handler signatures:
route(req, res, next)
route(err, req, res, next)
I've been using this npm module, expresspath. It separates your controllers/middlewares. :)