I am trying to understand a case I am running into with my Node/Express application.
Let's say I have the following which is a function called when a specific route is hit:
async function routeThatDoesStuff(req, res, next) {
doAsyncStuff()
res.status(200).json({ message: 'Completed' })
}
In this case doAsyncStuff() does database operations on resources that are not critical to send back to the user so theoretically we don't need to await it. However, it seems that this operation does not actually complete unless I put the await in front.
My guess is that this has to do with the event loop etc in Node. That potentially because the route is completing before the doAsyncStuff() completes, the function doesn't actually complete because Node terminated it prematurely.
My big question is how Node handles async children function execution when parent functions have already completed?
An example to show that this should work:
const sleep = (milliseconds, cb) => {
return new Promise(resolve => setTimeout(() => {
resolve(cb())
}, milliseconds))
}
async function routeThatDoesStuff(req, res, next) {
sleep(5000, () => console.log('done'))
res.status(200).json({ message: 'Completed' })
}
JSON response is shown immediately, callback is executed after 5 seconds even though the response has been sent.
I'm running a simple Fastify server and when I make a request and it fails, I wanna handle that exception in the setErrorHandler.
How can I achieve that?
This doesn't seem to work:
import fastify from 'fastify';
import fetch from 'node-fetch';
const app = fastify();
app.setErrorHandler(async (error, request, response) => {
// I want this to be called whenever there is an error
console.log('setErrorHandler');
return 'error';
});
app.get('/', (request, response) => {
// I do not want to use async / await or .catch here
fetch('https://...').then(() => { response.send(); });
});
I get the error UnhandledPromiseRejectionWarning and the server never sends a response.
I do NOT want to use async, await or .catch on the promise. The reason is I'm simulating a developer error. So if someone forgets to add a .catch and is not using async / await, I still wanna "catch" that error and return status 500 to the client.
I'm willing to change / wrap the request library adding a .catch in it, I just don't wanna change that endpoint handler code in general, since it's sort of out of my control (another developer might code it any way they want).
The reason is I'm simulating a developer error.
You can't manage these errors at runtime to reply to a request because you don't reference the reply object to act accordingly.
You can catch those errors with, but the reply cannot be fulfilled:
process.on('unhandledRejection', (err) => {
console.log(err)
})
As an alternative, I would set connectionTimeout, so the client will get a timeout error at least.
As you wrote, you already know that you should change the handler code to let Fastify be aware of the promise:
// add return
app.get('/', (request, response) => {
return fetch('https://...').then(() => { response.send(); });
})
For this reason, I think the solution to your problem should be taken offline adding a linter rule (like so) and integrate some Static Code Analysis in the CI pipeline to reject bad code.
I want to be able to exit execution of a post route when an event is sent from the client-side. I'm using socket.io but I'm not sure it can do what I want. I am using the uploads route to process a file, but if the user deletes the file, I want the app.post execution to end, similar to either a res.end() or return statement.
My app in the front-end receives a file from the user and immediately is sent to the post route for processing. If the user deletes the file and uploads a new one, the previous post route is still going. I want to make sure the previous one was terminated, cancelled, etc.
I'm currently using socket.io to communicate front-end to back-end.
How can I achieve this?
app.post('/uploads', async (req, res) => {
// async func1
// async func2
// if we receive an event from the front end while processing here, how can I exit the post route?
// async func3
});
You can add UUID for each request you make and return it to the front-end. The request will be resolved with the 202 ACCEPTED status code meaning the request was accepted and being handled but the HTTP request will be resolved.
Now you can implement a resourceManagerServeic that will allow APIs (http or ws) to change the state of a resource (like canceling it).
app.post('/uploads', async (req, res) => {
const resourceUuid = resourceManagerServeic.createResource();
res.status(202); // ACCEPTED
res.send({ uuid: resourceUuid });
// start besnise logic
await function1();
if(resourceManagerServeic.isCanceled(resourceUuid)) {
// cleanup
return; // stop request handling
}
await function2();
if(resourceManagerServeic.isCanceled(resourceUuid)) {
// cleanup
return; // stop request handling
}
await function3();
if(resourceManagerServeic.isCanceled(resourceUuid)) {
// cleanup
return; // stop request handling
}
});
app.del('/uploads/:resourceUuid', async (req, res) => {
resourceManagerServeic.cancle(req.params.resourceUuid);
res.end() // handle response
});
I guess that your are using Express. Take a look at express-async-handler
You can invoke it
const asyncHandler = require('express-async-handler')
app.post('/upload', asyncHandler(async(req, res) => {
await firstfunc()
await secondfunc()
}))
I'm starting out w/ NodeJS and Express. Coming from the other popular scripting languages and C++ background, asynchronously calling DB functions is a bit foreign. I've sorted out a pattern, but I'm still curious about catching exceptions. Below is my basic pattern.
var callback = function(req, res) {
// do stuff
connection.query(queryString, function(err,result){
if (err) throw err;
// process results.
};
};
var express = require('express');
var app = express();
app.get('/', callback);
app.listen(3000,function() {
console.log('listening');
};
Generally I have a lot of endpoints and callbacks. I'm a bit lost on where I set up ta try/catch block to catch errors thrown in the callback though. I've looked around for some suggestions, but a lot of them seem to be on the web framework (if any) being used.
When you throw in an asynchronous callback, the exception just goes back to the internals of the database event handler and there is NO way for you to ever catch or handle that exception. So, basically it does no good at all. It will just cause you to abort the handling of that request and you will never send a response on that request.
Basically, you have several choices for how to handle the error. You can handle it completely right in each endpoint and send some sort of error response.
Send Response right at each point of error
app.get('/', function(req, res) {
// do stuff
connection.query(queryString, function(err,result){
if (err) return res.status(500).send(someErrorResponse);
// process results.
};
});
Forward on to centralized error handler
Or, you can forward the error on to a centralized error handler by calling next(err):
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString, function(err,result){
// if error, forward it on to our centralized error handler
if (err) return next(err);
// process results.
};
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
See Nodejs handle unsupported URLs and request types for more info on the ways to have generalized error handlers in Express.
Use promises to collect errors within each route
If you are using more involved sequences of asynchronous operations where you may have more than one async operation sequenced together, then it does get to be a pain to handle errors at every single async operation. This is where using promises with all your async operations more easily allows all the errors to percolate up to one .catch() statement at the top level of each route. You don't say what database you're using, but here's an idea what that looks like. The general idea is that you can write your code so that all promise rejections (e.g. errors) will propagate up to one central .catch() in each route handler and you can then call next(err) from that .catch(), sending the error to your centralized error handler. Here's how that looks for a recent version of Mongoose (you didn't say which database you were using) with one database operation.
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString).exec().then(function(result){
// process results.
}).catch(next);
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
And, here's what it looks like if you have more than one operation:
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString).exec().then(function(result){
// process results, then make another query
// return the promise from this second operaton so both results
// and error are chained to the first promise
return connection.query(...).exec();
}).then(function(result) {
// process chained result
}).catch(next);
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
Since ES6 built in support for promises and ES7 will add support for async/await for asynchronous operations (which is based on promises) and all significant libraries that offer asynchronous operations have added or are adding support for promises, it is clear that promises are the future of the language for managing asynchronous operations. That would be my strong recommendation.
You should never, ever throw an error like that! :) The reason is that at some point your whole node app will just stop working, because of some db query failed. This should be handled instead of just die.
And because this is a route handler - handles specific url that the user is getting (for example /), there should be some output. You can always show a page with status 500 and a nice design, if there was such an error that you cannot handle or you might have your internal state messed up.
So basically just act as nothing happened - return respones of any kind, or even render a page, but provide information that something went wrong.
Also, a common scenario is something like what Alon Oz presented. All routes in express are actually a middleware functions, that are called one after another. If the route does not match the requested one, the function just skips and the next one is called. You can manually control that. The actual pattern of the router is this:
app.get('/', function(req, res, next) {
// you can have the request
// you can send response like res.send('hello')
// OR you can skip this function using NEXT
});
The actual signature of next is next(err). So if you call it without any arguments, it will just skip to the next middleware. If you call it with an argument, it will skip all regular functions and go to the last ones in the stack, or more specifically the ones that handle errors. They are like the regular ones, but taking four arguments instead of three:
app.use(function (err, req, res, next) { });
It's very important to understand that this function will be called if you call next with an argument. Throwing an error won't do any good! Of course if none of your routes match the specific criteria (url) the final one will in the call will be called, so you can still handle the "not found" error.
This is a common scenario that you will use:
// development error handler, will print stacktrace
if (app.get('env') === 'development') {
app.use(function(err, req, res, next) {
debug('ERROR [ip: %s]:: dev env -> ', req.ip, err); // I'm using debug library - very helpful
res.status(err.status || 500);
res.render('deverr', { // I render custom template with the whole stack beautifully displayed
errMessage: err.message,
error: err
});
});
}
// production error handler, no stacktraces leaked to user
app.use(function(err, req, res, next) {
res.status(err.status || 500);
res.render('pages/error', { // custom error page with nice design and a message
errMessage: err.message,
error: {}
});
});
Hope that helps! :)
Since you are using express, it has its own way to handle exceptions,
defined like this:
function clientErrorHandler (err, req, res, next) {
if (req.xhr) {
res.status(500).send({ error: 'Something failed!' })
} else {
next(err)
}
}
app.use(clientErrorHandler)
For more info:
https://expressjs.com/en/guide/error-handling.html
There are most commonly three major types of errors that we need to take into account.
Promise failures (Any failures that come up during async/await or result of a promise in then/catch)
In order to handle promise failures, as suggested in the strong loop document or node js 2018 best practices, it's important to have a common function that can handle it.
// app.js file
app.get('/:id', async (req,res,next) => {
if(!req.params.id) {
return res.status(412).send('enter a valid user id');
}
try {
const results = await UserDAL(id);
} catch(e) {
next(e);
}
}
// common error middleware defined in middleware/error.js
module.exports = function (err,req,res,next) {
logger.error(`${err.status || 500} - ${err.message} - ${req.originalUrl} - ${req.method} - ${req.ip}`);
return res.status(500).send('something failed.');
};
Unhandled Rejections
process.on('unhandledRejection', e => {
// do something
});
Unhandled exceptions
process.on('uncaughtException', e => {
// do something
});
If you see a lot of try/ catch blocks in your express methods you can abstract that to a separate async function like below:
module.exports = function asyncMiddleWare(handler) {
return async (req,res,next) => {
try {
await handler(req,res)
} catch(e) {
next(e);
}
}
};
I have some error handling in my Express app, for async/await functionality, ie. trying to handle the uncaught errors centrally, to respond with appropriate status codes/messages.
I'm doing like so:
const handleRejection = (res, reason) => {
const { code, message } = reason
console.trace()
logger.error(reason)
// use `res` to send response to client
}
app.use((req, res, next) => {
process.on('unhandledRejection', handleRejection.bind(this, res))
next()
})
process.on('SIGTERM', () => process.removeListener('unhandledRejection', handleRejection))
This works fine for catching/handling errors, however, my logs are filled up each time an error is triggered. I don't believe this event listener, process.on('unhandledRejection'), is being removed correctly at all...
Is there a solution for this?
It seems that you're attaching a new event handler on every request.
Then, on SIGTERM you try to remove an event handler handleRejection which was never attached - you didn't attach handleRejection but handleRejection.bind(this, res) which returns a different function.
It looks like you may also be leaking memory by binding the function to every res object for every request.
This is a very strange way to handle errors. I'm not even sure that this is indeed what you are trying to do. Do you want to add so many event hendlers (one for every request made to your server) and then remove all of them on SIGTERM, when you try to exit your server?
Here is my solution for an Express middleware to pass unhandledRejection to main error middleware
// Handle unhandledRejection and pass error to next middleware
app.use(function (req, res, next) {
function unhandledRejection(reason, p) {
console.error('Possibly Unhandled Rejection at: Promise ', p, " reason: ", reason);
next(reason);
}
process.on('unhandledRejection', unhandledRejection);
// Manage to get information from the response too, just like Connect.logger does:
var end = res.end;
res.end = function (chunk, encoding) {
// Prevent MaxListener on process.events
process.removeListener('unhandledRejection', unhandledRejection);
res.end = end;
res.end(chunk, encoding);
};
next();
});