Gracefully shutdown in node.js/express with async jobs - node.js

Question

I am trying to make gracefully shutdown in node.js using express 4.x http server.
Closing express server is easy but what worries me is that we have a lot of async jobs. Example of the flow :
Receive request
Do some stuff
Send response back to the client
In background continue to do some async stuff related to that request like making another request to some third part service
Receive response from third part service and save response to database etc.
Finish
So if I make my gracefully shutdown code like this :
process.on('SIGTERM', function () {
serverInstance.close(function(){
closeConnectionToDatabases(function(){
process.exit(0);
})
});
// shutdown anyway after some time
setTimeout(function(){
process.exit(0);
}, 8000);
});
How can I be sure that everything goes ok if SIGTERM has happened between first and second step in flow explained above? What about fourth, fifth and sixth step? Is there any nice way to handle this or it is just about manually watch to all requests going from your service in async way and wait for them?
Thanks,
Ivan

Answer 1

process.exit() will terminate node.js process immediately, without waiting for all aync tasks to finish.
If you want a truly graceful shutdown, you should close all resources (all open connections and file descriptors) manually, instead of calling process.exit(). In this case node.js will terminate immediately after finishing all pending async tasks.
So, just close all database connections and other i/o resources:
process.on('SIGTERM', function () {
serverInstance.close(function(){
closeConnectionToDatabases(function(){
// now node.js should close automatically
})
});
// shutdown anyway after some time
setTimeout(function(){
process.exit(0);
}, 8000);
});

Answer 2

Here's a better option:
Instead of using setTimeout to wait for all the async jobs to finish, we can create a new Promise, and resolve it when the async jobs are completed. Then after the promise is resolved, we can exit the process.
process.on('SIGTERM', () => {
new Promise((resolve) => {
serverInstance.close(async () => {
await closeConnectionToDatabases();
resolve();
});
}).then(() => process.exit(0))
});
Hope it helps you :)