Related
In my current FOSS Discord bot project I have this log.ts file which handles the logging for the bot.
It creates multiple fs.WriteStream objects, which write to each log file. There is a section in the code when await log('CLOSE_STREAMS') is called to run the WriteStream#close() functions on each WriteStream, returning a promise. This is used in the process.on('exit') handler to save the log files before we close.
The problem here is that the 'exit' event handler can not schedule any additional work into the event queue.
How could I handle calling the CLOSE_STREAMS in a way where I can run this exit handler as I am expecting?
Function Implementation, simplified
log.ts log('CLOSE_STREAMS')
// Main
export default function log(mode: 'CLOSE_STREAMS'): Promise<void>;
export default function log(mode: 'v' | 'i' | 'w' | 'e', message: any, _bypassStackPrint?: boolean): void;
// eslint-disable-next-line #typescript-eslint/explicit-module-boundary-types
export default function log(mode: 'v' | 'i' | 'w' | 'e' | 'CLOSE_STREAMS', message?: any, _bypassStackPrint = false): void | Promise<void> {
if (mode === 'CLOSE_STREAMS')
// Close all of the file streams
return new Promise((resolve) => {
errStr.end(() => {
warnStr.end(() => {
allStr.end(() => {
resolve();
});
});
});
});
else {
index.ts
This is the way the log is killed in the uncaught exceptions; this would be how I want to do it for the exit event.
// If we get an uncaught exception, close ASAP.
process.on('uncaughtException', async (error) => {
log('e', 'Killing client...', true);
client.destroy();
log('e', 'Client killed.', true);
log('e', 'Closing databases...', true);
client.closeDatabases();
log('e', 'Closed databases.', true);
log('e', 'An uncaught exception occured!', true);
log('e', `Error thrown was:`, true);
error.stack?.split('\n').forEach((item) => {
log('e', `${item}`, true);
});
log('e', 'Stack trace dump:', true);
let stack = new Error().stack?.split('\n');
stack?.shift();
if (!stack) stack = [];
stack.forEach((item) => {
log('e', `${item}`, true);
});
log('e', 'Process exiting.', true);
log('e', 'Exit code 5.', true);
log('e', 'Goodbye!', true);
await log('CLOSE_STREAMS'); // <<<<<<<<<<<< HERE
process.exit(5);
});
As you know, you can't reliably use asynchronous operations in the processing of an exit event because the process will exit before they complete. As such, I don't think there's any way to do this reliably with a nodejs stream. Streams have a completely asynchronous API and implementation, including flushing and closing.
In a few Google searches I found a few other people asking for the same thing for the same reasons and there was no solution offered. As best I know, these are the options:
Do some hacking on the stream object to add a synchronous flushAndClose() method to your stream. You'd have to work into the internals of the stream to get the buffer and the file handle and do your own synchronous write of any remaining buffer and then do a synchronous close on the file handle. Note, even this has a problem case if there's currently an asynchronous write operation in process.
Abandon the built-in stream and just implement your own lightweight logfile interface that makes it easy for you to have both asynchronous writing (for normal use) and a synchronous flushAndClose() operation for emergency shut-down. Note, even this has a problem case if there's currently an asynchronous write operation in process when you want to do the synchronous close.
Rather than using process.on('exit', ...) to trigger closing of the log files, go up one level higher in the chain. Whatever it is that triggers closing of your app, put it in an async function that will wait for the log files to be properly closed before calling process.exit() so you get the log files closed when you still have the ability to handle asynchronous operations.
Do logging from a different (more stable) process. This process can then message that logging process what it wants logged and that process can manage getting the logging info safely to disk independent of whether the source process shuts down abruptly or not.
Note: Exiting a process will automatically close any open file selectors (the OS takes care of that for you). So, as long as this is an edge case shut-down in some fatal error condition, not a common normal shut-down, then perhaps you don't have a big problem here to really solve.
The worst that could happen is that you might lose some very recently logged lines of data if they hadn't yet been flushed from the stream. Note that streams write data immediately to their descriptor when possible so they don't generally accumulate lots of buffered data. The time when they do buffer data is when a new write to the stream happens, but the previous write is still in operation. Then the data to be written gets buffered until the prior write operation completes. So, data is never left sitting in the buffer with an idle stream. This tends to minimize (though not eliminate) data loss on an immediate shut-down.
If this is a normal, regular shut-down, then you should be able to use option #3 above and reshape how the shut-down occurs so you can use asynchronous code where you want so you can properly shutdown the streams.
I discovered an odd behaviour in node's promisify() function and I cannot work out why it's doing what it's doing.
Consider the following script:
#!/usr/bin/env node
/**
* Module dependencies.
*/
var http = require('http')
var promisify = require('util').promisify
;(async () => {
try {
// UNCOMMENT THIS, AND NODE WILL QUIT
// var f = function () { return 'Straight value' }
// var fP = promisify(f)
// await fP()
/**
* Create HTTP server.
*/
var server = http.createServer()
/**
* Listen on provided port, on all network interfaces.
*/
server.listen(3000)
server.on('error', (e) => { console.log('Error:', e); process.exit() })
server.on('listening', () => { console.log('Listening') })
} catch (e) {
console.log('ERROR:', e)
}
})()
console.log('OUT OF THE ASYNC FUNCTION')
It's a straightforward self-invoking function that starts a node server.
And that's fine.
NOW... if you uncomment the lines under "UNCOMMENT THIS", node will quit without running the server.
I KNOW that I am using promisify() on a function that does not call the callback, but returns a value instead. So, I KNOW that that is in itself a problem.
However... why is node just quitting...?
This was really difficult to debug -- especially when you have something more complex that a tiny script.
If you change the function definition to something that actually calls a callback:
var f = function (cb) { setTimeout( () => { return cb( null, 'Straight value') }, 2000) }
Everything works as expected...
UPDATE
Huge simplification:
function f () {
return new Promise(resolve => {
console.log('AH')
})
}
f().then(() => {
console.log('Will this happen...?')
})
Will only print "AH"!
Call promisify() on a non-callback function: “interesting” results in node. Why?
Because you allow node.js to go to the event loop with nothing to do. Since there are no live asynchronous operations in play and no more code to run, node.js realizes that there is nothing else to do and no way for anything else to run so it exits.
When you hit the await and node.js goes back to the event loop, there is nothing keeping node.js running so it exits. There are no timers or open sockets or any of those types of things that keep node.js running so the node.js auto-exit-detection logic says that there's nothing else to do so it exits.
Because node.js is an event driven system, if your code returns back to the event loop and there are no asynchronous operations of any kind in flight (open sockets, listening servers, timers, file I/O operations, other hardware listeners, etc...), then there is nothing running that could ever insert any events in the event queue and the queue is currently empty. As such, node.js realizes that there can never be any way to run any more code in this app so it exits. This is an automatic behavior built into node.js.
A real async operation inside of fp() would have some sort of socket or timer or something open that keeps the process running. But because yours is fake, there's nothing there and nothing to keep node.js running.
If you put a setTimeout() for 1 second inside of f(), you will see that the process exit happens 1 second later. So, the process exit has nothing to do with the promise. It has to do with the fact that you've gone back to the event loop, but you haven't started anything yet that would keep node.js running.
Or, if you put a setInterval() at the top of your async function, you will similarly find that the process does not exit.
So, this would similarly happen if you did this:
var f = function () { return 'Straight value' }
var fP = promisify(f);
fP().then(() => {
// start your server here
});
Or this:
function f() {
return new Promise(resolve => {
// do nothing here
});
}
f().then(() => {
// start your server here
});
The issue isn't with the promisify() operation. It's because you are waiting on a non-existent async operation and thus node.js has nothing to do and it notices there's nothing to do so it auto-exits. Having an open promise with a .then() handler is not something that keeps node.js running. Rather there needs to be some active asynchronous operation (timer, network socket, listening server, file I/O operation underway, etc...) to keep node.js running.
In this particular case, node.js is essentially correct. Your promise will never resolve, nothing else is queued to ever run and thus your server will never get started and no other code in your app will ever run, thus it is not actually useful to keep running. There is nothing to do and no way for your code to actually do anything else.
If you change the function definition to something that actually calls a callback:
That's because you used a timer so node.js has something to actually do while waiting for the promise to resolve. A running timer that has not had .unref() called on it will prevent auto-exit.
Worth reading: How does a node.js process know when to stop?
FYI, you can "turn off" or "bypass" the node.js auto-exit logic by just adding this anywhere in your startup code:
// timer that fires once per day
let foreverInterval = setInterval(() => {
// do nothing
}, 1000 * 60 * 60 * 24);
That always gives node.js something to do so it will never auto-exit. Then when you do want your process to exit, you could either call clearInterval(foreverInterval) or just force things with process.exit(0).
I've been trying to perform an asynchronous operation before my process is terminated.
Saying 'terminated' I mean every possibility of termination:
ctrl+c
Uncaught exception
Crashes
End of code
Anything..
To my knowledge the exit event does that but for synchronous operations.
Reading the Nodejs docs i found the beforeExit event is for the async operations BUT :
The 'beforeExit' event is not emitted for conditions causing explicit termination, such as calling process.exit() or uncaught exceptions.
The 'beforeExit' should not be used as an alternative to the 'exit' event unless the intention is to schedule additional work.
Any suggestions?
You can trap the signals and perform your async task before exiting. Something like this will call terminator() function before exiting (even javascript error in the code):
process.on('exit', function () {
// Do some cleanup such as close db
if (db) {
db.close();
}
});
// catching signals and do something before exit
['SIGHUP', 'SIGINT', 'SIGQUIT', 'SIGILL', 'SIGTRAP', 'SIGABRT',
'SIGBUS', 'SIGFPE', 'SIGUSR1', 'SIGSEGV', 'SIGUSR2', 'SIGTERM'
].forEach(function (sig) {
process.on(sig, function () {
terminator(sig);
console.log('signal: ' + sig);
});
});
function terminator(sig) {
if (typeof sig === "string") {
// call your async task here and then call process.exit() after async task is done
myAsyncTaskBeforeExit(function() {
console.log('Received %s - terminating server app ...', sig);
process.exit(1);
});
}
console.log('Node server stopped.');
}
Add detail requested in comment:
Signals explained from node's documentation, this link refers to standard POSIX signal names
The signals should be string. However, I've seen others have done the check so there might be some other unexpected signals that I don't know about. Just want to make sure before calling process.exit(). I figure it doesn't take much time to do the check anyway.
for db.close(), I guess it depends on the driver you are using. Whether it's sync of async. Even if it's async, and you don't need to do anything after db closed, then it should be fine because async db.close() just emits close event and the event loop would continue to process it whether your server exited or not.
Using beforeExit hook
The 'beforeExit' event is emitted when Node.js empties its event loop and has no additional work to schedule. Normally, the Node.js process will exit when there is no work scheduled, but a listener registered on the 'beforeExit' event can make asynchronous calls, and thereby cause the Node.js process to continue.
process.on('beforeExit', async () => {
await something()
process.exit(0) // if you don't close yourself this will run forever
});
Here's my take on this. A bit long to post as a code snippet in here, so sharing a Github gist.
https://gist.github.com/nfantone/1eaa803772025df69d07f4dbf5df7e58
It's pretty straightforward. You use it like so:
'use strict';
const beforeShutdown = require('./before-shutdown');
// Register shutdown callbacks: they will be executed in the order they were provided
beforeShutdown(() => db.close());
beforeShutdown(() => server.close());
beforeShutdown(/* Do any async cleanup */);
The above will listen for a certain set of system signals (SIGINT, a.k.a Ctrl + C, and SIGTERM by default) and call each handler in order before shutting down the whole process.
It also,
Supports async callbacks (or returning a Promise).
Warns about failing shutdown handlers, but prevents the error/rejection from bubbling up.
Forces shutdown if handlers do not return after some time (15 seconds, by default).
Callbacks can be registered from any module in your code base.
Combining answers + handling for uncaught exceptions and promise rejections
async function exitHandler(evtOrExitCodeOrError: number | string | Error) {
try {
// await async code here
// Optionally: Handle evtOrExitCodeOrError here
} catch (e) {
console.error('EXIT HANDLER ERROR', e);
}
process.exit(isNaN(+evtOrExitCodeOrError) ? 1 : +evtOrExitCodeOrError);
}
[
'beforeExit', 'uncaughtException', 'unhandledRejection',
'SIGHUP', 'SIGINT', 'SIGQUIT', 'SIGILL', 'SIGTRAP',
'SIGABRT','SIGBUS', 'SIGFPE', 'SIGUSR1', 'SIGSEGV',
'SIGUSR2', 'SIGTERM',
].forEach(evt => process.on(evt, exitHandler));
You need to trap beforeExit, uncaughtException, and each of the listenable signals that can terminate the process: SIGHUP, SIGINT, SIGTERM, SIGUSR2 (Nodemon), and SIGBREAK (Windows). (You likely don't want to clean up on SIGQUIT, since that signal is used for core dumps.) After your async operations have completed, you then need to explicitly terminate the process using the appropriate mechanism, such as process.exit or process.kill. For SIGINT in particular, it's important to propagate the signal to the parent process (i.e. using process.kill as opposed to process.exit). Also note that you need to stop trapping the signals before calling process.kill. Because this is all relatively tricky, I published a library, async-cleanup, to make adding async exit hooks as easy as function call:
import { addCleanupListener } from "async-cleanup";
import { unlink, writeFile } from "fs/promises";
await writeFile(".lockfile", String(process.pid), { flag: "wx" });
addCleanupListener(async () => {
await unlink(".lockfile");
});
I want to tell Node.js to always do something just before it exits, for whatever reason — Ctrl+C, an exception, or any other reason.
I tried this:
process.on('exit', function (){
console.log('Goodbye!');
});
I started the process, killed it, and nothing happened. I started it again, pressed Ctrl+C, and still nothing happened...
UPDATE:
You can register a handler for `process.on('exit')` and in any other case(`SIGINT` or unhandled exception) to call `process.exit()`
process.stdin.resume();//so the program will not close instantly
function exitHandler(options, exitCode) {
if (options.cleanup) console.log('clean');
if (exitCode || exitCode === 0) console.log(exitCode);
if (options.exit) process.exit();
}
//do something when app is closing
process.on('exit', exitHandler.bind(null,{cleanup:true}));
//catches ctrl+c event
process.on('SIGINT', exitHandler.bind(null, {exit:true}));
// catches "kill pid" (for example: nodemon restart)
process.on('SIGUSR1', exitHandler.bind(null, {exit:true}));
process.on('SIGUSR2', exitHandler.bind(null, {exit:true}));
//catches uncaught exceptions
process.on('uncaughtException', exitHandler.bind(null, {exit:true}));
This only works if you call synchronous code inside the handler, otherwise it will call the handler indefinitely
The script below allows having a single handler for all exit conditions. It uses an app specific callback function to perform custom cleanup code.
cleanup.js
// Object to capture process exits and call app specific cleanup function
function noOp() {};
exports.Cleanup = function Cleanup(callback) {
// attach user callback to the process event emitter
// if no callback, it will still exit gracefully on Ctrl-C
callback = callback || noOp;
process.on('cleanup',callback);
// do app specific cleaning before exiting
process.on('exit', function () {
process.emit('cleanup');
});
// catch ctrl+c event and exit normally
process.on('SIGINT', function () {
console.log('Ctrl-C...');
process.exit(2);
});
//catch uncaught exceptions, trace, then exit normally
process.on('uncaughtException', function(e) {
console.log('Uncaught Exception...');
console.log(e.stack);
process.exit(99);
});
};
This code intercepts uncaught exceptions, Ctrl+C and normal exit events. It then calls a single optional user cleanup callback function before exiting, handling all exit conditions with a single object.
The module simply extends the process object instead of defining another event emitter. Without an app specific callback the cleanup defaults to a no op function. This was sufficient for my use where child processes were left running when exiting by Ctrl+C.
You can easily add other exit events such as SIGHUP as desired. Note: per NodeJS manual, SIGKILL cannot have a listener. The test code below demonstrates various ways of using cleanup.js
// test cleanup.js on version 0.10.21
// loads module and registers app specific cleanup callback...
var cleanup = require('./cleanup').Cleanup(myCleanup);
//var cleanup = require('./cleanup').Cleanup(); // will call noOp
// defines app specific callback...
function myCleanup() {
console.log('App specific cleanup code...');
};
// All of the following code is only needed for test demo
// Prevents the program from closing instantly
process.stdin.resume();
// Emits an uncaught exception when called because module does not exist
function error() {
console.log('error');
var x = require('');
};
// Try each of the following one at a time:
// Uncomment the next line to test exiting on an uncaught exception
//setTimeout(error,2000);
// Uncomment the next line to test exiting normally
//setTimeout(function(){process.exit(3)}, 2000);
// Type Ctrl-C to test forced exit
This catches every exit event I can find that can be handled. Seems quite reliable and clean so far.
[`exit`, `SIGINT`, `SIGUSR1`, `SIGUSR2`, `uncaughtException`, `SIGTERM`].forEach((eventType) => {
process.on(eventType, cleanUpServer.bind(null, eventType));
})
"exit" is an event that gets triggered when node finish it's event loop internally, it's not triggered when you terminate the process externally.
What you're looking for is executing something on a SIGINT.
The docs at http://nodejs.org/api/process.html#process_signal_events give an example:
Example of listening for SIGINT:
// Start reading from stdin so we don't exit.
process.stdin.resume();
process.on('SIGINT', function () {
console.log('Got SIGINT. Press Control-D to exit.');
});
Note: this seems to interrupt the sigint and you would need to call process.exit() when you finish with your code.
function fnAsyncTest(callback) {
require('fs').writeFile('async.txt', 'bye!', callback);
}
function fnSyncTest() {
for (var i = 0; i < 10; i++) {}
}
function killProcess() {
if (process.exitTimeoutId) {
return;
}
process.exitTimeoutId = setTimeout(() => process.exit, 5000);
console.log('process will exit in 5 seconds');
fnAsyncTest(function() {
console.log('async op. done', arguments);
});
if (!fnSyncTest()) {
console.log('sync op. done');
}
}
// https://nodejs.org/api/process.html#process_signal_events
process.on('SIGTERM', killProcess);
process.on('SIGINT', killProcess);
process.on('uncaughtException', function(e) {
console.log('[uncaughtException] app will be terminated: ', e.stack);
killProcess();
/**
* #https://nodejs.org/api/process.html#process_event_uncaughtexception
*
* 'uncaughtException' should be used to perform synchronous cleanup before shutting down the process.
* It is not safe to resume normal operation after 'uncaughtException'.
* If you do use it, restart your application after every unhandled exception!
*
* You have been warned.
*/
});
console.log('App is running...');
console.log('Try to press CTRL+C or SIGNAL the process with PID: ', process.pid);
process.stdin.resume();
// just for testing
Just wanted to mention death package here: https://github.com/jprichardson/node-death
Example:
var ON_DEATH = require('death')({uncaughtException: true}); //this is intentionally ugly
ON_DEATH(function(signal, err) {
//clean up code here
})
async-exit-hook seems to be the most up-to-date solution for handling this problem. It's a forked/re-written version of exit-hook that supports async code before exiting.
I need to do an asynchronous cleanup action on exit, none of the answers in this question worked for me.
So I tried it myself, and finally found this:
process.once('uncaughtException', async () => {
await cleanup()
process.exit(0)
})
process.once('SIGINT', () => { throw new Error() })
After playing around with other answer, here is my solution for this task. Implementing this way helps me centralize cleanup in one place, preventing double handling the cleanup.
I would like to route all other exiting codes to 'exit' code.
const others = [`SIGINT`, `SIGUSR1`, `SIGUSR2`, `uncaughtException`, `SIGTERM`]
others.forEach((eventType) => {
process.on(eventType, exitRouter.bind(null, { exit: true }));
})
What the exitRouter does is calling process.exit()
function exitRouter(options, exitCode) {
if (exitCode || exitCode === 0) console.log(`ExitCode ${exitCode}`);
if (options.exit) process.exit();
}
On 'exit', handle the clean up with a new function
function exitHandler(exitCode) {
console.log(`ExitCode ${exitCode}`);
console.log('Exiting finally...')
}
process.on('exit', exitHandler)
For the demo purpose, this is link to my gist. In the file, i add a setTimeout to fake the process running.
If you run node node-exit-demo.js and do nothing, then after 2 seconds, you see the log:
The service is finish after a while.
ExitCode 0
Exiting finally...
Else if before the service finish, you terminate by ctrl+C, you'll see:
^CExitCode SIGINT
ExitCode 0
Exiting finally...
What happened is the Node process exited initially with code SIGINT, then it routes to process.exit() and finally exited with exit code 0.
io.js has an exit and a beforeExit event, which do what you want.
In the case where the process was spawned by another node process, like:
var child = spawn('gulp', ['watch'], {
stdio: 'inherit',
});
And you try to kill it later, via:
child.kill();
This is how you handle the event [on the child]:
process.on('SIGTERM', function() {
console.log('Goodbye!');
});
Here's a nice hack for windows
process.on('exit', async () => {
require('fs').writeFileSync('./tmp.js', 'crash', 'utf-8')
});
I am working on a websocket oriented node.js server using Socket.IO. I noticed a bug where certain browsers aren't following the correct connect procedure to the server, and the code isn't written to gracefully handle it, and in short, it calls a method to an object that was never set up, thus killing the server due to an error.
My concern isn't with the bug in particular, but the fact that when such errors occur, the entire server goes down. Is there anything I can do on a global level in node to make it so if an error occurs it will simply log a message, perhaps kill the event, but the server process will keep on running?
I don't want other users' connections to go down due to one clever user exploiting an uncaught error in a large included codebase.
You can attach a listener to the uncaughtException event of the process object.
Code taken from the actual Node.js API reference (it's the second item under "process"):
process.on('uncaughtException', function (err) {
console.log('Caught exception: ', err);
});
setTimeout(function () {
console.log('This will still run.');
}, 500);
// Intentionally cause an exception, but don't catch it.
nonexistentFunc();
console.log('This will not run.');
All you've got to do now is to log it or do something with it, in case you know under what circumstances the bug occurs, you should file a bug over at Socket.IO's GitHub page:
https://github.com/LearnBoost/Socket.IO-node/issues
Using uncaughtException is a very bad idea.
The best alternative is to use domains in Node.js 0.8. If you're on an earlier version of Node.js rather use forever to restart your processes or even better use node cluster to spawn multiple worker processes and restart a worker on the event of an uncaughtException.
From: http://nodejs.org/api/process.html#process_event_uncaughtexception
Warning: Using 'uncaughtException' correctly
Note that 'uncaughtException' is a crude mechanism for exception handling intended to be used only as a last resort. The event should not be used as an equivalent to On Error Resume Next. Unhandled exceptions inherently mean that an application is in an undefined state. Attempting to resume application code without properly recovering from the exception can cause additional unforeseen and unpredictable issues.
Exceptions thrown from within the event handler will not be caught. Instead the process will exit with a non-zero exit code and the stack trace will be printed. This is to avoid infinite recursion.
Attempting to resume normally after an uncaught exception can be similar to pulling out of the power cord when upgrading a computer -- nine out of ten times nothing happens - but the 10th time, the system becomes corrupted.
The correct use of 'uncaughtException' is to perform synchronous cleanup of allocated resources (e.g. file descriptors, handles, etc) before shutting down the process. It is not safe to resume normal operation after 'uncaughtException'.
To restart a crashed application in a more reliable way, whether uncaughtException is emitted or not, an external monitor should be employed in a separate process to detect application failures and recover or restart as needed.
I just did a bunch of research on this (see here, here, here, and here) and the answer to your question is that Node will not allow you to write one error handler that will catch every error scenario that could possibly occur in your system.
Some frameworks like express will allow you to catch certain types of errors (when an async method returns an error object), but there are other conditions that you cannot catch with a global error handler. This is a limitation (in my opinion) of Node and possibly inherent to async programming in general.
For example, say you have the following express handler:
app.get("/test", function(req, res, next) {
require("fs").readFile("/some/file", function(err, data) {
if(err)
next(err);
else
res.send("yay");
});
});
Let's say that the file "some/file" does not actually exist. In this case fs.readFile will return an error as the first argument to the callback method. If you check for that and do next(err) when it happens, the default express error handler will take over and do whatever you make it do (e.g. return a 500 to the user). That's a graceful way to handle an error. Of course, if you forget to call next(err), it doesn't work.
So that's the error condition that a global handler can deal with, however consider another case:
app.get("/test", function(req, res, next) {
require("fs").readFile("/some/file", function(err, data) {
if(err)
next(err);
else {
nullObject.someMethod(); //throws a null reference exception
res.send("yay");
}
});
});
In this case, there is a bug if your code that results in you calling a method on a null object. Here an exception will be thrown, it will not be caught by the global error handler, and your node app will terminate. All clients currently executing requests on that service will get suddenly disconnected with no explanation as to why. Ungraceful.
There is currently no global error handler functionality in Node to handle this case. You cannot put a giant try/catch around all your express handlers because by the time your asyn callback executes, those try/catch blocks are no longer in scope. That's just the nature of async code, it breaks the try/catch error handling paradigm.
AFAIK, your only recourse here is to put try/catch blocks around the synchronous parts of your code inside each one of your async callbacks, something like this:
app.get("/test", function(req, res, next) {
require("fs").readFile("/some/file", function(err, data) {
if(err) {
next(err);
}
else {
try {
nullObject.someMethod(); //throws a null reference exception
res.send("yay");
}
catch(e) {
res.send(500);
}
}
});
});
That's going to make for some nasty code, especially once you start getting into nested async calls.
Some people think that what Node does in these cases (that is, die) is the proper thing to do because your system is in an inconsistent state and you have no other option. I disagree with that reasoning but I won't get into a philosophical debate about it. The point is that with Node, your options are lots of little try/catch blocks or hope that your test coverage is good enough so that this doesn't happen. You can put something like upstart or supervisor in place to restart your app when it goes down but that's simply mitigation of the problem, not a solution.
Node.js has a currently unstable feature called domains that appears to address this issue, though I don't know much about it.
I've just put together a class which listens for unhandled exceptions, and when it see's one it:
prints the stack trace to the console
logs it in it's own logfile
emails you the stack trace
restarts the server (or kills it, up to you)
It will require a little tweaking for your application as I haven't made it generic as yet, but it's only a few lines and it might be what you're looking for!
Check it out!
Note: this is over 4 years old at this point, unfinished, and there may now be a better way - I don't know!)
process.on
(
'uncaughtException',
function (err)
{
var stack = err.stack;
var timeout = 1;
// print note to logger
logger.log("SERVER CRASHED!");
// logger.printLastLogs();
logger.log(err, stack);
// save log to timestamped logfile
// var filename = "crash_" + _2.formatDate(new Date()) + ".log";
// logger.log("LOGGING ERROR TO "+filename);
// var fs = require('fs');
// fs.writeFile('logs/'+filename, log);
// email log to developer
if(helper.Config.get('email_on_error') == 'true')
{
logger.log("EMAILING ERROR");
require('./Mailer'); // this is a simple wrapper around nodemailer http://documentup.com/andris9/nodemailer/
helper.Mailer.sendMail("GAMEHUB NODE SERVER CRASHED", stack);
timeout = 10;
}
// Send signal to clients
// logger.log("EMITTING SERVER DOWN CODE");
// helper.IO.emit(SIGNALS.SERVER.DOWN, "The server has crashed unexpectedly. Restarting in 10s..");
// If we exit straight away, the write log and send email operations wont have time to run
setTimeout
(
function()
{
logger.log("KILLING PROCESS");
process.exit();
},
// timeout * 1000
timeout * 100000 // extra time. pm2 auto-restarts on crash...
);
}
);
Had a similar problem. Ivo's answer is good. But how can you catch an error in a loop and continue?
var folder='/anyFolder';
fs.readdir(folder, function(err,files){
for(var i=0; i<files.length; i++){
var stats = fs.statSync(folder+'/'+files[i]);
}
});
Here, fs.statSynch throws an error (against a hidden file in Windows that barfs I don't know why). The error can be caught by the process.on(...) trick, but the loop stops.
I tried adding a handler directly:
var stats = fs.statSync(folder+'/'+files[i]).on('error',function(err){console.log(err);});
This did not work either.
Adding a try/catch around the questionable fs.statSynch() was the best solution for me:
var stats;
try{
stats = fs.statSync(path);
}catch(err){console.log(err);}
This then led to the code fix (making a clean path var from folder and file).
I found PM2 as the best solution for handling node servers, single and multiple instances
One way of doing this would be spinning the child process and communicate with the parent process via 'message' event.
In the child process where the error occurs, catch that with 'uncaughtException' to avoid crashing the application. Mind that Exceptions thrown from within the event handler will not be caught. Once the error is caught safely, send a message like: {finish: false}.
Parent Process would listen to the message event and send the message again to the child process to re-run the function.
Child Process:
// In child.js
// function causing an exception
const errorComputation = function() {
for (let i = 0; i < 50; i ++) {
console.log('i is.......', i);
if (i === 25) {
throw new Error('i = 25');
}
}
process.send({finish: true});
}
// Instead the process will exit with a non-zero exit code and the stack trace will be printed. This is to avoid infinite recursion.
process.on('uncaughtException', err => {
console.log('uncaught exception..',err.message);
process.send({finish: false});
});
// listen to the parent process and run the errorComputation again
process.on('message', () => {
console.log('starting process ...');
errorComputation();
})
Parent Process:
// In parent.js
const { fork } = require('child_process');
const compute = fork('child.js');
// listen onto the child process
compute.on('message', (data) => {
if (!data.finish) {
compute.send('start');
} else {
console.log('Child process finish successfully!')
}
});
// send initial message to start the child process.
compute.send('start');