Error handling principles for Node.js + Express.js applications? - node.js

It seems like error reporting/handling is done differently in Node.js+Express.js applications compared to other frameworks. Am I correct in understanding that it works as follows?
A) Detect errors by receiving them as parameters to your callback functions. For example:
doSomethingAndRunCallback(function(err) {
if(err) { … }
});
B) Report errors in MIDDLEWARE by calling next(err). Example:
handleRequest(req, res, next) {
// An error occurs…
next(err);
}
C) Report errors in ROUTES by throwing the error. Example:
app.get('/home', function(req, res) {
// An error occurs
throw err;
});
D) Handle errors by configuring your own error handler via app.error() or use the generic Connect error handler. Example:
app.error(function(err, req, res, next) {
console.error(err);
res.send('Fail Whale, yo.');
});
Are these four principles the basis for all error handling/reporting in Node.js+Express.js applications?

Error handling in Node.js is generally of the format A). Most callbacks return an error object as the first argument or null.
Express.js uses middleware and the middleware syntax uses B) and E) (mentioned below).
C) is bad practice if you ask me.
app.get('/home', function(req, res) {
// An error occurs
throw err;
});
You can easily rewrite the above as
app.get('/home', function(req, res, next) {
// An error occurs
next(err);
});
Middleware syntax is valid in a get request.
As for D)
(07:26:37 PM) tjholowaychuk: app.error is removed in 3.x
TJ just confirmed that app.error is deprecated in favor of E
E)
app.use(function(err, req, res, next) {
// Only handle `next(err)` calls
});
Any middleware that has a length of 4 (4 arguments) is considered error middleware. When one calls next(err) connect goes and calls error-based middleware.

People at Joyent have published a really insightful best-practices document on this. A must-read article for any Node.js developer.

Why first-parameter?
Because of the asynchronous nature of Node.js, the first-parameter-as-err pattern has become well established as a convention for userland Node.js error handling. This is because asynchronous:
try {
setTimeout(function() {
throw 'something broke' //Some random error
}, 5)
}
catch(e) {
//Will never get caught
}
So instead having the first argument of the callback is pretty much the only sensible way to pass errors asynchronously other than just throwing them.
To do so will result in an unhandled exception which, just in the way it sounds, implies that nothing was done to get the application out of its confused state.
Exceptions, why do they exist
It is worth noting however, that virtually all part of Node.js are event-emitters and the throwing of an exception is a low-level event which can be handled like all events:
//This won't immediately crash if connection fails
var socket = require("net").createConnection(5000);
socket.on("error", function(err) {
console.error("calm down...", err)
});
This can-but-shouldn't be taken to the extreme to catch all errors and make an application which will try very hard to never crash. This is a terrible idea in nearly every use-case, because it will leave the developer without any idea of what's going on in the application state and is analogous to wrapping main in try-catch.
Domains - grouping events logically
As part of dealing with this problem of exceptions making applications fall over, domains allow the developer to take, for example the Express.js application, and try and close off connections sensibly in the event of catastrophic failure.
ES6
It's probably mentioning that this will change again as ES6 allows the generator pattern to create asynchronous events which are still catchable with try/catch blocks.
Koa (written by TJ Holowaychuck, same original author of Express.js) noticeably does this. It uses the ES6 yield statement to create blocks that, while appearing nearly syncronous, are handled in the usual node asynchronous fashion:
app.use(function *(next) {
try {
yield next;
}
catch (err) {
this.status = err.status || 500;
this.body = err.message;
this.app.emit('error', err, this);
}
});
app.use(function *(next) {
throw new Error('some error');
})
This example was shamelessly stolen from here.

Related

What am I doing wrong with this async get request?

I am struggling to figure out what the error is in this async function. I keep getting this error message:
"Unexpected token catch"
and if that error is fixed I get this error Message:
"UnhandledPromiseRejectionWarning"
"DeprecationWarning"
router.get('/Views', async (req, res) => {
console.log("Google auth ", googleAuth)
const organizationId = req.verifiedToken.OrganizationId;
console.log("Got here")
const url = myUrl;
try{
const client = await auth.getClient({
scopes: [
'https://www.googleapis.com/auth/analytics.readonly'
]
})catch(err){
console.log(err);
throw err;
};
const outcome = await client.request({ url })catch(err){
console.log(err);
throw err;
};
};
console.log("Successfully connected!", organizationId);
return responses.success(res, outcome.data);
});
The line
const outcome = await client.request({ url })catch(err){
introduces a catch exception handler without a prior try block. You appear to have caught (no pun intended) this syntax error (though you haven't detailed the code changes to get rid of it).
Unfortunately you haven't posted a self-contained code fragment and you haven't specified which framework/libraries you use on top of node.js. If you are using the Express framework and an ajax library, it might be that your try block is missing an exception handler and the catch statements are meant to be method calls:
router.get('/Views', async (req, res) => {
console.log("Google auth ", googleAuth)
const organizationId = req.verifiedToken.OrganizationId;
console.log("Got here")
const url = myUrl;
try{
const client = await auth.getClient({
scopes: [
'https://www.googleapis.com/auth/analytics.readonly'
]
})
.catch(err){ //*** syntax error in your code sample
console.log(err);
throw err;
};
const outcome = await client.request({
url
})
.catch(err){ //*** syntax error in your code sample
console.log(err);
throw err;
};
} catch (e) { //*** exception handler missing from your original code missing
// exception handling code, possibly empty; there are the .catch calls after all
}
console.log("Successfully connected!", organizationId);
return responses
.success ( res, outcome.data )
.error ( )
//*** This call is missing from your code.
// It is assumed that 'responses' holds a "Promise", see explanation
;
});
A 'Promise' in asynchronous programming is an abstraction of a value not yet known (think of it as a placeholder for that value). There are 2 basic possible scenarios: either that value will eventually be computed (#1) or it is ascertained that it will never be computed at all (#2). A promise library handles these scenarios. The reference API sports promise objects with a .then ( fn_ok, fn_fail ) method taking 2 functions as arguments, one being associated the first scenario, one with the second. As soon as a scenario is established, the respective function will be called. Promise libraries may add additional layers of abstraction, possibly producing the .success/.error calls from the code sample. Note that you promise libraries do usually support 'chaining': In the code sample, the call to .success (.error) would actually make sure that res and outcome.data (nothing) will be preserved and fed to the handler for the scenario #1 (#2) and would return a promise (technically the same object with some properties being redefined).
The second error you have received ( UnhandledPromiseRejectionWarning ) would thus stem from not handling scenario #2. However, the case that the future computation of a value will fail is a very possible outcome and should be accounted for in general. Otherwise your code is amenable to run-time errors that are very hard to track down as you will neither be notified of the code section the error occurs nor (due to async programming) will you have guarantees on the execution order of code fragments. Moreover this coding style easily rsults in a deadlock or the eventual exhaustion of some resource (memory, file handles, ...)
This explanation is deliberately written following a (hopefully) intuitive approach and is technically somewhat sloppy. For a detailed explanation consult https://www.promisejs.org/, MDN: Promises, or similar resources.
Caveat
There is some guesswork involved in this answer. However the general outline of the problem's origin should remain valid.

Node.js Uncaught Exception - Passing more details

First of all, I know one way to catch uncaught errors is use process.on('uncaughtException', err, function() {});.
I want to know how I can pass more details, or more context to the error. I want to be able to get the variables used. I'm not trying to recover from the error, only get more details of my environment when the error happened before shutting down the process. Yeah, the stack trace is nice, but I'd like to know how to replicate the error.
For example, I have this function:
function doTheBatman(var1) {
var var2 = 'whatever';
// this causes an uncaught exception later in the code
}
On process.on, I want to be able to access var1 and var2.
process.on('uncaughtException', function(err) {
// process.whatever doesn't contain any active variables
});
A synchronous exception in an Express route handler will be caught by Express itself and the exception will be passed off the default Express error handler where you can catch it yourself and the exception context is passed to that default express error handler.
You can see this code inside of Express where a route handler gets called:
Layer.prototype.handle_request = function handle(req, res, next) {
var fn = this.handle;
if (fn.length > 3) {
// not a standard request handler
return next();
}
try {
fn(req, res, next);
} catch (err) {
next(err);
}
};
The call to next(err) will pass the exception object off to the default Express error handler (which you can install a handler for).
If your code is throwing an exception inside of an asynchronous callback, then that is more complicated to catch in action. If you're using regular async callbacks (not promises), then the only way I know of to catch those at a meaningful spot is to put a try/catch inside of every async callback so you can capture the local stack info.
If you use promises at the lowest level and only program your logic and asynchronous code flow use promise functionality, then an exception in a promise handler will automatically turn into a rejected promise and certain promise libraries (like the Bluebird library can be configured to give you a pretty full stack trace of where things went wrong). Promises have this advantage because every promise .then() or .catch() handler is automatically wrapped in a try/catch handler and those are turned into promise rejections which propagate up the chain to the first .catch() handler they find.
For the non-Express example you just added to your question, you will just have to put a try/catch somehwere in the local neighborhood to catch a local synchronous exception:
function doTheBatman(var1) {
try {
var var2 = 'whatever';
// this causes an uncaught exception later in the code
} catch(e) {
console.log(e);
}
}
You can even set a debugger breakpoint in the catch handler and then examine variables in that scope when it is hit. I don't know of any way to examine the local variables at the point of the exception from the uncaughtException handler. err.stack should give you the stack trace, but not variable values.
Express/frameworks may offer more elegant/robust solutions, but if you're truly after what your question asks for... why not just capture variables outside of the functions scope? This is typically nasty and not considered best practice, but if you have a function that you know could be susceptive to failing often, perhaps the quick and dirty solution is what you need. You could always refine this later, but hopefully this demonstrates the approach...
var transactionVars = {};
function clearTransaction() {
transactionVars = {};
}
function doTheBatman(var1) {
transactionVars['var2'] = 'whatever';
// [...] bunch of stuff, possibly blowing up...
clearTransaction(); // we made it this far? cool, reset...
}
process.on('uncaughtException', function(err) {
console.log(transactionVars['var2']); // whatever
});
Furthermore, if you want to really dirty (in case these two functions are in two files) you can always tack transactionVars on the global object.
This is essentially a poor mans event emitter pattern, which I would highly recommend refactoring into once you grasp the general flow of how this works...

NodeJS : prevent application to crash on error

When an error occurs in NodeJS application, the server may crash and stop if an exception occurs.
How can I prevent this situation so that the server never stops on error, but returns and error code instead ?
--
EDIT
Here is a method which causes a server crash (table foo doesn't exist) :
app.get('/articles/list', function(req, res) {
connection.query('select * from foo', function(err, rows, fields) {
if (err) throw err;
res.send(JSON.stringify(rows));
});
});
-- BEST SOLUTION
Finally, I found the best solution for me :
http://rowanmanning.com/posts/node-cluster-and-express/
In summary, it consists in using a cluster of server, and to restart a server when it exits
If you are using express, you can use this
function clientErrorHandler (err, req, res, next) {
if (req.xhr) {
res.status(500).send({ error: 'Something failed!' })
} else {
next(err)
}
}
app.use(clientErrorHandler);
I suggest you to ready this article, it will clarify a lot for you.
JFK, be aware of async error catching, most common error, once you have async stuff which is wrapped in try-catch.
Also, avoid usage of
process.on("uncaughException", () => {....})
This is quite bad, because when an uncaught exception is thrown you can’t reliably continue your program at this point.Let it fail and restart with daemonizer's like pm2, forever and etc.
As general rule, you should handle all errors and promise rejects in your code to avoid uncaught Exceptions.
For errors: using try-catch statement
For promises: using .catch method
Btw Node permits you to intercept an uncaughtExceptions or unhandledRejection events, but is not a good practice to handle errors at this level and prevent program exits, because can generate unpredictable behaviours.
Use try catch... for parts of code, which throws exception
try{
//youre code here
}catch(error){
console.log(error)
}

Error thrown from a mongoose-promise callback function is not caught

I spent too much time trying to figure out why my express.js controller did not respond to a simple query, and figured out that runtime errors fired from a Mongoose-promise callback were silently interrupting the callback process.
Here is a simplified version of my code:
server.get('/api/test', function (req, res, next) {
User.find({}).exec().then(function success(users){
console.log('SUCCESS');
typo[0] = 1; // throws a runtime error
res.json(users);
}, function error(err){
console.log('ERROR');
res.json({error: err});
});
});
This results in SUCCESS showing up in my console, but nothing happens then. No response is given to the user, the error caused by my typo is not appearing in my console, and the error callback is not called either.
I am aware that one should not throw exceptions from a callback function, but in that case, this was just a typo, and it would make sense to me to be warned (e.g. a stack trace in my standard output) whenever one makes this kind of mistake. (we're humans, after all...)
In your opinion, what's the best way to get feedback whenever this kind of mistakes are made in promise callbacks?
This is Mongoose's fault for using a bad promise implementation. Promises are throw-safe so exceptions are caught (so they can be later handled by future code) - the future code never comes and Mongoose never reports that it did not. Good promise implementations do not suffer from this issue.
Your options are two:
Use a library like Bluebird:
var Promise = require("bluebird");
var mongoose = Promise.promisifyAll(require("mongoose"));
User.findAsync({}).then(function(data){
JSON.prase("dsa"); // not a silent failure, will show up, easy debugging
});
This has the advantage of being faster than mongoose promises so there is no performance penalty. Alternatively, if you're super conservative and don't want the performance and API gains of bluebird - you can use native promises:
// Promise is the native promise
Promise.resolve(User.find({}).exec()).then(function(data){
JSON.prase("dsa");
});
And then, assuming you're running a modern variant of nodejs (read: io.js v 1.4.1 or upper), you can subscribe to promise rejections:
process.on("unhandledRejection", function(p, why){
console.log("FOUND ERROR!!!!", p , why);
});
So exceptions are not silently suppressed.
The exec() has two promises
.then(function)
.then(null , function)
try this, I think it will help
server.get('/api/test', function(req, res, next) {
User.find({}).exec()
.then(function success(users) {
console.log('SUCCESS');
typo[0] = 1; // throws a runtime error
res.json(users);
})
.then(null, function error(err) {
console.log('ERROR');
res.json({
error: err
});
});
});

Unable to handle exception with node.js domains using express

I want to use Node.js Domains to catch exceptions. It is working so far, but there is one place I can't get domains to catch the exception. exception2 in the callback is caught and handled in the domain.on('error') handler, but exception1 is not caught. The odd thing is that when exception1 is thrown, it doesn't shutdown Node like I would expect. Here is my example app:
var domain = require('domain');
var request = require('request');
var express = require('express');
var serverDomain = domain.create();
serverDomain.on('error', function(err) {
console.log("Server Domain Error: " + err);
});
var app;
serverDomain.run(function() {
app = express();
app.listen(3000);
});
app.use(function(req, res, next) {
var reqDomain = domain.create();
reqDomain.add(req);
reqDomain.add(res);
reqDomain.on('error', function(err) {
console.log("Req Domain Error: " + err);
reqDomain.dispose();
next(err);
});
next();
});
app.get('/', function(req, res) {
var uri = "http://google.com";
exception1.go();
request.get({url:uri, json: {}},
function (error, response, body) {
if(response.statusCode === 200) {
exception2.go();
res.send('Success getting google response');
}
});
});
To get exception2 to execute, I comment out exception 1.
The problem is that the exception happens during Connect's routing, which has both a try/catch block around its execution, as well as a default error handler which prints out stack trace details when running in a non-production mode. Since the exception is handled inside of Express, it never reaches your outer layer for the domains to handle.
How it differs from exception2 is that the handler function for the '/' route is executed directly by that Connect block, in the same stack as the original call that went through Express. The second exception occurs in a callback, after some I/O operation has returned, and therefore is executed by a stack originating from Node's event loop I/O handler, and so the try/catch of Express isn't available to snag that exception and save the app server. In fact, if you comment out all the domain stuff, and trip exception2 it crashes Node.
Since only unhandled exceptions are routed to the domain mechanism, and since exception1 has a try/catch visible in it's call stack above it, the exception is handled, and not forwarded to the domain.
Connect-domain allows you to catch all errors for connect modules including express.
Connect-domain
https://github.com/baryshev/connect-domain
The example for express3:
http://masashi-k.blogspot.com/2012/12/express3-global-error-handling-domain.html
#user1007983
No, in production, the try/catch handling still exists in Connect/Express. The way to solve it is to add your own try/catch block in the "root" which you can then use to emit an "error" event to the domain before connect sends the error response.
try {
// .. code here ..
} catch (err) {
domain.emit('error', err);
}
Another way to get around it is to just disconnect from the handler, like wrapping your code in a setImmediate block
I've tried try/catch blocks (which may not work the way you think with async code). I've tried node domains in several different ways. But I was unable to handle an exception thrown in a 3rd party lib (sequelize). Why did I get the exception? Well, it was because the SQL that was generated was not well formed. (My fault).
Anywho, the solution that worked best for me and my (small) project was to use forever. Let the exceptions happen, but fire up node again if they do. I doubt it is the most elegant solution, but it works for me and my small project.
With a larger project, domains combined with the clustering API might be a good choice.
Winston is another choice. It might be cool to combine forever with winston so that if you DO get an exception, you can have winston email you, so you can fix the code. Meanwhile, forever will happily restart the app for you.
I came across this problem while testing my domain-based error handling.
I went with the approach suggested here by Issac, with a couple of minor changes: use 'domain.active' to get the currently active domain and emit the error event on that, and I used a wrapper function to avoid having to modify all of my handler functions:
domainWrapper = function(func) {
return function(req, res) {
try {
return func(req, res);
} catch (err) {
domain.active.emit('error', err);
}
}
}
Then changed this sort of thing:
app.get('/jobs', handlerFunc);
to this:
app.get('/jobs', domainWrapper(handlerFunc));

Resources