try
p = req.params.name
Item.update('name': p, req.body , {upsert: true}, (err) ->
if err?
throw err
res.send("ok")
)
catch e
handle_error(e, "Error salvando hoja de vida.", res)
This produces an error in my code right now - that's alright, but why does my nodejs program crash even if I have a try catch here?
The error is:
MongoError: Mod on _id not allowed
(so it must be in the update call)
I am specifically looking for a way to catch an error, I already know how to get rid of it.
Ah yes, you have entered the realm of asynchronous code. The reason you pass a callback into your MongoDB calls like Item.update(..., callback) is because the MongoDB driver is written to be asynchronous. Consider this (mongoose code):
var user = User.findOne({ name: 'joe' });
doSomethingElse();
If it had structured its API like the above code then your entire application would halt until User.findOne() returned the results from the database. Your doSomethingElse() call wouldn't be invoked until the user was retrieved, even if you don't do anything with user inside doSomethingElse. This is a big no no these days and especially in node which has been written to be asynchronous as much as possible. Have a look at the following:
User.findOne({ name: 'joe' }, function (err, user) {
if (err) return console.error(err);
console.log('Do stuff with user... ;)');
});
doSomethingElse();
The above code is asynchronous. The findOne function returns immediately and doSomethingElse is able to begin even before the user is ever retrieved from the database. But of course we still want to do stuff with our user, so in order to accomplish this we pass an anonymous function in to be used as a callback. The MongoDB driver is smart enough to call that function when it's all done retrieving data.
Wrapping the above code in a try/catch would be pointless, unless you suspected the findOne function to throw an exception (Which you shouldn't. It returns immediately remember?).
try {
User.findOne({ name: 'joe' }, function (err, user) {
throw new Error("Something went wrong...");
});
} catch (err) {
console.error(err);
}
The above error would still crash your program because it happened long after findOne returned and your program moved on way beyond that precious try/catch. This is the very reason why your callback function receives an err argument. The MongoDB driver knows that if some error occurs while fetching your data it would be no good to throw an exception and call it good. This is because all this processing has happened later on subsequent trips through the event loop, having left your try/catch behind several iterations ago.
To get around this the MongoDB driver wraps its own internal synchronous code in a try/catch where it actually will handle the exception and then it invokes your callback passing in the error as the first argument. This allows you to check that argument and handle any errors within your callback.
I wrote an entire blog post explaining callbacks, as well as another way of handling asynchronous code called "promises" and I think it would help clarify some things for you :) http://codetunnel.io/what-are-callbacks-and-promises
I hope that helps answer your question.
Maybe I'm wrong (because I don't know Coffeescript), but I suppose the problem is, that your try/catch is outside the callback function. The catch doesn't affect your err -> so the exception is not caught by you, but by node.js.
If I'm wrong, could you perhaps provide the compiled JavaScript code?
Edit
Usually one would handle the error more like this (without try/catch):
Item.update('name': p, req.body , {upsert: true}, (err) ->
if err?
log.error('error updating Item name=' + name)
res.status(500).end()
return
res.send("ok")
)
Probably your handle_error will do this already.
When programming in node.js, I use try/catch only for JSON.parse or if I have a lot of unchecked input which might have wrong types or might be null and I'm too lazy to check all input manually. Surrounding asynchronous functions with try/catch is only helpful if the function has programming errors (e.g. not checking your input correctly).
Related
Assume an Express route that makes a call to Mongoose and has to be async so it can await on the mongoose.find(). Also assume we are receiving XML but we have to change it to JSON, and that also needs to be async so I can call await inside of it.
If I do this:
app.post('/ams', async (req, res) => {
try {
xml2js.parseString(xml, async (err, json) => {
if (err) {
throw new XMLException();
}
// assume many more clauses here that can throw exceptions
res.status(200);
res.send("Data saved")
});
} catch(err) {
if (err instanceof XML2JSException) {
res.status(400);
message = "Malformed XML error: " + err;
res.send(message);
}
}
}
The server hangs forever. I'm assuming the async/await means that the server hits a timeout before something concludes.
If I put this:
res.status(200);
res.send("Data saved")
on the line before the catch(), then that is returned, but it is the only thing every returned. The client gets a 200, even if an XMLException is thrown.
I can see the XMLException throw in the console, but I cannot get a 400 to send back. I cannot get anything I that catch block to execute in a way that communicates the response to the client.
Is there a way to do this?
In a nutshell, there is no way to propagate an error from the xml2js.parseString() callback up to the higher code because that parent function has already exited and returned. This is how plain callbacks work with asynchronous code.
To understand the problem here, you have to follow the code flow for xml2js.parseString() in your function. If you instrumented it like this:
app.post('/ams', async (req, res) => {
try {
console.log("1");
xml2js.parseString(xml, async (err, json) => {
console.log("2");
if (err) {
throw new XMLException();
}
// assume many more clauses here that can throw exceptions
res.status(200);
res.send("Data saved")
});
console.log("3");
} catch (err) {
if (err instanceof XML2JSException) {
res.status(400);
message = "Malformed XML error: " + err;
res.send(message);
}
}
console.log("4");
});
Then, you would see this in the logs:
1 // about to call xml2js.parseString()
3 // after the call to xml2js.parseString()
4 // function about to exit
2 // callback called last after function returned
The outer function has finished and returned BEFORE your callback has been called. This is because xml2js.parseString() is asynchronous and non-blocking. That means that calling it just initiates the operation and then it immediately returns and the rest of your function continues to execute. It works in the background and some time later, it posts an event to the Javascript event queue and when the interpreter is done with whatever else it was doing, it will pick up that event and call the callback.
The callback will get called with an almost empty call stack. So, you can't use traditional try/catch exceptions with these plain, asynchronous callbacks. Instead, you must either handle the error inside the callback or call some function from within the callback to handle the error for you.
When you try to throw inside that plain, asynchronous callback, the exception just goes back into the event handler that triggered the completion of the asynchronous operation and no further because there's nothing else on the call stack. Your try/catch you show in your code cannot catch that exception. In fact, no other code can catch that exception - only code within the exception.
This is not a great way to write code, but nodejs survived with it for many years (by not using throw in these circumstances). However, this is why promises were invented and when used with the newer language features async/await, they provide a cleaner way to do things.
And, fortunately in this circumstance xml2js.parseString() has a promise interface already.
So, you can do this:
app.post('/ams', async (req, res) => {
try {
// get the xml data from somewhere
const json = await xml2js.parseString(xml);
// do something with json here
res.send("Data saved");
} catch (err) {
console.log(err);
res.status(400).send("Malformed XML error: " + err.message);
}
});
With the xml2js.parseString() interface, if you do NOT pass it a callback, it will return a promise instead that resolves to the final value or rejects with an error. This is not something all asynchronous interfaces can do, but is fairly common these days if the interface had the older style callback originally and then they want to now support promises. Newer interfaces are generally just built with only promise-based interfaces. Anyway, per the doc, this interface will return a promise if you don't pass a callback.
You can then use await with that promise that the function returns. If the promise resolves, the await will retrieve the resolved value of the promise. If the promise rejects, because you awaiting the rejection will be caught by the try/catch. FYI, you can also use .then() and .catch() with the promise, but in many cases, async and await are simpler so that's what I've shown here.
So, in this code, if there is invalid XML, then the promise that xml2js.parseString() returns will reject and control flow will go to the catch block where you can handle the error.
If you want to capture only the xml2js.parseString() error separately from other exceptions that could occur elsewhere in your code, you can put a try/catch around just it (though this code didn't show anything else that would likely throw an exception so I didn't add another try/catch). In fact, this form of try/catch can be used pretty much like you would normally use it with synchronous code. You can throw up to a higher level of try/catch too.
A few other notes, many people who first start programming with asynchronous operations try to just put await in front of anything asynchronous and hope that it solves their problem. await only does anything useful when you await a promise so your asynchronous function must return a promise that resolves/rejects when the asynchronous operation is complete for the await to do anything useful.
It is also possible to take a plain callback asynchronous function that does not have a promise interface and wrap a promise interface around it. You pretty much never want to mix promise interface functions with plain callback asynchronous operations because error handling and propagation is a nightmare with a mixed model. So, sometimes you have to "promisify" an older interface so you can use promises with it. In most cases, you can do that with util.promisify() built into the util library in nodejs. Fortunately, since promises and async/await are the modern and easier way to do asynchronous things, most newer asynchronous interfaces in the nodejs world come with promise interfaces already.
You are throwing exceptions inside the callback function. So you cant expect the catch block of the router to receive it.
One way to handle this is by using util.promisify.
try{
const util = require('util');
const parseString = util.promisify(xml2js.parseString);
let json = await parsestring(xml);
}catch(err)
{
...
}
I'm new to node.js and using it for a backend that takes data from syslog messages and stores it to a database.
I've run into the following type of serial operations:
1. Query the database
2. If the query value is X do A. otherwise do B.
A. 1. Store "this" in DB
2. Query the database again
3. If the query value is Y do P. otherwise do Q.
P. Store "something"
Q. Store "something else"
B. 1. Store "that" in the DB
2. Store "the other thing" in the DB
The gist here is that I have some operations that need to happen in order, but there is branching logic in the order.
I end up in callback hell (I didn't know what that is when I came to Node.. I do now).
I have used the async library for things that are more straight forward - like doing things in order with async.forEachOfSeries or async.queue. But I don't think there's a way to use that if things have to happen in order but there is branching.
Is there a way to handle this that doesn't lead to callback hell?
Any sort of complicated logic like this is really, really going to benefit from using promises for every async operation, especially when you get to handling errors, but also just for structuring the logic flow.
Since you haven't provided any actual code, I will make up an example. Suppose you have two core async operations that both return a promise: query(...) and store(...).
Then, you could implement your above logic like this:
query(...).then(function(value) {
if (value === X) {
return store(value).then(function() {
return query(...).then(function(newValue) {
if (newValue === Y) {
return store("something");
} else {
return store("something else");
}
})
});
} else {
return store("that").then(function() {
return store("the other thing");
});
}
}).then(function() {
// everything succeeded here
}, function(err) {
// error occurred in anyone of the async operations
});
I won't pretend this is simple. Implemented logic flow among seven different async operation is just going to be a bit of code no matter how you do it. But, promises can make it less painful than otherwise and massively easier to make robust error handling work and to interface this with other async operations.
The main keys of promises used here are:
Make all your async operations returning promises that resolve with the value or reject with the failure as the reason.
Then, you can attach a .then() handler to any promise to see when the async operation is finished successfully or with error.
The first callback to .then() is the success handler, the second callback is the error handler.
If you return a promise from within a .then() handler, then that promise gets "chained" to the previous one so all success and error state becomes linked and gets returned back to the original promise state. That's why you see the above code always returning the nested promises. This automates the returning of errors all the way back to the caller and allows you to return a value all the way back to the caller, even from deeply nested promises.
If any promise in the above code rejects, then it will stop that chain of promises up until an actual reject handler is found and propagate that error back to that reject handler. Since the only reject handler in the above code is at the to level, then all errors can be caught and seen there, no matter how deep into the nested async mess it occurred (try doing that reliably with plain callbacks - it's quite difficult).
Native Promises
Node.js has promises built in now so you can use native promises for this. But, the promise specification that node.js implements is not particularly feature rich so many folks use a promise library (I use Bluebird for all my node.js development) to get additional features. One of the more prominent features is the ability to "promisify" an existing non-promise API. This allows you to take a function that works only with a callback and create a new function that works via a promise. I find this particularly useful since many APIs that have been around awhile do not natively return promises.
Promisifying a Non-Promise Interface
Suppose you had an async operation that used a traditional callback and you wanted to "promisify it". Here's an example of how you could do that manually. Suppose you had db.query(whatToSearchFor, callback). You can manually promisify it like this:
function queryAsync(whatToSearchFor) {
return new Promise(function(resolve, reject) {
db.query(whatToSearchFor, function(err, data) {
if (!err) {
reject(err);
} else {
resolve(data);
}
});
});
}
Then, you can just call queryAsync(whatToSearchFor) and use the returned promise.
queryAsync("foo").then(function(data) {
// data here
}, function(err) {
// error here
});
Or, if you use something like the Bluebird promise library, it has a single function for promisifying any async function that communicates its result via a node.js-style callback passed as the last argument:
var queryAsync = Promise.promisify(db.query, db);
It is a convention in node to pass an error parameter to asynchronous operations:
async.someMagicalDust(function(callback) {
// some asynchronous task
// […]
callback();
}, function(err) {
// final callback
if(err) throw err;
// […]
});
Maybe I'm too naive, but I've never been a big fan of the if(variable) notation — probably inherited from C, for reasons that have already been discussed many times in the past.
On the other hand, I have sometimes encountered a null parameter and this error check:
if(typeof err !== 'undefined' && err !== null)
is a bit too verbose.
Another solution would be
if(err != null)
but I think the non-strict check can be tricky, even though I consider it normal when comparing with null.
What's the best way to check error parameters in node?
Use if(err).
It is designed to be used in this fashion. Node-style callbacks are supposed to set the error to a non-falsy value only in case of actual error. You won't find any sane example of setting err to '' or 0 to signify error.
Just like YlinaGreed has noted, if module conventions cnahge from null to undefined to 0 to maybe even NaN, you are still safe. I've never been hit by this having only used if(err).
On the other hand, you may want to use coffescript, which would translate for you
unless err? then...
into
if (typeof err === "undefined" || err === null) {
mimicking the most common pattern.
Some links to corroborate the if(err) approach:
https://gist.github.com/klovadis/2548942
http://caolanmcmahon.com/posts/nodejs_style_and_structure/ (by Caolan McMahon, author of async)
http://thenodeway.io/posts/understanding-error-first-callbacks/
The convention seems to be passing an error object as the first argument and null for no error so even if you pass an empty object, it's still an error.
If you use the popular express framework, you should have used the next callback to return from middleware, which follows the errback convention.
I believe that most people prefer more concise next() than next(null), which means that the first argument will evaluate to undefined rather than null, and this is certainly perfectly normal usage.
To me, the best way to handle error is the "if (err == null)"
This is the only case I am using the non strict operator, for these reasons:
The only over solution which always works is very verbose, as you said before
You could also check only on "null" or "undefined", but I did this once, and a few month later, I just updated my dependencies and... The convention changed, and the module was sending null instead of undefined.
This is mostly a matters of "convention", I have mine, and you certairnly have your's too... Just be careful to chose one of the two "good" ways.
Node's primary callback convention is to pass a function with err as the first parameter. In my experience its always been safe to check if error is truthy - in practice if your error comes out null when there IS an error then the problem lies more with the implementation. I would always expect if err is null that no error occured. It may be confusing due to the use of separate functions for errors and successes, something that is more in the style of JQuery.Ajax and promises. I tend to find double callbacks to be a bit too wordy to call.
Given your example it seems you're using the async library which is excellent. If I am looking to perform a parallel option this is how I set it up:
function doAThing(callback) {
var err;
// do stuff here, maybe fill the err var
callback(err);
}
function doAsyncThings(callback) {
var tasks = [function(done) { // stuff to do in async
doAThing(function(err) {
done(err);
});
}];
async.parallel(tasks, function(err) { // single callback function
callback(err); // I send the error back up
});
}
Note that instead of throwing the error I bubbled it back up the request chain. There are few instances which I'd want to actually throw an error since you're basically saying "crash the whole app".
I find it to be simpler and reduces the amount of parameters you have to use to call your functions. When you use this convention throughout you you can simplify by simply passing the callback as a parameter instead of creating a new anonymous function, like so:
function doAThing(callback) {
var err;
// do stuff here, maybe fill the err var
callback(err);
}
function doAsyncThings(callback) {
var tasks = [function(done) { // stuff to do in async
doAThing(done);
}];
async.parallel(tasks, callback); // the error is sent back to the original function
}
I find that generally you want to handle those errors in the functions they are called in. So in this case the caller of doAsyncThings can check if there is an error and handle it appropriate to its own scope (and perhaps provide better information to the user if it is say an API).
I am using Azure Mobile services-NodeJS backend, and when programming it, I always face this doubt - let me explain using the below code snippet
//--------------------------------------------------------------------------
function addUserToDB(request, response){
///some code here
var theUser = request.user;
///get the user's entity object
try {
objAppUser = buildAppUserEntityObj(theUser, request); //for simplicity sake, lets say this is not asynchronous function
}
catch (err) {
console.log ('error in addUserToDB when calling buildAppUserEntityObj'); //****????****
request.respond(statusCodes.BAD_REQUEST, err);
return; // ##????## is a 'return' needed here to avoid the execution of the code below, or should I assume that the function will return once request is responded (request.respond) in above line.
}
....code to add userEntity to DB
//some more code in case of successful try above, can I assume there is no way this code will be reached in case of error in the above try-catch
// ofcourse I can move this code in the 'try' block above, but I am just trying to understand what happens if above try ends in catch block for some reason and there is no 'return' at the end that catch block.
}
//--------------------------------------------------------------------------
function buildAppUserEntityObj(user, request) {
if ( user.level === 'anonymous' ) { //ideally this would be called in above function, but I am putting this here just to throw an example.
console.error('Anonymous User' );
request.respond(statusCodes.BAD_REQUEST, message); //will this request.respond will send the response to client immediately, or will it be passed on as error the try-catch of above 'addUserToDB' function
return; // ##????## also, is 'return' needed here to avoid the execution of the code below,
}
....code to build a User entity object based on some business logic
}
//--------------------------------------------------------------------------
I guess, it all boils down to three questions:
1. Is 'return' needed in the two places (marked by ##????## in the above two functions?
2. Will the message (marked by //****????**** ) be logged in case user.level === 'anonymous'
3. request.respond vs response.send , whats the difference?
I believe these doubts are because of my lack of thorough expressJS knowledge, so while I am going thru azure/express.js documentation again, I thought I would throw my doubt here to the expert community to get a more clear explanation.
Many thanks.
First
In the second return (insode of buildAppUserEntityObj function, I believe you want it to be:
throw new Error("Anonymous user is not allowed")
Otherwise, even if user is anonymous, your catch code will never execute anyway.
You need the first return;, otherwise it will continue executing the code below.
Second
Message will be logged, if you fix the code described in First paragraph.
Third
There is no request.respond in standard Node.js http module. Can you clarify, what module are you using? That module's API shall answer your question anyway.
I have the node.js code running on a server and would like to know if it is blocking or not. It is kind of similar to this:
function addUserIfNoneExists(name, callback) {
userAccounts.findOne({name:name}, function(err, obj) {
if (obj) {
callback('user exists');
} else {
// Add the user 'name' to DB and run the callback when done.
// This is non-blocking to here.
user = addUser(name, callback)
// Do something heavy, doesn't matter when this completes.
// Is this part blocking?
doSomeHeavyWork(user);
}
});
};
Once addUser completes the doSomeHeavyWork function is run and eventually places something back into the database. It does not matter how long this function takes, but it should not block other events on the server.
With that, is it possible to test if node.js code ends up blocking or not?
Generally, if it reaches out to another service, like a database or a webservice, then it is non-blocking and you'll need to have some sort of callback. However, any function will block until something (even if nothing) is returned...
If the doSomeHeavyWork function is non-blocking, then it's likely that whatever library you're using will allow for some sort of callback. So you could write the function to accept a callback like so:
var doSomHeavyWork = function(user, callback) {
callTheNonBlockingStuff(function(error, whatever) { // Whatever that is it likely takes a callback which returns an error (in case something bad happened) and possible a "whatever" which is what you're looking to get or something.
if (error) {
console.log('There was an error!!!!');
console.log(error);
callback(error, null); //Call callback with error
}
callback(null, whatever); //Call callback with object you're hoping to get back.
});
return; //This line will most likely run before the callback gets called which makes it a non-blocking (asynchronous) function. Which is why you need the callback.
};
You should avoid in any part of your Node.js code synchronous blocks which don't call system or I/O operations and which computation takes long time (in computer meaning), e.g iterating over big arrays. Instead move this type of code to the separate worker or divide it to smaller synchronous pieces using process.nextTick(). You can find explanation for process.nextTick() here but read all comments too.