Feel a bit sheepish asking something so remedial given I've been using Node & Mongo for a year now. That whole time I've stuck to the form;
collection.find({field:value}, function(error, cursor) {
if (error) return ErrorHandler(error);
cursor.toArray(function(error, results) {
if (error) return ErrorHandler(error);
...
});
});
Mainly because (back when I was first trying to get my head around Node's single-threaded ways) each database call is explicitly non-blocking. But obviously it's a bit verbose compared to;
collection.find({field:value}).toArray(function(error, results) {
if (error) return ErrorHandler(error);
...
});
I wonder if someone could just confirm for me that the second form is equivalently asynchronous and that the driver is dealing with the find callback in the background and only calling me back when both things are done?
Is there reason to suppose the second form might be a bit more performant (on the basis that the driver is presumably implemented in a low-level language)?
Related
I have two layers in my application(express), first is module with function which is handling database queries, fs , and so on. Second is handling requests(also known as controller/route). I just tired of all this conditions.
Sample code:
exports.updateImage = function(image, userId, callback) {
fs.readFile(image.path, function (err, imageBinary) {
if (err) callback(err);
else {
pg.connect(conString, function(err, client, done) {
done();
if (err) callback(err);
else {
client.query('UPDATE images SET data=$1, filesize=$2, filename=$3 WHERE user_id=$4', [imageBinary, image.size, image.originalFilename, userId], function(err) {
if (err) callback(err);
else callback(null);
});
}
});
}
});
};
As you can see, I callback all my errors to my controller, then it handled as internal server error. I handle database, file system possible errors, and there is too much repetitions in my code. I suppose it is bad design, and it hard to support in production. Please help me.
When you say "tired of all these conditions" I assume you're talking about all the nested callbacks and the "march off the right side of the screen" that results from that kind of directly nested callbacks? If I'm assuming incorrectly please clarify your question and I'll delete everything I'm about to write as not related. :-)
One cheap way to avoid the else structure is to instead of doing
if(err) callback(err);
else { ... stuff ... }
is to do this:
if(err) return callback(err);
Note the return: that causes execution of your function to end, nobody cares about the return values from a callback so they just get ignored. So that potentially gets rid of a layer of braces and elses.
To handle this better in general, you'll want to look at some sort of async helpers. There's three general categories of these things:
Helper libraries that manage the sequencing of multiple callbacks,
Promises, which let you represent async operations as objects, or
Language support to hide the details.
Examples of the three different types of libraries include step, flow, or async as helper libraries, for promises there's Q or when.js, and for language support look at streamline.
For more details, I did a presentation on exactly this topic about a year ago; the slides are here are there's a recording of the presentation as well.
I have a codebase which contains code similar to the code below many times:
function(doc, callback) {
doSomething(function(err) {
if(err) return callback(err);
callback(null, doc);
});
}
I'm wondering if there are any downsides to just combining the explicit error check into:
function(doc, callback) {
doSomething(function(err) {
callback(err, doc);
});
}
I understand that callback handlers are expected to check the err on callback, but in this case it's just bubbling up.
I suppose I'm wondering if based on the way callbacks are generally used, if this is an issue?
There is no difference, the code is doing the same thing. First one is just easier to edit later if you want to add some postprocessing.
Technically, second example provides a "doc" and first don't, but if somebody rely on that, they're doing it very wrong.
try
p = req.params.name
Item.update('name': p, req.body , {upsert: true}, (err) ->
if err?
throw err
res.send("ok")
)
catch e
handle_error(e, "Error salvando hoja de vida.", res)
This produces an error in my code right now - that's alright, but why does my nodejs program crash even if I have a try catch here?
The error is:
MongoError: Mod on _id not allowed
(so it must be in the update call)
I am specifically looking for a way to catch an error, I already know how to get rid of it.
Ah yes, you have entered the realm of asynchronous code. The reason you pass a callback into your MongoDB calls like Item.update(..., callback) is because the MongoDB driver is written to be asynchronous. Consider this (mongoose code):
var user = User.findOne({ name: 'joe' });
doSomethingElse();
If it had structured its API like the above code then your entire application would halt until User.findOne() returned the results from the database. Your doSomethingElse() call wouldn't be invoked until the user was retrieved, even if you don't do anything with user inside doSomethingElse. This is a big no no these days and especially in node which has been written to be asynchronous as much as possible. Have a look at the following:
User.findOne({ name: 'joe' }, function (err, user) {
if (err) return console.error(err);
console.log('Do stuff with user... ;)');
});
doSomethingElse();
The above code is asynchronous. The findOne function returns immediately and doSomethingElse is able to begin even before the user is ever retrieved from the database. But of course we still want to do stuff with our user, so in order to accomplish this we pass an anonymous function in to be used as a callback. The MongoDB driver is smart enough to call that function when it's all done retrieving data.
Wrapping the above code in a try/catch would be pointless, unless you suspected the findOne function to throw an exception (Which you shouldn't. It returns immediately remember?).
try {
User.findOne({ name: 'joe' }, function (err, user) {
throw new Error("Something went wrong...");
});
} catch (err) {
console.error(err);
}
The above error would still crash your program because it happened long after findOne returned and your program moved on way beyond that precious try/catch. This is the very reason why your callback function receives an err argument. The MongoDB driver knows that if some error occurs while fetching your data it would be no good to throw an exception and call it good. This is because all this processing has happened later on subsequent trips through the event loop, having left your try/catch behind several iterations ago.
To get around this the MongoDB driver wraps its own internal synchronous code in a try/catch where it actually will handle the exception and then it invokes your callback passing in the error as the first argument. This allows you to check that argument and handle any errors within your callback.
I wrote an entire blog post explaining callbacks, as well as another way of handling asynchronous code called "promises" and I think it would help clarify some things for you :) http://codetunnel.io/what-are-callbacks-and-promises
I hope that helps answer your question.
Maybe I'm wrong (because I don't know Coffeescript), but I suppose the problem is, that your try/catch is outside the callback function. The catch doesn't affect your err -> so the exception is not caught by you, but by node.js.
If I'm wrong, could you perhaps provide the compiled JavaScript code?
Edit
Usually one would handle the error more like this (without try/catch):
Item.update('name': p, req.body , {upsert: true}, (err) ->
if err?
log.error('error updating Item name=' + name)
res.status(500).end()
return
res.send("ok")
)
Probably your handle_error will do this already.
When programming in node.js, I use try/catch only for JSON.parse or if I have a lot of unchecked input which might have wrong types or might be null and I'm too lazy to check all input manually. Surrounding asynchronous functions with try/catch is only helpful if the function has programming errors (e.g. not checking your input correctly).
I have the node.js code running on a server and would like to know if it is blocking or not. It is kind of similar to this:
function addUserIfNoneExists(name, callback) {
userAccounts.findOne({name:name}, function(err, obj) {
if (obj) {
callback('user exists');
} else {
// Add the user 'name' to DB and run the callback when done.
// This is non-blocking to here.
user = addUser(name, callback)
// Do something heavy, doesn't matter when this completes.
// Is this part blocking?
doSomeHeavyWork(user);
}
});
};
Once addUser completes the doSomeHeavyWork function is run and eventually places something back into the database. It does not matter how long this function takes, but it should not block other events on the server.
With that, is it possible to test if node.js code ends up blocking or not?
Generally, if it reaches out to another service, like a database or a webservice, then it is non-blocking and you'll need to have some sort of callback. However, any function will block until something (even if nothing) is returned...
If the doSomeHeavyWork function is non-blocking, then it's likely that whatever library you're using will allow for some sort of callback. So you could write the function to accept a callback like so:
var doSomHeavyWork = function(user, callback) {
callTheNonBlockingStuff(function(error, whatever) { // Whatever that is it likely takes a callback which returns an error (in case something bad happened) and possible a "whatever" which is what you're looking to get or something.
if (error) {
console.log('There was an error!!!!');
console.log(error);
callback(error, null); //Call callback with error
}
callback(null, whatever); //Call callback with object you're hoping to get back.
});
return; //This line will most likely run before the callback gets called which makes it a non-blocking (asynchronous) function. Which is why you need the callback.
};
You should avoid in any part of your Node.js code synchronous blocks which don't call system or I/O operations and which computation takes long time (in computer meaning), e.g iterating over big arrays. Instead move this type of code to the separate worker or divide it to smaller synchronous pieces using process.nextTick(). You can find explanation for process.nextTick() here but read all comments too.
Using node.js I'm creating a function to update a file that contains a JSON list by appending a new element to the list. The updated list is rewritten back to the file. If the file doesn't exist I create it.
Below, __list_append(..) does the list append and file update.
My question is if (and should) I can restructure this code to not have two calls to __list_append. I'm a bit new to node.js, and don't have a good feel for the asynchronous tactics.
function list_append(filename, doc) {
fs.exists(filename, function(exists) {
if (exists) {
fs.readFile(filename, function(err, data) {
if (err)
throw err;
__list_append(filename, JSON.parse(data), doc);
});
} else
__list_append(filename, [], doc);
});
}
It's easy to get a bit pedantic with "best practices," but when I'm writing code and I get a gut feeling that something's not right or that something could be changed, I go over some well known best practices and attempt to see if the code I'm writing adheres to them. SOLID, while being principles of object oriented programming, can be useful to think about in other contexts. In this case, it seems to me that the function is violating the Single Responsibility Principle:
One of the most foundational principles of good design is:
Gather together those things that change for the same reason, and separate those things that change for different reasons.
This principle is often known as the Single Responsibility Principle or SRP. In short, it says that a subsystem, module, class, or even a function, should not have more than one reason to change.
(This could perhaps be exchanged for Separation of Concerns or other similar principles for this example, but the concept is the same.)
In this case, the function has two responsibilities: (1) getting the current (or default) list associated with a filename, and (2) appending data to said list. A first pass at separating these concerns might look something like this:
function get_current_list(filename, callback) {
fs.exists(filename, function(exists) {
if (exists) {
fs.readFile(filename, function(err, data) {
if (err)
return callback(err);
callback(null, JSON.parse(data));
});
} else
callback(null, []);
});
}
function list_append(filename, doc) {
get_current_list(filename, function(err, list) {
if(err) throw err;
__list_append(filename, list, doc);
});
}
Now, get_current_list is only responsible for getting the current list in a file (or an empty array if there is no file), and __list_append is (assumed to be) only responsible for appending to it; list_append is now a simple integration point between these two functions. The functions are a bit more reusable and can also be tested more easily (as an aside, a test-first or TDD approach to programming can help you notice these kinds of things up front). Furthermore, repeating callback in get_current_list is quite a bit more generic than repeating __list_append; if you need to change __list_append to something else, it now is only called in one place.
This case always feels unsatisfying to me because yes, you do have to repeat your call to __list_append on both branches because only one of the branches is synchronous.
I like and up voted Brandon's answer, but this also works:
function list_append(filename, doc) {
fs.exists(filename, function(exists) {
var data = [];
if (exists) {
data = fs.readFileSync(filename, "utf8");
}
__list_append(filename, data, doc);
});
}