According to the meteor docs, inserts block:
On the server, if you don't provide a callback, then insert blocks
until the database acknowledges the write, or throws an exception if
something went wrong. If you do provide a callback, insert still
returns the ID immediately.
So this would be wrong:
Meteor.methods({
post: function (options) {
return Stories.insert(options)
}
});
I need to do this:
Meteor.methods({
post: function (options) {
return Stories.insert(options, function(){})
}
});
Can somebody confirm that this is the case? The former will block the ENTIRE SERVER until the db returns?
Yeah, it will block, but not the entire server.
In Meteor, your server code runs in a single thread per request, not in the asynchronous callback style typical of Node. We find the linear execution model a better fit for the typical server code in a Meteor application.
So, if you are worried about that it will block the entire server as it will do in typical Node, don't be.
Related
I am new to nodejs so I have a basic question and this is my scanrio
I have a javascript client which is making a http request to a node server to read a value from the database.
Once the node server receives the request it makes a simple db call and returns the data to the client in the response, and this is where the problem is.
router.get('/state', function(req, res){
var result = dbServer.makeDBCall();//Before this line executes and returns the result the next line executes
res.send(result);
}
The database call from the node server is asynchronous, therefore before the result is returned the node server has already sent a blank response to the client. What is the standard/acceptable way of getting this achieved, I know I can block the node thread using async, but then the whole purpose of node is gone right?
It depends on what kind of database node module you are using.
Other than the standard callback approach, there are also the promise way. The pg-promise library is 1 of those kind.
See sample code:
this.databaseConnection.makeDBCall('your query...')
.then(function(dbResponse) {
// Parse the response to the format you want then...
res.send(result);
})
.catch(function(error) {
// Handle error
res.send(error.message);
});
#spdev : I saw 1 of your comments about you being worried about how Node actually knows who to reply the response to, especially when there are multiple requests.
This is a very good question, and to be honest with you - I don't know much about it as well.
In short the answer is yes, Node somehow handles this by creating a corresponding ServerResponse object when a HTTP request comes through. This object seems to have some smartness to tell the Nodejs network stack how to route itself back to the caller when it gets parsed as data packets.
I tried Googling a bit for an answer but didn't got too far. I hope the ServerResponseObject documentation can provide more insight for you. Share with me if you got an answer thanks!
https://nodejs.org/api/all.html#http_class_http_serverresponse
Try below code.
router.get('/state', function(req, res){
var result = dbServer.makeDBCall(function(err,result){
if(!err) {
res.send(result);
}
});
}
Hope this Help.
The dbServer.makeDBCall(); must have a callback that runs when the statement completes executing.
Something like -
dbServer.makeDBCall({query: 'args'}, function(err, result){
if (err) // handle error
res.send(result);
})
You return the response from db from that callback function.
Learn more about callback from here-
nodeJs callbacks simple example
https://docs.nodejitsu.com/articles/getting-started/control-flow/what-are-callbacks/
I have created a simple web interface for vertica.
I expose simple operation above a vertica cluster.
one of the functionality I expose is querying vertica.
when my user enters a multi-query the node modul throws an exception and my process exits with exit 1.
Is there any way to catch this exception?
Is there any way overcome the problem in a different way?
Right now there's no way to overcome this when using a callback for the query result.
Preventing this from happening would involve making sure there's only one query in the user's input. This is hard because it involves parsing SQL.
The callback API isn't built to deal with multi-queries. I simply haven't bothered implementing proper handling of this case, because this has never been an issue for me.
Instead of a callback, you could use the event listener API, which will send you lower level messages, and handle this yourself.
q = conn.query("SELECT...; SELECT...");
q.on("fields", function(fields) { ... }); // 1 time per query
q.on("row", function(row) { ... }); // 0...* time per query
q.on("end", function(status) { ... }); // 1 time per query
I'm still learning the node.js ropes and am just trying to get my head around what I should be deferring, and what I should just be executing.
I know there are other questions relating to this subject generally, but I'm afraid without a more relatable example I'm struggling to 'get it'.
My general understanding is that if the code being executed is non-trivial, then it's probably a good idea to async it, as to avoid it holding up someone else's session. There's clearly more to it than that, and callbacks get mentioned a lot, and I'm not 100% on why you wouldn't just synch everything. I've got some ways to go.
So here's some basic code I've put together in an express.js app:
app.get('/directory', function(req, res) {
process.nextTick(function() {
Item.
find().
sort( 'date-modified' ).
exec( function ( err, items ){
if ( err ) {
return next( err );
}
res.render('directory.ejs', {
items : items
});
});
});
});
Am I right to be using process.nextTick() here? My reasoning is that as it's a database call then some actual work is having to be done, and it's the kind of thing that could slow down active sessions. Or is that wrong?
Secondly, I have a feeling that if I'm deferring the database query then it should be in a callback, and I should have the actual page rendering happening synchronously, on condition of receiving the callback response. I'm only assuming this because it seems like a more common format from some of the examples I've seen - if it's a correct assumption can anyone explain why that's the case?
Thanks!
You are using it wrong in this case, because .exec() is already asynchronous (You can tell by the fact that is accepts a callback as a parameter).
To be fair, most of what needs to be asynchronous in nodejs already is.
As for page rendering, if you require the results from the database to render the page, and those arrive asynchronously, you can't really render the page synchronously.
Generally speaking it's best practice to make everything you can asynchronous rather than relying on synchronous functions ... in most cases that would be something like readFile vs. readFileSync. In your example, you're not doing anything synchronously with i/o. The only synchronous code you have is the logic of your program (which requires CPU and thus has to be synchronous in node) but these are tiny little things by comparison.
I'm not sure what Item is, but if I had to guess what .find().sort() does is build a query string internally to the system. It does not actually run the query (talk to the DB) until .exec is called. .exec takes a callback, so it will communicate with the DB asynchronously. When that communication is done, the callback is called.
Using process.nextTick does nothing in this case. That would just delay the calling of its code until the next event loop which there is no need to do. It has no effect on synchronicity or not.
I don't really understand your second question, but if the rendering of the page depends on the result of the query, you have to defer rendering of the page until the query completes -- you are doing this by rendering in the callback. The rendering itself res.render may not be entirely synchronous either. It depends on the internal mechanism of the library that defines the render function.
In your example, next is not defined. Instead your code should probably look like:
app.get('/directory', function(req, res) {
Item.
find().
sort( 'date-modified' ).
exec(function (err, items) {
if (err) {
console.error(err);
res.status(500).end("Database error");
}
else {
res.render('directory.ejs', {
items : items
});
}
});
});
});
We are using node-mysql and I'm exposing createPool of mysql where from any file, we can require the database file and obtain a connection like this
var db = ("./database");
db(function(err, connection) {
//use connection to do stuff
}
I noticed that people don't always remember to call connection.end() that would return the connection to the pool and then we hit the limit..
How can I design the acquiring of the connection so that no matter when they decide to terminate the callback function that the connection.end() is called? I can't figure out a way to make a single place where we can do this so developers are only concerned with getting the connection..
I don't know the createPool from mysql, but can't you just wrap it?
People will provide a function stuff(err, connection) { ... do whatever they want ... }.
Why don't you take that function and create a save_stuff-function? Something like
function save_stuff_creator(stuff) {
return function(err, connection) { stuff(err, connection); connection.end() }
}
You maybe want some try..catch around the stuff()-call.
If you want the connection to stay with someone for some other callbacks, you could modify the function to something like stuff(connection, callback) and use a module like async and a series-Object.
But I have no idea how to enforce a final call of connection.end()if you want to wait for the end of "the user thread": that is actually the issue. There is no such thread and there is no way of figuring out which event in the event loop comes from whom. As far as I know the event loop is not exposed to the js-developer.
If you can't trust your callee, maybe you can put his code in an extra child node (see module cluster. So you have a bit of control: when the child finishes you become notified.
In my app (node / express / redis), I use some code to update several items in DB at the same time:
app.put('myaction', function(req, res){
// delete stuff
db.del("key1");
db.srem("set1", "test");
// Add stuff
db.sadd("set2", "test2");
db.sadd("set3", "test3");
db.hmset("hash1", "k11", "v11", "k21", "v21");
db.hmset("hash2", "k12", "v12", "k22", "v22");
// ...
// Send response back
res.writeHead(200, {'content-type': 'application/json'});
res.write(JSON.stringify({ "status" : "ok" }));
res.end();
});
Can I be sure ALL those actions will be performed before the method returns ? My concern is the asynchronous processing. As I do not use callback function in the db actions, will this be alright ?
While all of the commands are sent and responses parsed asynchronously, it's useful to note that the callbacks are all invoked in order. So you can use the callback of the last Redis command to send the response to the client, and then you'll know that all of the Redis commands have been executed before responding.
Use the MULTI/EXEC command to create a queue of your commands and execute them in a row. Then use a callback to send a coherent response back (success/failure).
Note that you must use Redis' AOF to avoid that - in case of crash - the db state is not coherent with your logic because only a part of the commands in the queue were executed: i.e. MULTI/EXEC is not transactional upon execution. This is a useful reference.
I haven't worked with redis, but if this works(if you it doesn't call undefined function) and it should be asynchronous, then you can use it. But if there is an error in updating, then you can't handle it, this way.
No, you can't be sure if all those actions complete successfully, because your redis server might crash.. To speed things up, you can group all your update commands into one with pipelining (does your redis driver support that?), then get the success or failure of the whole operation via a callback and proceed..