I am using express and async.js for a node application. I have this code in my app.js:
var resultObject = {};
app.get('/average', function(req, res) {
async.series([
function(callback) {
//This does some complex computations and updates the resultObject
routes.avg(req.query.url, resultObject);
console.log('1');
callback();
}
,
function(callback) {
res.send(resultObject);
console.log('2');
callback();
}
]);
});
The problem is that the res.send(...) fires before the complex computation in the first function finishes. Therefore, the object sent is empty. Is there some error in my code, or am I not using async.js correctly?
According to your comments, the problem is that your routes.avg method is asynchronous. This means that the method gets executed but doesn't block the program from continuing, which means that the 2nd closure in your series is being called almost immediately after the 1st one.
The problem isn't that the closures in the async.series call are being called out of order, it's that there's nothing that keeps your callback in closure 1 from being executed before routes.avg is finished.
To fix this, you could pass your callback into your routes.avg call, and edit the routes.avg method so that it calls the callback closure when all the calculation is done. That way, the 2nd closure in your series will not be executed until the callback passed to closure 1 is called by the routes.avg method.
If you don't have access to change the routes.avg method, then you have to find another way to detect when it's done it's work before you call the callback param passed to closure 1 from async.
Perhaps try something like this:
async.series([
function(callback) {
routes.avg(req.query.url, resultObject);
console.log('1');
callback();
},
function(callback) {
console.log('2');
callback();
}
],
function(err, results){
if(!err)
res.send(resultObject);
else
//do something here for error?
});
Related
So I'm making a web application and I'm trying to send variables to an EJS file but when they are sent out of the mongo functions they come out as undefined because it's a different scope for some reason. It's hard to explain so let me try to show you.
router.get("/", function(req, res){
var bookCount;
var userCount;
Books.count({}, function(err, stats){
if(err){
console.log("Books count failed to load.");
}else{
bookCount = stats;
}
});
User.count({}, function(err, count){
if(err){
console.log("User count failed to load.")
}else{
userCount = count;
console.log(userCount);
}
});
console.log(userCount);
//Get All books from DB
Books.find({}, function(err, allbooks){
if(err){
console.log("Problem getting all books");
}else{
res.render("index", {allbooks: allbooks, bookCount: bookCount, userCount: userCount});
}
});
});
So in the User.Count and Books.count I'm finding the number of documents in a collection which works and the number is stored inside of the variables declared at the very top.
After assigning the numbers like userCount i did console.log(userCount) which outputs the correct number which is 3, If was to do console.log(userCount) out of the User.count function it would return undefined, which is a reference to the declaration at the very top.
What is really weird is that Book.Find() has the correct userCount even though its a totally different function. The whole goal im trying to accomplish is doing res.render("index", {userCount: userCount}); outside of the Books.find(). I can do it but of course for some reason it passes undefined instead of 3. I hope this made a shred of sense.
I seem to have found a solution. but if anyone knows a different way I would love to know. So basically all you need to do is move the User.Count function outside of the router.get() function. Not completely sure about the logic of that but it works...
This is a classic asynchronous-operation problem: Your methods (Books.count, Books.find, User.count) are called immediately, but the callback functions you pass to them are not. userCount is undefined in your log because console.log is called before the assignment in the callback function is made. Your code is similar to:
var userCount;
setTimeout(function() {
userCount = 3;
}, 1000);
console.log(userCount); // undefined
User.count takes time to execute before calling back with the result, just like setTimeout takes the specified time to execute before calling its callback. The problem is JS doesn't pause and wait for the timeout to complete before moving on and calling console.log below it, it calls setTimeout, calls console.log immediately after, then the callback function is called one second later.
To render a complete view, you need to be sure you have all of the data before you call res.render. To do so you need to wait for all of the methods to call back before calling res.render. But wait, I just told you that JS doesn't pause and wait, so how can this be accomplished? Promise is the answer. Multiple promises, actually.
It looks like you are using Mongoose models. Mongoose has been written so that if you don't pass a callback function to your methods, they return a promise.
Books.count({}) // returns a promise
JS promises have a method then which takes a callback function that is called when the promise has been resolved with the value of the asynchronous method call.
Books.count({}) // takes some time
.then(function(bookCount) { // called when Books.count is done
// use the bookCount here
})
The problem is, you want to wait for multiple operations to complete, and multiple promises, before continuing. Luckily JS has a utility just for this purpose:
Promise.all( // wait for all of these operations to finish before calling the callback
Books.count({}),
User.count({}),
Books.find({})
)
.then(function(array) { // all done!
// the results are in an array
bookCount = array[0];
userC0unt = array[1];
allBooks = array[2];
})
I was reading this article on howtonode,
but don't get why it's fake async ?
So fake async is described as:
function asyncFake(data, callback) {
if(data === 'foo') callback(true);
else callback(false);
}
asyncFake('bar', function(result) {
// this callback is actually called synchronously!
});
Correct code: always async
function asyncReal(data, callback) {
process.nextTick(function() {
callback(data === 'foo');
});
}
My question is what's wrong with the first part of code ?
Why nextTick() can promise the 'right' effect ?...
Please explain to me. Thanks.
Nothing's wrong with the first case. However, it's just that you have to be proactive (as the developer) to know exactly what's happening in your code.
In the first case, you are not doing any I/O, and the callback is not actually being put into the Event Loop. It just the same as doing this:
if(data === 'foo')
return true;
else
return false;
However, in second case, you are putting the callback into the event loop until the next iteration.
In terms of how things work, nothing's wrong. However, you need to be aware of what implications are.
For instance:
function maybeSync(a, cb) {
if(a === 'a') {
cb('maybeSync called');
} else {
// put the called into event-loop for the next iteration
process.nextTick(function() {
cb('maybeSync called');
});
}
}
function definitelySync() {
console.log('definitelySync called');
}
someAsync('a', function(out) {
console.log(out);
});
definitelySync();
In the top case, which one gets called first?
If the input is "a" then the output is:
maybeSync called
definitelySync called
If the input is something else (e.g. not "a") then the output would be:
definitelySync called
maybeSync called
You need to make them consistent, as it would be easy to distinguish if the callback can be called sync/async based on the condition. Again comes back to being responsible, being consistent and being aware of what's happening in your code. =)
In the async fake function - you can see that callback is called as a normal function invocation i.e in the same call stack.
someone calls async fake fn
async fake fn calls the callback passed to it
If that callback takes time to complete - the async fake fn is kept waiting i.e it remains in call stack.
By using process tick - we are just submitting the callback function to be scheduled for execution.
This process tick call completes immediately and the asyncreal will return immediately. Thus the callback will be executed in asynchronous way.
Folks,
I have the following function, and am wondering whats the correct way to call the callback() only when the database operation completes on all items:
function mapSomething (callback) {
_.each(someArray, function (item) {
dao.dosomething(item.foo, function (err, account) {
item.email = account.email;
});
});
callback();
},
What I need is to iterate over someArray and do a database call for each element. After replacing all items in the array, I need to only then call the callback. Ofcourse the callback is in the incorrect place right now
Thanks!
The way you currently have it, callback is executed before any of the (async) tasks finish.
The async module has an each() that allows for a final callback:
var async = require('async');
// ...
function mapSomething (callback) {
async.each(someArray, function(item, cb) {
dao.dosomething(item.foo, function(err, account) {
if (err)
return cb(err);
item.email = account.email;
cb();
});
}, callback);
}
This will not wait for all your database calls to be done before calling callback(). It will launch all the database calls at once in parallel (I'm assuming that's what dao.dosomething() is). And, then immediately call callback() before any of the database calls have finished.
You have several choices to solve the problem.
You can use promises (by promisifying the database call) and then use Promise.all() to wait for all the database calls to be done.
You can use the async library to manage the coordination for you.
You can keep track of when each one is done yourself and when the last one is done, call your callback.
I would recommend options 1. or 2. Personally, I prefer to use promises and since you're interfacing with a database, this is probably not the only place you're making database calls, so I'd promisify the interface (bluebird will do that for you in one function call) and then use promises.
Here's what a promise solution could look like:
var Promise = require('bluebird');
// make promise version of your DB function
// ideally, you'd promisify the whole DB API with .promisifyAll()
var dosomething = Promise.promisify(dao.dosomething, dao);
function mapSomething (callback, errCallback) {
Promise.all(_.map(someArray, function(item) {
return dosomething(item.foo).then(function (account) {
item.email = account.email;
});
}).then(callback, errCallback);
}
This assumes you want to run all the DB calls in parallel and then call the callback when they are all done.
FYI, here's a link to how Bluebird promisify's existing APIs. I use this mechanism for all async file I/O in node and it saves a ton of coding time and makes error handling much more sane. Async callbacks are a nightmare for proper error handling, especially if exceptions can be thrown from async callbacks.
P.S. You may actually want your mapSomething() function to just return a promise itself so the caller is then responsible for specifying their own .then() handler and it allows the caller to use the returned promise for their own synchronization with other things (e.g. it's just more flexible that way).
function mapSomething() {
return Promise.all(_.map(someArray, function(item) {
return dosomething(item.foo).then(function (account) {
item.email = account.email;
});
})
}
mapSomething.then(mapSucessHandler, mapErrorHandler);
I haven't tried Bluebird's .map() myself, but once you've promisified the database call, I think it would simplify it a bit more like this:
function mapSomething() {
return Promise.map(someArray, function(item) {
return dosomething(item.foo).then(function (account) {
item.email = account.email;
});
})
}
mapSomething.then(mapSucessHandler, mapErrorHandler);
I need to call 3 functions in series x1() then x2() then x3(). But since x1 and x2 are time consuming operations x3 executes before them giving unexpected results. I have used the async library in NodeJS with the series method as below.
How can I solve this without using setTimeout for x3()
async.series([function(callback) { x1();callback(null, null);},
function(callback) { x2();callback(null, null);},
function(callback) {
x3();
callback(null, null); }],
function(err, results) { }
);
use recursive try catch..
a=true;
while(a){
try{
your work.....
a=false;
}catch(exp ){
a=true
}
}
You can use javascript's async nature to solve it. Here how it's done with callbacks. Call x3() instead of onMethodsDone().
Here, suppose x1() accepts an array. Forget it if you don't need any parameters. x3() gets the modified array from x2() which is also another time consuming function. setTimeout() is used to show the async behaviour since x1() and x2() are time consuming.
x1([1,2],function(results){
x2(results,onMethodsDone);
});
function onMethodsDone(results){
alert(results);
}
function x1(records,cb){
setTimeout(function (){
records.push(3,4,5);
cb(records); //parse updated records to callback
},2000);
}
function x2(records,cb){
setTimeout(function (){
records.push(6,7,8);
cb(records);
},2000);
}
javascripts Listeners or Promise/defers also can be used to handle it.
As the callback function had to be overloaded with different number of arguments the best option was to structure the method as follows
function callback(cbargs){
var arg1 = cbargs.cbarg1;
var arg2 = cbargs.cbarg2;
if(arg3!=null){
.....
}
}
function x1(arg1 , arg2, callback, cbargs){
.....
.....
callback(cbargs);
}
function x3(arg1 , arg2, callback, cbargs){
.....
.....
callback(cbargs);
}
x1(arg1, arg2,callback,{cbarg1, cbarg2});
x1(arg1, arg2,callback,{cbarg1, cbarg2, cbarg3});
I'm using the node-soap client from milewise (create API by the way), but I have some difficulties to get the results of the callback to the right scope.
Here is the code I have for now:
function generateSoapRequest(req, res, next)
{
soap.createClient('http://127.0.0.1:' + cfg.service_port + cfg.service_url_path,
function(err, client) {
client.CXIf.CXIf.CXProcessXML(
{"XMLRequestData": {"CXLogon": {"UserID":"1901007", "Password":"2580", "LogonType":11 } } },
function(err, result, body) {
if (err) {
console.log(err);
return;
}
console.log(result);
var cxresponse = result.XMLResponse[0].CXResponse;
console.log('SessionID:' + cxresponse.SessionID + ' SessionInstanceID:' + cxresponse.SessionInstanceID);
});
});
}
function getVoiceMailInformation(req, res, next) {
var cxresponse = generateSoapRequest(req, res, next);
var something = doSomethingNext(cxresponse);
}
function doSomethingNext(cxresponse){....; return something;}
Basically, when I launch the getVoiceMailInformation(), it creates a soap client and request some information through the generateSoapRequest().
The next step would be to get the result of that function (not implemented in the code above, because I don't know how) and do something else.
My problem is soap.createClient is asynchronous, so the callback is fired well after the function is complete.
What would be the best approach ?
(Maybe it's something trivial, but the scope of an anonymous function in javascript is something that is killing me.)
Any help will be very appreciated.
Basically you can't do something like:
var cxresponse = generateSoapRequest(req, res, next);
because the function you're calling invokes asynchronous code, and therefore can't return a value that's determined by that code. The normal way around this is to give the function an extra callback parameter for a function that will be called with the result once the result becomes available. It doesn't have to be an anonymous function; it can be a named function. In your case, (assuming you've modified generateSoapRequest to take a callback as its fourth argument and call it when the results are ready, you could write
generateSoapRequest(req, res, next, doSomethingNext);
and then doSomethingNext will be called with cxresponse as an argument. Of course, since doSomethingNext also gets called asynchronously, it can't return a value either, so you'll have to apply the same technique to it.
The async module can make this sort of thing easier: in particular, its "waterfall" pattern is useful when you have a bunch of functions that have to run in sequence, each being called back from the previous one.