I am trying to use async module on my nodejs application with not lucky.
Suppose i have the following:
/**
* parents=['parent1Template','parent2TEmplate',...]
* children=[[{id: 'child1Parent1'}{id: 'child2Parent1'}],[{id: 'child1Parent2"}{id: 'child2Parent2'}],...]
*
*/
function createTemplate(parents,children){
var final=[]
async.each(Object.keys(parents), function(item,done){
if(children[item].length!==0) addChildsByParent(parents[item],children[item], function (result) {
final.push(result);
});
done();
});
console.log("Final results: "+final);
return final;
}
function addChildsByParent (parent,childs,callback){
var template=[];
//some operations..
async.each(childs,function(child){
Children.findone({"_id": child.id}, function (err,ch)){
template.push(ch);
}
});
return callback(template)
}
I need to get in final all results when all operations have finished.
I have seen also functions as parallel and waterfall on async module but the main problem is that i need to work always with two arrays and do the query when i get the single value.
What's the best way, maybe something like this?
async.waterfall([
each()...
async.waterfall([
each()...
])
])
I would use async map instead of each to build the response array using the map callback.
Also, I would use async parallel instead of waterfall to improve speed, since operations don't depend on each other and can be executed in parallel.
async.parallel({
final : function(next) {
async.map(Object.keys(parents), function(item,done){
if(children[item].length!==0) addChildsByParent(parents[item],children[item], function (result) {
done(null, result);
});
}, next);
},
template : function(next) {
async.map(childs, function(child, done) {
Children.findone({"_id" : child.id}, function (err, ch) {
done(err, ch);
});
});
}
}, function(error, results){
if (!error) {
console.log(results);
// This will be {final:[], template:[]}
}
});
Related
I'm struggling right now with "async".
I'm starting with NestJs Framework and i need run a bunch of task on parallel.
Before, using Nodejs + Express only, i have been useing something like this
...
...
.get('/some/path', function(){
async.parallel({
todayBalance: async.apply(function (cb) {
return SomeFunction(params, cb);
}),
currentWeek: async.apply(function (cb) {
return SomeFunction2(params, cb);
}),
...
...
// more tasks
},
function (err, results) {
# some code for logic
# here i get my result and gather data and RETURN
});
});
Nowadays, using NestJs framework, i have got something like this
myservice.ts
This is a service created for doing this.
// Service Definitions
async someFunction(userBusinessId: string): Promise<any> {
// i tried to use same strategy from Async
// but nothing accuring
async.parallel({
todayBalance: async.apply(function (cb) {
return SomeFunction(params, cb);
}),
currentWeek: async.apply(function (cb) {
return SomeFunction2(params, cb);
}),
...
...
// more tasks
},
function (err, results) {
# some code for logic
# here i get my result and gather data and RETURN
# DOESNT RETURN, NEVER EVER GETS HERE
});
}
Any idea what's wrong?
Thanks for your support!
First off I made two basics mistake
1. I Forgot use cb Function
using Nodejs + Express we just made something like this
In this case I use mysql
const SomeFunction= (userBusinessId, cb) => {
cnx.connection.query(`some pretty query`,
cb // <===== use Callback here
);
};
now using Nestjs i tried(badly results) to made something like this. Ignoring cb
const SomeFunction= (userBusinessId, cb) => {
const data = await getManager().query(`some pretty query`);
return data; // <===== WRONG INSTEAD USE cb Function, using like this parallel function will never triggered
};
Instead return single data, we must trigger the cb function overloading with result
Documentation
const SomeFunction= (userBusinessId, cb) => {
...
cb(null, data); // EXECUTE LIKE THIS, this will resume the pipe. null param suppose no error
};
2. Try to return the function service value inside the Async callback
Even if you try to do something like
async list():Promise<any>{
async.parallel({
...
// more tasks
},
function (err, results) {
return data;
});
}
OR
async list():Promise<any>{
const data = await async.parallel({
...
// more tasks
},
function (err, results) {
return data;
});
return data;
}
this funtion with always return undefined. even if you remove the Funtion Type Promise<any>
For avoid this you must return a Promise like
async list():Promise<any>{
return new Promise( (resolver, reject) => {
async.parallel({
...
// more tasks
},
function (err, results) {
if(err)
reject(err);
...
resolver(results);
});
} )
}
Hope this help you too!
Let's say I have 3 files.
index.js makes a call to the backend like this
$.post('/test/', data, function(response) {
//handle success here
})
routes.js handles the route like this
app.post('/test/', function(req, res){
item.getItems(function(response){
res.json(response);
});
});
items.js is the model which accesses the database and makes a POST request for each item
function getItems(callback) {
database.query('SELECT * from items', function(result){
result.forEach(function(item){
request.post('/api/', item, function(req, res) {
//finished posting item
});
});
});
//callback here doesnt wait for calls to finish
}
where/when should I call the callback passed to getItems() to handle a success/failure in index.js?
Because your request.post() operations are async, you have to use some method of keeping track of when they are all done and then you can call your callback. There are multiple ways of doing that. I'll outline a few:
Manually Keeping Track of Count of Request Operations
function getItems(callback) {
database.query('SELECT * from items', function(result){
var remaining = result.length;
result.forEach(function(item){
request.post('/api/', item, function(err, res) {
--remaining;
//finished posting item
if (remaining === 0) {
callback();
}
});
});
});
}
The main problem with doing this manually is that propagating error in nested async operations is difficult when you get to actually figuring out how you're going to handle errors. This is much easier in the other methods shown here.
Using Promises
// load Bluebird promise library
var Promise = require('bluebird');
// promisify async operations
Promise.promisifyAll(request);
function queryAsync(query) {
return new Promise(function(resolve, reject) {
// this needs proper error handling from the database query
database.query('SELECT * from items', function(result){
resolve(result);
});
});
}
function getItems(callback) {
return queryAsync('SELECT * from items').then(function(result) {
return Promise.map(result, function(item) {
return request.postAsync('/api/', item);
});
});
}
getItems.then(function(results) {
// success here
}, function(err) {
// error here
})
It seems strange that you're making an API request in your server-side code, unless this is some sort of middle tier code that interacts with the API... but you're interacting with a database, so I'm still confused on why you can't just do a database insert, or have a bulk insert API call?
Anyway, if you must do it the way you're asking, I've done this in the past with a recursive method that trims down the result array... I really don't know if this is a good approach, so I'd like to hear any feedback. Something like this:
function recursiveResult(result, successfulResults, callback) {
var item = result.shift();
// if item is undefined, then we've hit the end of the array, so we'll call the original callback
if (item !== undefined) {
console.log(item, result);
// do the POST in here, and in its callback, call recursiveResult(result, successfulResults, callback);
successfulResults.push(item);
return recursiveResult(result, successfulResults, callback);
}
// make sure callback is defined, otherwise, server will crash
else if (callback) {
return callback(successfulResults);
}
else {
// log error... callback undefined
}
}
function getItems(callback) {
var successfulResults = [];
var result = [1, 2, 3, 4];
recursiveResult(result, successfulResults, callback);
}
console.log('starting');
getItems(function(finalResult) {
console.log('done', finalResult);
});
I'm just doing my first bit of Node async stuff, I wanted to do two queries to a DB and then print the results one after the other, my code is as follows:
console.log(req.query);
function logAllThings(err,things){
if (err){
console.log(err);
} else {
console.log(things);
};
};
async.parallel([
function(callback) { //This is the first task, and callback is its callback task
Event.find({}, function(err, events) {
logAllThings(err,events);
});
},
function(callback) { //This is the second task, and callback is its callback task
Organisation.find({}, function(err,organisations) {
logAllThings(err,organisations);
}); //Since we don't do anything interesting in db.save()'s callback, we might as well just pass in the task callback
}
], function(err) { //This is the final callback
console.log('Both should now be printed out');
});
The issue I have is that the second function (the one that returns organisations) prints them fine. However the one that is meant to return events simply does not and returns {} despite the fact I know the query works as I've tested it elsewhere.
Any help would be appreciated
Thanks
You have to call the callback function passed to each of your waterfall functions, otherwise it won't know when it's finished. Try this:
async.parallel([
function(callback) { //This is the first task, and callback is its callback task
Event.find({}, function(err, events) {
if (err) return callback(err);
logAllThings(err,events);
callback();
});
},
function(callback) { //This is the second task, and callback is its callback task
Organisation.find({}, function(err,organisations) {
if (err) return callback(err);
logAllThings(err,organisations);
callback();
}); //Since we don't do anything interesting in db.save()'s callback, we might as well just pass in the task callback
}
], function(err) { //This is the final callback
console.log('Both should now be printed out');
});
I've inherited some Node.js code and I need to add some functionality. However, I'm not sure syntactically how to accomplish my goal due to the asynchronous nature of Node. Currently, I have a function defined like this:
return {
myEntryPoint: function(req, res) {
var results = getResults();
res.send(200, results);
}
};
Several pieces are calling this function already. Inside of it, I'm calling a function called getResults which is defined like this:
var getResults = function() {
var results = [];
async.series([
function(callback) {
// add to results
},
function(callback) {
// add to results
}
]);
return results;
};
My problem is, I need to wait until all of the functions inside of the async.series call are made before I return the results. How do I do this?
You could change it to add a callback. Series has an optional callback to run once all the functions have completed within.
var getResults = function(finished) {
var results = [];
async.series([
function(callback) {
// add to results
},
function(callback) {
// add to results
}
], function() {
//Called when series is finished
finished(results);
});
};
And to get the results,
return {
myEntryPoint: function(req, res) {
getResults(function(results) {
res.send(200, results);
});
}
};
The usual way to handle this is to take a callback that uses the argument in question.
results = getResults(args);
someFunction(results);
Is equivalent to
function getResults(args, callback) {
var results;
//do stuff and populate results.
callback(results);
}
getResults(args, someFunction);
Except you don't have the issue of trying to wait for async stuff to happen, which you can't do, without weird logic.
I'm stuck when it comes to returning response, from method that includes database call.
Here's sample of what I need...
service.js
module.exports = {
getUserStatus: function(userId){
return User
.find()
.where({userId: userId)
.exec(function(err, user){
return user.status;
}
}
}
In this service.js, I should fetch user's status and if I console.log(user.status) inside exec method, that is printed OK (I got status).
The problem is I need this result outside service.js:
controller.js
// this code is extracted from longer file, just for demo purpose.
// userService is required 'service.js'
index: function(req, res) {
var status = userService.getUserStatus(req.session.User.id);
console.log(status);
return res.view({userStatus: status});
}
If I console.log(status) here, it will be undefined.
I guess that it has something to do with promises and stuff (because of the async calls), but not sure what is the right way to do it.
getUserStatus contains asynchronous code, so it needs a callback:
module.exports = {
getUserStatus: function(userId, cb){
User.findOne().where({userId: userId}).exec(function(err, user){
if (err) return cb(err);
return cb(null, user.status);
});
}
}
then in the code that uses it:
index: function(req, res) {
userService.getUserStatus(req.session.User.id, function(err, status) {
// If an error was returned from the service, bail out
if (err) {return res.serverError(err);}
console.log(status);
return res.view({userStatus: status});
});
}
Note the use of findOne instead of find; find will return an array.
An alternative would be to return a promise from the service function, and chain your controller code with .then() and .fail():
module.exports = {
getUserStatus: function(userId, cb){
return User.findOne().where({userId: userId});
}
}
index: function(req, res) {
userService.getUserStatus(req.session.User.id)
.then(function(user) {
console.log(user.status);
return res.view({userStatus: user.status});
})
.fail(function(err) {
return res.serverError(err);
});
});
}
It's a matter of preference, but I think the first method is better especially in your case, since it allows the service call to deliver just the status, rather than the whole user object. In general, getting used to the standard (err, result) callback method in Node will serve you well.