I'm using the Q library and async library in nodejs.
Here's an example of my code:
async.each(items, cb, function(item) {
saveItem.then(function(doc) {
cb();
});
}, function() {
});
saveItem is a promise. When I run this, I always get cb is undefined, I guess then() doesn't have access. Any ideas how to work around this?
Your issue doesn't lie with promises, it lies with your usage of async.
async.each(items, handler, finalCallback) applies handler to every item of the items array. The handler function is asynchronous, i.e. it is handed a callback, that it must call when it has finished its work. When all handlers are done, the final callback is called.
Here's how you'd fix your current issue:
var handler = function (item, cb) {
saveItem(item)
.then(
function () { // all is well!
cb();
},
function (err) { // something bad happened!
cb(err);
}
);
}
var finalCallback = function (err, results) {
// ...
}
async.each(items, handler, finalCallback);
However, you don't need to use async for this particular piece of code: promises alone fill this job quite nicely, especially with Q.all():
// Create an array of promises
var promises = items.map(saveItem);
// Wait for all promises to be resolved
Q.all(promises)
.then(
function () { // all is well!
cb();
},
function (err) { // something bad happened!
cb(err);
}
)
Related
I'm struggling right now with "async".
I'm starting with NestJs Framework and i need run a bunch of task on parallel.
Before, using Nodejs + Express only, i have been useing something like this
...
...
.get('/some/path', function(){
async.parallel({
todayBalance: async.apply(function (cb) {
return SomeFunction(params, cb);
}),
currentWeek: async.apply(function (cb) {
return SomeFunction2(params, cb);
}),
...
...
// more tasks
},
function (err, results) {
# some code for logic
# here i get my result and gather data and RETURN
});
});
Nowadays, using NestJs framework, i have got something like this
myservice.ts
This is a service created for doing this.
// Service Definitions
async someFunction(userBusinessId: string): Promise<any> {
// i tried to use same strategy from Async
// but nothing accuring
async.parallel({
todayBalance: async.apply(function (cb) {
return SomeFunction(params, cb);
}),
currentWeek: async.apply(function (cb) {
return SomeFunction2(params, cb);
}),
...
...
// more tasks
},
function (err, results) {
# some code for logic
# here i get my result and gather data and RETURN
# DOESNT RETURN, NEVER EVER GETS HERE
});
}
Any idea what's wrong?
Thanks for your support!
First off I made two basics mistake
1. I Forgot use cb Function
using Nodejs + Express we just made something like this
In this case I use mysql
const SomeFunction= (userBusinessId, cb) => {
cnx.connection.query(`some pretty query`,
cb // <===== use Callback here
);
};
now using Nestjs i tried(badly results) to made something like this. Ignoring cb
const SomeFunction= (userBusinessId, cb) => {
const data = await getManager().query(`some pretty query`);
return data; // <===== WRONG INSTEAD USE cb Function, using like this parallel function will never triggered
};
Instead return single data, we must trigger the cb function overloading with result
Documentation
const SomeFunction= (userBusinessId, cb) => {
...
cb(null, data); // EXECUTE LIKE THIS, this will resume the pipe. null param suppose no error
};
2. Try to return the function service value inside the Async callback
Even if you try to do something like
async list():Promise<any>{
async.parallel({
...
// more tasks
},
function (err, results) {
return data;
});
}
OR
async list():Promise<any>{
const data = await async.parallel({
...
// more tasks
},
function (err, results) {
return data;
});
return data;
}
this funtion with always return undefined. even if you remove the Funtion Type Promise<any>
For avoid this you must return a Promise like
async list():Promise<any>{
return new Promise( (resolver, reject) => {
async.parallel({
...
// more tasks
},
function (err, results) {
if(err)
reject(err);
...
resolver(results);
});
} )
}
Hope this help you too!
I need the callback function of find from Node.js mongodb 3.1.6 to be triggered before the return statement of an async function, however the return statement is called before the callback function even-though there is a wait in front.
async function(myId) {
const myObject = MyObject()
await collection.find({where: {id: myId}}, async (err, results) => {
if (err) {
logger.error('error');
}
myObject.add(results);
});
return myObject
}
I have seen some examples where instead of find(query, callback) the pattern find(query).toArray() was used. But this doesn't run at all in my case. We use Node.js mongodb 3.1.6 with loopback-connector-mongodb maybe this is related to the problem.
If mongo doesn't provide a promise-answering function, then promisify this one yourself. Neither that promise-creating wrapper nor the anonymous callback it uses should be declared async, but the caller should....
function findById(collection, myId) {
return new Promise((resolve, reject) => {
collection.find({where: {id: myId}}, (err, results) => {
(err)? reject(err): resolve(results);
});
});
}
// now callers can use the async await pattern...
async someFunction() {
try {
let myId = // ...
let collection = // ...
let results = await findById(collection, myId);
// do something with results
} catch (err) {
// error
}
}
The key idea is that collection.find with the callback isn't eligible for await, because it doesn't return a promise. The anonymous callback function you pass to it isn't an async function... it does its work right away, as soon as find calls it back. So we build a promise around mongo, then use the new async/await syntax with that promise.
I tried aysnc.eachOfSeries but the code does not loop. It stops executing in the first iteration itself and freezes. I guess I made some error in returning the callback.
I also tried putting a callback inside the else block but gives callback already called error.
This async.eachOfSeries is nested inside one more eachOfSeries.
async.eachOfSeries(data, function (value2, val, callback) {
let jsonObj = data[val];
let email = jsonObj.ToEmail;
jsonObj['retailer'] = res[camp].retailer;
jsonObj['summary'] = 'f';
let tempObj = {};
tempObj[id] = jsonObj;
let options = { new: true};
let campId = id;
User.addCampaignResponse(email, campId, tempObj, options, function (err, results) {
if (err) {
throw err;
} else {
console.log("gets printed once");
Campaign.updateResponse(_id, function (err, results2) {
if (err)
throw err;
else {
console.log("Gets printed once");
callback(); //tried this but gives callback already called error
}
}) // console.log(results);
}
})
}, function (err) {
console.log("Doesn't print");
callback();
})
You must use it like this:
//async.eachOfSeries(data, function (value2, val, callback) { => wrong
async.eachOfSeries(data, function (value2, callback) {
//
})
Thought I'd leave this here in case someone else hits this thread.
The problem is that on your optional callback at the end:
}, function (err) {
console.log("Doesn't print");
callback();
});
You're calling the iterator callback and not an outer callback.
In the context of your code you probably have your async inside other function which callback is the one you should call on your async's optional callback at the end. Or just don't use the optional function, but never let it go through your iterator callback after ending the loop.
You should also test for error, something like:
}, function(err){
if(err){
debug('Something went wrong in async: ' +err);
outerCallback(err, somethingIfitsthecase);
}else{
outerCallback(null, somethingIfitsthecase);
}
});
Check caolan's documentation as Parth Acharya recommended.
Thanks
Best regards
Pedro Velez Silva
It is supposed to work that way.
if an error occurs, which is happening in your case. It runs the final callback function that you've provided.
Link :- https://caolan.github.io/async/docs.html#eachOfSeries
A callback which is called when all iteratee functions have finished,
or an error occurs. Invoked with (err).
I had a similar dilemma and I solved it by adding a callback in the async param and calling it at the end of the method as shown below.
async.eachOfSeries(data, async (item, index, callback: any) => {
// method body
callback();
})
Let's say I have 3 files.
index.js makes a call to the backend like this
$.post('/test/', data, function(response) {
//handle success here
})
routes.js handles the route like this
app.post('/test/', function(req, res){
item.getItems(function(response){
res.json(response);
});
});
items.js is the model which accesses the database and makes a POST request for each item
function getItems(callback) {
database.query('SELECT * from items', function(result){
result.forEach(function(item){
request.post('/api/', item, function(req, res) {
//finished posting item
});
});
});
//callback here doesnt wait for calls to finish
}
where/when should I call the callback passed to getItems() to handle a success/failure in index.js?
Because your request.post() operations are async, you have to use some method of keeping track of when they are all done and then you can call your callback. There are multiple ways of doing that. I'll outline a few:
Manually Keeping Track of Count of Request Operations
function getItems(callback) {
database.query('SELECT * from items', function(result){
var remaining = result.length;
result.forEach(function(item){
request.post('/api/', item, function(err, res) {
--remaining;
//finished posting item
if (remaining === 0) {
callback();
}
});
});
});
}
The main problem with doing this manually is that propagating error in nested async operations is difficult when you get to actually figuring out how you're going to handle errors. This is much easier in the other methods shown here.
Using Promises
// load Bluebird promise library
var Promise = require('bluebird');
// promisify async operations
Promise.promisifyAll(request);
function queryAsync(query) {
return new Promise(function(resolve, reject) {
// this needs proper error handling from the database query
database.query('SELECT * from items', function(result){
resolve(result);
});
});
}
function getItems(callback) {
return queryAsync('SELECT * from items').then(function(result) {
return Promise.map(result, function(item) {
return request.postAsync('/api/', item);
});
});
}
getItems.then(function(results) {
// success here
}, function(err) {
// error here
})
It seems strange that you're making an API request in your server-side code, unless this is some sort of middle tier code that interacts with the API... but you're interacting with a database, so I'm still confused on why you can't just do a database insert, or have a bulk insert API call?
Anyway, if you must do it the way you're asking, I've done this in the past with a recursive method that trims down the result array... I really don't know if this is a good approach, so I'd like to hear any feedback. Something like this:
function recursiveResult(result, successfulResults, callback) {
var item = result.shift();
// if item is undefined, then we've hit the end of the array, so we'll call the original callback
if (item !== undefined) {
console.log(item, result);
// do the POST in here, and in its callback, call recursiveResult(result, successfulResults, callback);
successfulResults.push(item);
return recursiveResult(result, successfulResults, callback);
}
// make sure callback is defined, otherwise, server will crash
else if (callback) {
return callback(successfulResults);
}
else {
// log error... callback undefined
}
}
function getItems(callback) {
var successfulResults = [];
var result = [1, 2, 3, 4];
recursiveResult(result, successfulResults, callback);
}
console.log('starting');
getItems(function(finalResult) {
console.log('done', finalResult);
});
I have a loop that I need to run inside of a callback, unfortunately accessing the array outside of the callback leaves me with a blank array. I know why this happens, but I want to know the best solution to tackle this.
Gallery.prototype.getGallery = function(cb) {
self = this;
var cos = new pb.CustomObjectService();
var ms = new pb.MediaService();
var s = [];
cos.loadTypeByName('Gallery Image', function(err, gallery){
cos.findByType(gallery._id.toString(), function(err, rpy){
for(var i = 0; i < rpy.length; i++){
ms.loadById(rpy[i].Image, function(e,r){
s.push(r.location);
console.log(r.location); /* <-- logs expected data */
});
}
console.log(s[0]); /* <-- this is undefined */
});
});
};
Replace your for loop with a call to async.*; in this case async.map seems right. Pass a callback to async.map; it will be invoked when all the individual calls to ms.loadById are done, with the array of results.
async.map(
rpy,
function(elt, callback) {
ms.loadById(elt.Image, callback);
},
function(err, data) {
// comes here after all individual async calls have completed
// check errors; array of results is in data
}
);
If you want to go into the promises world, then wrap the calls to ms.loadById in a promise. Here's a roll-your-own version, but various versions of what is usually called promisify are also out there.
function loadByIdPromise(elt) {
return new Promise(function(resolve, reject) {
ms.loadById(elt.image, function(err, data) {
if (err) return reject(err);
resolve(data);
});
});
}
Then do a Promise.all on the resulting promises:
Promise.all(rpy.map(loadByIdPromise))
.then(function(data) {
// comes here when all individual async calls complete successfully
// data is your array of results
});
Using the promises style, your entire code would look like:
loadTypeByNamePromise('Gallery Image') .
then(function(gallery) { return findByTypePromise(gallery._id.toString(); }) .
then(function(rpy) { return Promise.all(rpy.map(loadByIdPromise)); }) .
then(function(results) { /* do something with [results] */ });