Nestjs Framework: Run parallel task using Async - node.js

I'm struggling right now with "async".
I'm starting with NestJs Framework and i need run a bunch of task on parallel.
Before, using Nodejs + Express only, i have been useing something like this
...
...
.get('/some/path', function(){
async.parallel({
todayBalance: async.apply(function (cb) {
return SomeFunction(params, cb);
}),
currentWeek: async.apply(function (cb) {
return SomeFunction2(params, cb);
}),
...
...
// more tasks
},
function (err, results) {
# some code for logic
# here i get my result and gather data and RETURN
});
});
Nowadays, using NestJs framework, i have got something like this
myservice.ts
This is a service created for doing this.
// Service Definitions
async someFunction(userBusinessId: string): Promise<any> {
// i tried to use same strategy from Async
// but nothing accuring
async.parallel({
todayBalance: async.apply(function (cb) {
return SomeFunction(params, cb);
}),
currentWeek: async.apply(function (cb) {
return SomeFunction2(params, cb);
}),
...
...
// more tasks
},
function (err, results) {
# some code for logic
# here i get my result and gather data and RETURN
# DOESNT RETURN, NEVER EVER GETS HERE
});
}
Any idea what's wrong?
Thanks for your support!

First off I made two basics mistake
1. I Forgot use cb Function
using Nodejs + Express we just made something like this
In this case I use mysql
const SomeFunction= (userBusinessId, cb) => {
cnx.connection.query(`some pretty query`,
cb // <===== use Callback here
);
};
now using Nestjs i tried(badly results) to made something like this. Ignoring cb
const SomeFunction= (userBusinessId, cb) => {
const data = await getManager().query(`some pretty query`);
return data; // <===== WRONG INSTEAD USE cb Function, using like this parallel function will never triggered
};
Instead return single data, we must trigger the cb function overloading with result
Documentation
const SomeFunction= (userBusinessId, cb) => {
...
cb(null, data); // EXECUTE LIKE THIS, this will resume the pipe. null param suppose no error
};
2. Try to return the function service value inside the Async callback
Even if you try to do something like
async list():Promise<any>{
async.parallel({
...
// more tasks
},
function (err, results) {
return data;
});
}
OR
async list():Promise<any>{
const data = await async.parallel({
...
// more tasks
},
function (err, results) {
return data;
});
return data;
}
this funtion with always return undefined. even if you remove the Funtion Type Promise<any>
For avoid this you must return a Promise like
async list():Promise<any>{
return new Promise( (resolver, reject) => {
async.parallel({
...
// more tasks
},
function (err, results) {
if(err)
reject(err);
...
resolver(results);
});
} )
}
Hope this help you too!

Related

NEDB NodeJS find document and wait for the result

I am using NEDB for some local storage in an NodeJS Application. Therefore I do have an handlerscript "nedbhandler.js" which I do require in my main.js.
var NEDB = require('./nedbhandler.js');
async function test(){
var value = await NEDB.getValue_byID(1);
console.log(value)
}
test()
while in the nedbhandler.js is my query handled like this:
async function getValue_byID(id){
db.config.findOne({ _id: id }, function (err, doc) {
callback(doc);
});
function callback(doc) {
console.log(doc)
return doc;
}
}
exports.getValue_byID = getValue_byID;
While the console from nedbhandler.js logs the expected value, the main.js still is undefined.
What would be the best practice to load all config querys before loading next function in the main.js?
You want your getValue_byID function to return a promise
function getValue_byID(id){
return new Promise((resolve, reject) => {
db.config.findOne({ _id: id }, (err, doc) => {
err ? reject(err) : resolve(doc);
});
});
}
Note how I changed the function keyword to the more modern => syntax.
It is also considered better practice to use const and let insteadof var
There is a wrapper module for nedb, called NeDB-promises. It does more or less what I have done here, so you can use async/await instead of callbacks.

how to iterate two arrays one after other using async forEach

I have two arrays. I would like to iterate the arrays using async.foreach.
But when I do so, only the second array is getting executed.How to execute both.
Below is my code:
var _sizesArray1 = [_2000px, _1400px]
var _sizesArray2 = [_800px, _400px, _200px, _100px]
async.forEachOf(_sizesArray1, function(value, key, callback) {
async.waterfall([
function download(next){
//code
},
function convert(response, next) {
//code
},
function process(response, next) {
gm(response).command('convert')
.resize(_sizesArray[key].width,_sizesArray[key].width)
.gravity('Center')
.extent(_sizesArray[key].width,_sizesArray[key].width)
.quality('20')
.toBuffer(
'JPG', function(err,
buffer) {
if (err) {
next(err);
} else {
console.timeEnd(
"processImage array1"
);
next(null,
buffer,
key);
}
});
}
});
async.forEachOf(_sizesArray2, function(value, key, callback) {
async.waterfall([
function download1(next){
//code
},
function convert2(response, next) {
//code
},
function process3(response, next) {
//code
}
});
In my code, only array2 is getting invoked.Why don't first one get executed.
Is there any mistake in my code. Can somebody help resolve this.
How about this simple technique:
var allSizes = _sizesArray1.concat(_sizesArray2);
async.foreach(allSizes, function(value, key, next) {
// do whatever you like with each item in the concatenated arrays
// once you are done move to the next item
next()
})
Updated based on comments
Version 1, based on callbacks (welcome callback hell):
function asyncIterate(array, onEach, onDone) {
async.forEach(array, (val, index, onNext) => {
onEach(val, key, function onDecoratedNext() {
// tell the async.forEach it's ready for the next item
onNext();
// but if it got to the end,
// then mark the completion of the whole iteration
if (index === array.length) {
onDone();
}
});
});
}
and implement it like:
function doAsyncStuffWithEachItem(val, index, cb) {
// do async stuff with each item
// make sure to mark the completion of the async operation
cb();
}
asyncIterate(
_sizesArray1,
doAsyncStuffWithEachItem,
function onDone() {
// once the 1st iteration finished, time to move to the 2nd array
asyncIterate(
_sizesArray2,
doAsyncStuffWithEachItem,
function onDone2() {
// great we are done
// What if we have _sizesArray3, 4, 5 ... ?
// Well, we can continue to nest more callback here
// but in that way we'll soon end up with callback hell
// and that's a big NoNo
}
)
}
);
Version 2, based on Promises:
To avoid callback hell, luckily we can use Promises. Something like this should do it:
const promisifiedAsyncIterate = (array, onEach) =>
new Promise((resolve) => {
asyncIterate(array, onEach, resolve);
});
and use it like:
promisifiedAsyncIterate(_sizeArray1, doAsyncStuffWithEachItem)
.then(() => promisifiedAsyncIterate(_sizeArray2, doAsyncStuffWithEachItem))
.then(() => promisifiedAsyncIterate(_sizeArray3, doAsyncStuffWithEachItem))
.then(() => promisifiedAsyncIterate(_sizeArray4, doAsyncStuffWithEachItem))
It could be abstracted and cleaned up even more, or even made completely dynamic – say you have an array of sizeArrays that you pass in to your function, but I think this is enough for now :). Hope it helps.

nodejs async two arrays and mongoose find

I am trying to use async module on my nodejs application with not lucky.
Suppose i have the following:
/**
* parents=['parent1Template','parent2TEmplate',...]
* children=[[{id: 'child1Parent1'}{id: 'child2Parent1'}],[{id: 'child1Parent2"}{id: 'child2Parent2'}],...]
*
*/
function createTemplate(parents,children){
var final=[]
async.each(Object.keys(parents), function(item,done){
if(children[item].length!==0) addChildsByParent(parents[item],children[item], function (result) {
final.push(result);
});
done();
});
console.log("Final results: "+final);
return final;
}
function addChildsByParent (parent,childs,callback){
var template=[];
//some operations..
async.each(childs,function(child){
Children.findone({"_id": child.id}, function (err,ch)){
template.push(ch);
}
});
return callback(template)
}
I need to get in final all results when all operations have finished.
I have seen also functions as parallel and waterfall on async module but the main problem is that i need to work always with two arrays and do the query when i get the single value.
What's the best way, maybe something like this?
async.waterfall([
each()...
async.waterfall([
each()...
])
])
I would use async map instead of each to build the response array using the map callback.
Also, I would use async parallel instead of waterfall to improve speed, since operations don't depend on each other and can be executed in parallel.
async.parallel({
final : function(next) {
async.map(Object.keys(parents), function(item,done){
if(children[item].length!==0) addChildsByParent(parents[item],children[item], function (result) {
done(null, result);
});
}, next);
},
template : function(next) {
async.map(childs, function(child, done) {
Children.findone({"_id" : child.id}, function (err, ch) {
done(err, ch);
});
});
}
}, function(error, results){
if (!error) {
console.log(results);
// This will be {final:[], template:[]}
}
});

async and Q promises in nodejs

I'm using the Q library and async library in nodejs.
Here's an example of my code:
async.each(items, cb, function(item) {
saveItem.then(function(doc) {
cb();
});
}, function() {
});
saveItem is a promise. When I run this, I always get cb is undefined, I guess then() doesn't have access. Any ideas how to work around this?
Your issue doesn't lie with promises, it lies with your usage of async.
async.each(items, handler, finalCallback) applies handler to every item of the items array. The handler function is asynchronous, i.e. it is handed a callback, that it must call when it has finished its work. When all handlers are done, the final callback is called.
Here's how you'd fix your current issue:
var handler = function (item, cb) {
saveItem(item)
.then(
function () { // all is well!
cb();
},
function (err) { // something bad happened!
cb(err);
}
);
}
var finalCallback = function (err, results) {
// ...
}
async.each(items, handler, finalCallback);
However, you don't need to use async for this particular piece of code: promises alone fill this job quite nicely, especially with Q.all():
// Create an array of promises
var promises = items.map(saveItem);
// Wait for all promises to be resolved
Q.all(promises)
.then(
function () { // all is well!
cb();
},
function (err) { // something bad happened!
cb(err);
}
)

Pass function to next function, fire depending on condition. nodejs

Im using express.js to create a node.js REST server, as a part o this i am also creating a simple session system. I have 3 modules:
app.js
highscoreMan.js
userSession.js
The app.js url for http://localhost/api/highscores now calls userSession with given parameters:
//Get all highscores
app.get('/api/highscores', function (req, res) {
userSession.checkValidity(req.query['username'], req.query['sessionid'], highscoreMan.getAll(req, res));
});
However, in checkValidity the function that i pass is automatically called:
function checkValidity(username, sessionId, callback) {
userSession.findOne({ userid: username, sessionid: sessionId }, function (err, result) {
if (err) {
console.log(err);
}
if(result) {
callback;
}
});
}
I only want to run the function being passed given that i get the proper results from the database(other checks will be added later for session dates etc.). How would i do this?
To delay calling highscoreMan.getAll(), you'll need to make it a statement of another function that can be called later:
app.get('/api/highscores', function (req, res) {
userSession.checkValidity(req.query['username'], req.query['sessionid'], function () {
highscoreMan.getAll(req, res);
});
});
Otherwise, it's being called immediately and its return value is instead being passed to userSession.checkValidity().
Note that you'll also need to adjust checkValidity to call the passed callback:
// ...
if(result) {
callback();
}
// ...
Unless I don't fully understand your problem, couldn't you just do something like this?
if (result && some_validator(result)) {
callback();
}

Resources