I'm trying to send my node server multiple entries in order to update every person inside the database.
I thought a way to do it would be to loop through each person through their unique Ids and save the updated information based on this.
However, inside the Employee.findById function I cannot access the value of [i] so I cannot get the relevant employee. Running this line of code when trying to modify 2 Employees var i will output;
router.patch('/edit', function(req, res, next) {
for (var i = 0; i < req.body.length; i++) {
console.log("outside " + i)
Employee.findById(req.body[i].employeeId, function(err, employee) {
console.log("inside " + i)
/*
employee = new Employee({
name: req.body[i].name,
})
employee.save();
*/
})
}
})
I'm not sure why the console.log("inside " + i) isn't outputting the same number as the outside log?
Also, I'm not sure if the approach I am taking is the correct way of doing this?
Thanks for any advice!
When your /edit route is called, the for loop runs, logs out the "outside" statement and calls the Employee.findById method for each element in req.body, which will execute and call the "inside" console log statement when the find operation has finished. Note that this is an asynchronous operation.
The i variable does not change inside the callback of the Employee.findById call. This is because the for loop already incremented i to the number of elements in req.body.
By doing this, you will not be able to tell if all operations finished or an error occured while saving the employee documents and forward this information to the client in the response.
I would suggest using async or another flow controll library.
Your example using async:
router.patch('/edit', function(req, res, next) {
async.each(req.body, function(obj, cb) {
Employee.findById(obj.employeeId, function(err, employee) {
if (err) {
return cb(err);
}
employee = new Employee({name: obj.name});
employee.save(cb);
});
}, function (err) {
// any errors from callback will be handled here
// inform you user in the response or whatever
});
});
It's because the function findById() is asynchronous. The iteration of the loop doesn't wait findById() terminated the query.
I suggest you in order to fix this problem to use the lib Async.
It's provide some useful methods for doing things in asynchronous and wait the response.
In your case something like that can fix the problem:
async.each(req.body, function (value, callback) {
Employee.findById(value.employeeId, function(err, employee) {
employee = new Employee({
name: value.name,
})
// The callback indicate that this iteration is terminated.
employee.save(callback);
});
}, function (err) {
// When all callback() are triggered we pass here, and this tells we finished all operation.
});
I suggest you to read carefully the documentation of each
Related
I query one collection (messages) with mongoose. The result is an array of documents. Each document contains an ID for a different collection (users). Now I want to query the users collection for each ID from the messages collection.
The idea is to update each message object with the information from the user collection before returning it to the front end.
I tried using async.each. For some reason the final function is never called even though I am making sure the callback() function is called after each iteration.
app.get('/getmsg', function(req, res){
messages.find({query})
.exec(function(err, ms){
if(err) throw err;
async.each(ms, function(m, callback){
users.findOne({_id : m.userId})
.lean()
.exec(function(err, user){
if(err) {
console.log('error' , err);
callback();
} else {
m.userName = user.name;
// everything is working up to here
callback();
}
}), function(err){
res.send(ms); // this is never returned!
}
});
});
});
Is there a better way of achieving this? I assume this must be a common issue.
Thanks!
You can't use res.send. Instead create a function to get notified about it. Something like this.
// 1st para in async.each() is the array of items
async.each(items,
// 2nd param is the function that each item is passed to
function(item, callback){
// Call an asynchronous function, often a save() to DB
item.someAsyncCall(function (){
// Async call is done, alert via callback
callback();
});
},
// 3rd param is the function to call when everything's done
function(err){
// All tasks are done now
doSomethingOnceAllAreDone();
}
);
I've got an Express GET request that pulls data from a Mongoose query and for each doc returned, external functions perform calculations on each doc passed as x and return the results so I can render them on the front-end using Handlebars. Ideally, I would like to carry out my calculations calcA, calcB, calcC, calcD and then once they're completed for each of the documents, render the test-env.hbs template. Currently, the page doesn't render when I call it, in the logs, after some time, it shows GET /test-env - - ms - -, presumably because the callback gets stuck somewhere.
Express GET Request
var updates = require('./updates.js');
app.get('/test-env', function(req, res, next){
Market.find({"marketname" : 'To Win'})
.then(function(doc){
async.forEach(doc, function(x, callback){
updates.calcA(x);
updates.calcB(x);
updates.calcC(x);
updates.calcD(x);
}, function(err){
if(err)
return console.log(err);
res.render('test-env', {title: 'Test Page', items: doc});
});
});
});
Typical calc Function
I want to add returnA and returnB to the docs data so I can render it in an {{#each}} expression in Handlebars
calcA: function(x) {
Market.find({"student": x.student, "marketname": x.marketname})
.sort({btotal: -1})
.limit(1)
.then(function(ret) {
var valueA = ret[0].btotal;
var valueB = ret[0].back;
Market.find({"student": x.student, "marketname": x.marketname, "back": {$lt: valueB}})
.sort({odds: -1})
.limit(1)
.then(function(doc) {
var res = doc[0];
if (res == null) {
x.returnA = 0;
x.returnB = 0;
} else {
x.returnA = res.back;
x.returnB = res.btotal;
}
});
});
}
You aren't calling your callback in your async iterator. If the calc functions are asynchronous, those callbacks should be handled individually. If they are not asynchronous, a normal forEach will do the trick.
...
async.forEach(doc, function(x, callback){
updates.calcA(x);
updates.calcB(x);
updates.calcC(x);
updates.calcD(x);
callback(); // Call the callback to move on to the next item
}, function(err){
if(err)
return console.log(err);
res.render('test-env', {title: 'Test Page', items: doc});
});
...
It looks as if you're not actually calling your callback function.
async.forEach(doc, function(x, callback){
updates.calcA(x);
updates.calcB(x);
updates.calcC(x);
updates.calcD(x);
// <--- callback() needs to go here.
}
Look at: http://caolan.github.io/async/docs.html#each
The example shows you need to explicitly call the callback() function.
By the way, do your calculations (and the callback function) depend on the order of the contents of the doc collection? Because the async.forEach method actually does not guarantee that each of the elements in doc are going to be calculated in the same order as they are in the collection. So if you are, then it's worth considering if an out of order calculation may result in a different result.
I need to edit a Game object by adding a User object to it. This is done by the mongoose query findById. This query works. But however, when I want to add the modified game to the resulting array, something goes wrong. The 'console.log(game);' returns the right output, but the 'console.log(result);' is always empty. What is going wrong? Is it something with inner functions?
var result = [];
games.forEach(function(game){
User.findById(game.user1_id, function(err, user){
if(err)
console.log(err);
game.user1 = user;
result.push(game);
console.log(game);
});
});
console.log(result);
You have run into a class callback problem. When you call forEach on games the code will actually continue outside the callback and the result will therefore be the value you first assigned to it, which is []. This is due to the code being evaluated asynchronous.
Solve this by moving your code into a function with a callback that is called when the loop is done, like this:
var utils = require('restberry-utils');
var getResult = function(next) {
var result = [];
utils.forEachAndDone(games, function(game, iter) {
User.findById(game.user1_id, function(err, user) {
if (err) console.log(err);
game.user1 = user;
result.push(game);
iter();
});
}, function() {
next(result);
});
};
getResult(function(result) {
console.log(result);
});
Notice I've imported the restberry-utils package and used the forEachAndDone method. This method will loop through the objects but won't continue unless you call the iter method. When it has looped through all the objects, the last callback is called which is where I'm returning the result.
Hope this makes sense.
I have folowing script
var email_list = ['email1#email.com', 'email2#email.com',....'email100#email.com'];
for(i=0;i<email_list.length;i++){
if(checkEmail(email_list[i])){
//do processing save in db and email to email addresses.
}
}
This code will be blocking in nodejs how to make this non blocking?
You can do this without blocking the event loop at all, by using a recursive loop. This way what you end up with is only launching one database worker per call, at a give time. Assuming the database work you were doing was asynchronous, your code didn't really block the event loop. But the foor loop still launched a bunch of workers simultaneously, which will tend to clog the event loop(not block it). And you are right in that it is blocking the event loop while your for loop is counting from 0, to whatever the size of your array is. The following does exactly the same thing, but you only launch one database worker at a time(good), and you never count from 0 to length. Each worker is popped off the list after the work on the current email is done, and your global event loop is left to process other things, not email_list.length database requests simultaneously.
var email_list = ['email1#email.com', 'email2#email.com', 'email100#email.com'];
function checkEmailList(emails, emailCallBack, completionCallback) {
var someDataCollectdOverAllEmails = '';
function checkEmailAsync(email) {
db.doSomeDBWorkAsync(email, function (data) {
someDataCollectdOverAllEmails += data;
if (email_list.length) {
checkEmail(email_list.pop()); //If there are still emails to be checked, check the next one ine line
} else {
completionCallback(someDataCollectdOverAllEmails);//IF not, call the completionCallBack
}
emailCallBack(data);
});
}
checkEmailAsync(emails.pop());
}
function logIndividualEmailData(data) {
console.log('Sningle Email: ' + data);
}
function logGlobalEmailData(data) {
console.log('All Email Data: ' + data);
}
checkEmailList(email_list, logIndividualEmailData, logGlobalEmailData);
Process.nextTick example
process.nextTick(function () {
'use strict';
console.log('printed second');
while (true);
});
process.nextTick(function () {
'use strict';
console.log('never printed');
});
console.log('printed first');
Note however that in the example below, despite the fact that loopForever will run forever, it still allows both of our files to be read out. If we just had while(true) it would of course block and not allow this and one of our files data would not be printed out.
var files = ['blah.js', 'file.js'];
for(var i = 0; i < files.length; i++) {
fs.readFile(files[i], function (err, data) {
console.log('File data' + data);
function loopForver(loop) {//asynchronously loop forever, pretty cool, but only useful for really specific situations!
process.nextTick(function () {
if(loop) {
console.log('looping');
loopForver(true);
}
});
}
loopForver(true);
});
}
If I need to do stuff after the emails all send, I use the async library (docs), which provides some useful functions for control flow.
You will still need to rewrite checkEmail(email) into checkEmail(email, callback) as #S.D. suggests. In checkEmail you will want to call callback after everything is completed. This probably means that you will nest callbacks, calling the second async thing (sending the email) only after the first (db query) has completed successfully.
I also suggest that you follow convention by using the first callback argument as an err parameter. If you callback(null) you are explicitly saying 'there was no error'. #S.D.'s solution suggests instead callback(ok) which is the opposite of convention.
Here is an example showing a couple nested asynchronous functions and the async library.
edit - use async.eachLimit instead of async.each so you don't execute all 100 calls simultaneously
(function main(){
var emails = ["a#b", "c#d"];
var async = require('async');
async.eachLimit(
emails // array to iterate across
,10 // max simultaneous iterations
,checkEmail // an asynchronous iterator function
,function(err){ // executed on any error or every item successful
console.log('Callback of async.eachLimit');
if(err){
console.log('Error: '+err)
} else {
console.log('All emails succeeded');
};
}
);
console.log('Code below the async.eachLimit call will continue executing after starting the asynchronous jobs');
})();
function checkEmail(email, callback){
fetchFromDb(email, function(err, obj){
if(err){ return callback(err) };
sendEmail(email, function(err, obj){
if(err){ return callback(err)};
console.log('Both fetchFromDb and sendEmail have completed successfully for '+email);
callback(null);
});
});
};
function fetchFromDb(email, callback){
process.nextTick(function(){ // placeholder, insert real async function here
callback(null);
});
};
function checkEmail(email, callback){
process.nextTick(function(){ // placeholder, insert real async function here
callback(null);
});
};
I am trying to put a number of documents from a MongoDB collection into an array, using node.js&mongoose. Logging the userDoc in the _.each-loop works fine, but not appending them to an array.
What am I doing wrong?
My best guess is that I have misunderstood something regarding node's asynchronous design, but I have no idea on what I should change.
The code with comments:
returnObject.list = [];
Users.find({}, function (err, user){
_.each(user, function(userDoc){
console.log(userDoc); // Works
returnObject.list.push(userDoc); // No errors, but no users appended
});
});
console.log(returnObject); // No users here!
res.send(JSON.stringify(returnObject)); // Aint no users here either!
Ah this is a nice one, you're trying to do something in an synchronous style:
Users.find({}, function (err, user){
// here you are iterating through the users
// but you don't know when it will finish
});
// no users here because this gets called before any user
// is inserted into the array
console.log(returnObject);
Instead you should do something like this:
var callback = function (obj) {
console.log(obj);
}
Users.find({}, function (err, user){
var counter = user.length;
_.each(user, function(userDoc) {
if (counter) {
returnObject.list.push(userDoc);
// we decrease the counter until
// it's 0 and the callback gets called
counter--;
} else {
// since the counter is 0
// this means all the users have been inserted into the array
callback(returnObject);
}
});
});
do a util.inspect(user) to see what you have before the each loop.