I need to edit a Game object by adding a User object to it. This is done by the mongoose query findById. This query works. But however, when I want to add the modified game to the resulting array, something goes wrong. The 'console.log(game);' returns the right output, but the 'console.log(result);' is always empty. What is going wrong? Is it something with inner functions?
var result = [];
games.forEach(function(game){
User.findById(game.user1_id, function(err, user){
if(err)
console.log(err);
game.user1 = user;
result.push(game);
console.log(game);
});
});
console.log(result);
You have run into a class callback problem. When you call forEach on games the code will actually continue outside the callback and the result will therefore be the value you first assigned to it, which is []. This is due to the code being evaluated asynchronous.
Solve this by moving your code into a function with a callback that is called when the loop is done, like this:
var utils = require('restberry-utils');
var getResult = function(next) {
var result = [];
utils.forEachAndDone(games, function(game, iter) {
User.findById(game.user1_id, function(err, user) {
if (err) console.log(err);
game.user1 = user;
result.push(game);
iter();
});
}, function() {
next(result);
});
};
getResult(function(result) {
console.log(result);
});
Notice I've imported the restberry-utils package and used the forEachAndDone method. This method will loop through the objects but won't continue unless you call the iter method. When it has looped through all the objects, the last callback is called which is where I'm returning the result.
Hope this makes sense.
Related
I query one collection (messages) with mongoose. The result is an array of documents. Each document contains an ID for a different collection (users). Now I want to query the users collection for each ID from the messages collection.
The idea is to update each message object with the information from the user collection before returning it to the front end.
I tried using async.each. For some reason the final function is never called even though I am making sure the callback() function is called after each iteration.
app.get('/getmsg', function(req, res){
messages.find({query})
.exec(function(err, ms){
if(err) throw err;
async.each(ms, function(m, callback){
users.findOne({_id : m.userId})
.lean()
.exec(function(err, user){
if(err) {
console.log('error' , err);
callback();
} else {
m.userName = user.name;
// everything is working up to here
callback();
}
}), function(err){
res.send(ms); // this is never returned!
}
});
});
});
Is there a better way of achieving this? I assume this must be a common issue.
Thanks!
You can't use res.send. Instead create a function to get notified about it. Something like this.
// 1st para in async.each() is the array of items
async.each(items,
// 2nd param is the function that each item is passed to
function(item, callback){
// Call an asynchronous function, often a save() to DB
item.someAsyncCall(function (){
// Async call is done, alert via callback
callback();
});
},
// 3rd param is the function to call when everything's done
function(err){
// All tasks are done now
doSomethingOnceAllAreDone();
}
);
I've got an Express GET request that pulls data from a Mongoose query and for each doc returned, external functions perform calculations on each doc passed as x and return the results so I can render them on the front-end using Handlebars. Ideally, I would like to carry out my calculations calcA, calcB, calcC, calcD and then once they're completed for each of the documents, render the test-env.hbs template. Currently, the page doesn't render when I call it, in the logs, after some time, it shows GET /test-env - - ms - -, presumably because the callback gets stuck somewhere.
Express GET Request
var updates = require('./updates.js');
app.get('/test-env', function(req, res, next){
Market.find({"marketname" : 'To Win'})
.then(function(doc){
async.forEach(doc, function(x, callback){
updates.calcA(x);
updates.calcB(x);
updates.calcC(x);
updates.calcD(x);
}, function(err){
if(err)
return console.log(err);
res.render('test-env', {title: 'Test Page', items: doc});
});
});
});
Typical calc Function
I want to add returnA and returnB to the docs data so I can render it in an {{#each}} expression in Handlebars
calcA: function(x) {
Market.find({"student": x.student, "marketname": x.marketname})
.sort({btotal: -1})
.limit(1)
.then(function(ret) {
var valueA = ret[0].btotal;
var valueB = ret[0].back;
Market.find({"student": x.student, "marketname": x.marketname, "back": {$lt: valueB}})
.sort({odds: -1})
.limit(1)
.then(function(doc) {
var res = doc[0];
if (res == null) {
x.returnA = 0;
x.returnB = 0;
} else {
x.returnA = res.back;
x.returnB = res.btotal;
}
});
});
}
You aren't calling your callback in your async iterator. If the calc functions are asynchronous, those callbacks should be handled individually. If they are not asynchronous, a normal forEach will do the trick.
...
async.forEach(doc, function(x, callback){
updates.calcA(x);
updates.calcB(x);
updates.calcC(x);
updates.calcD(x);
callback(); // Call the callback to move on to the next item
}, function(err){
if(err)
return console.log(err);
res.render('test-env', {title: 'Test Page', items: doc});
});
...
It looks as if you're not actually calling your callback function.
async.forEach(doc, function(x, callback){
updates.calcA(x);
updates.calcB(x);
updates.calcC(x);
updates.calcD(x);
// <--- callback() needs to go here.
}
Look at: http://caolan.github.io/async/docs.html#each
The example shows you need to explicitly call the callback() function.
By the way, do your calculations (and the callback function) depend on the order of the contents of the doc collection? Because the async.forEach method actually does not guarantee that each of the elements in doc are going to be calculated in the same order as they are in the collection. So if you are, then it's worth considering if an out of order calculation may result in a different result.
var pg = require('pg');
var conString = "postgres://postgres:abk#localhost/bot";
var res = [];
pg.connect(conString, function(err, client, done) {
if(err) {
return console.error('error fetching client', err);
}
client.query('SELECT * FROM bot.questions', function(err, result)
{
done();
if(err) {
return console.error('error running query', err);
}
res.push(result.rows[0]);
console.log(res); //inside function scope
});
});
console.log(res); // outside function scope
"console.log()" which has called inside the function gives proper resulting array, but outside function the same array variable shows an empty array. I have tried lot of things, such as callbacks, promises functionalities in node js but was not able to see the resulting "res" outside the function.
Please suggest me how do I make that "res" variable accessible from outside of function.
NOTE: Specifically, I need "console.log(res)" to print the "res" outside of function as mentioned in above code.
res variable is indeed accessible outside the function, as it is defined outside the function
res variable is indeed populated outside of the function, as it is seen as populated inside the function
root cause of your problem is that the outside print happens chronologically earlier than the inside activity, as the function is asynchronous, the call comes out immediately, and proceeds to execute the last line in the code.
If you are just particular about getting the value outside, it is indeed is the case
If you are wanting to just print the value outside, use a delayed callback in which to print the content
If you are wanting to post-process the value outside, you will need to link the connect function with a next function, through synchronization primitives.
Hope this helps.
As explained before, the query's result will exist only when the callback is fired, so it won't make sense accessing it in a part of the code that would execute before that.
It seems like you need to read a bit about node's asynchronous programming.
I suggest you read about promises and generators.
I found a solution finally, I have created seperate file for db config along with resulting array of values using callback function to export. Here is the "db.js" file which I have created.
var pg = require('pg');
var conString = "postgres://postgres:abk#localhost/chatbot";
var resp = [];
function executeQuery (callback) {
pg.connect(conString, function(err, client, done) {
if(err) {
return console.error('error fetching client', err);
}
client.query('SELECT * FROM bot.questions', function(err, result) {
done();
if(err) {
return console.error('error running query', err);
}
resp.push(result.rows[0]);
// console.log(res);
callback(resp);
});
});
};
exports.executeQuery = executeQuery;
and this I have imported into my "app.js" like bellow
var express = require('express')
var bodyParser = require('body-parser')
var request = require('request')
var app = express()
var db = require('./db')
app.post('/webhook/', function (req, res) {
db.executeQuery(function ( resp) {
console.log(resp);
});
});
This is how I can now access "resp" as resulting array, and can pass it to any function as an argument wherever required.
There is specific need as per my requirements to make the db config file separate from app.js file, otherwise I know it can also be manipulated in same app.js file.
So finally, This works for me perfectly.
This is what a kind of example I was expecting. But I found it by myself.
Anyways thanks for your involvement, and suggessions.
I want to track the number of documents I have within a collection in a node.js server
using mongodb driver. I can insert, delete and update propperly but when I try to count, it works until I try to store that value, moment in which it returns nothing.
Here is my code:
var db_collection = db.collection('collection');
var countCollections = function () {
var response_count_collections = null;
db_mensajes.count(function(err,number_of_collections){
if (err){
console.log(err);
} else {
response_of_collections = number_of_collections;
console.log('Number of collections: '+number_of_collections);
}
});
return response_count_collections;
};
It logs the number_of_collections correctly but it doesn't return me the right value. In my example it returns null (how I defined my var response_count_collections;) and if I
try to return number_of_collections; within the db_collections.count()'s callback like this:
var db_collection = db.collection('collection');
var countCollections = function () {
var response_count_collections = null;
db_mensajes.count(function(err,number_of_collections){
if (err){
console.log(err);
} else {
console.log('Number of collections: '+number_of_collections);
return number_of_collections;
}
});
};
It returns me "undefined". Is there any way to achieve what I want?
Its because it returns the variable before the function is completely executed.
If you want it to be asynchronous then you will have to learn to control the flow of the program using some node modules like async.
Update:
Have a look at this question, it shows how to return value from an async function using callback correctly.
I am trying to put a number of documents from a MongoDB collection into an array, using node.js&mongoose. Logging the userDoc in the _.each-loop works fine, but not appending them to an array.
What am I doing wrong?
My best guess is that I have misunderstood something regarding node's asynchronous design, but I have no idea on what I should change.
The code with comments:
returnObject.list = [];
Users.find({}, function (err, user){
_.each(user, function(userDoc){
console.log(userDoc); // Works
returnObject.list.push(userDoc); // No errors, but no users appended
});
});
console.log(returnObject); // No users here!
res.send(JSON.stringify(returnObject)); // Aint no users here either!
Ah this is a nice one, you're trying to do something in an synchronous style:
Users.find({}, function (err, user){
// here you are iterating through the users
// but you don't know when it will finish
});
// no users here because this gets called before any user
// is inserted into the array
console.log(returnObject);
Instead you should do something like this:
var callback = function (obj) {
console.log(obj);
}
Users.find({}, function (err, user){
var counter = user.length;
_.each(user, function(userDoc) {
if (counter) {
returnObject.list.push(userDoc);
// we decrease the counter until
// it's 0 and the callback gets called
counter--;
} else {
// since the counter is 0
// this means all the users have been inserted into the array
callback(returnObject);
}
});
});
do a util.inspect(user) to see what you have before the each loop.