I am having my code as
function updateData(data){
data.forEach(function(obj){
users.find({_id:obj.userId}).toArray(
function(e, res) {
obj.userData = res;
console.log(res)
});
});
return data;
}
The problem is I am unable to find the updated data, I am trying to update my data and adding one more field to it based on userId. The data parameter is an array containing the output from comments table. hope you understand the scenario.
It looks that users.find({...}).toArray(function(...){...}) is going to be asynchronous, so there is no way that you can be sure that the db call has been completed and that each data.obj has been updated before data is returned.
Instead of using the javascript Array.prototype.forEach() function, you could use the NodeJS async.each function from the async library by Caolan which would iterate through the array, update each object and then return the data object only when all functions calls have completed.
For example:
var async = require("async");
function updateData(data){
async.each(data, function(obj, callback) {
users.find({_id:obj.userId}).toArray(
function(e, res) {
obj.userData = res;
callback(e);
}
);
},
function(error){
return data;
}
}
Related
I've tried a few approaches to this - what I have is all my modules in one file that interact with mongodb, and in another, the express route functions that call into those async functions looking for data. The problem is that the data is available in the async function, but is not returned to the calling function, and I'm not sure how to pass it back properly (not sure if it's an issue of not waiting for the async function to return and returning the array early, or if I'm actually returning it wrong).
Here is the code for the calling function
router.route('/').get((req, res) => {
db.getItemsFromCollection("Plants").then( (itemArr) => {
console.log(itemArr);
})
});
And the db function (two attempts, one commented out)
getItemsFromCollection: async function(collectionName) {
let itemArr = [];
const collection = client.db().collection(collectionName);
/*collection.find().forEach(function( doc) {
itemArr.push(doc);
//console.log(doc);
})
return itemArr;*/
collection.find({}).toArray(function(err, item) {
if (err) throw err;
console.log(item);
itemArr.push(item);
})
return itemArr;
},
If you are using mongodb then it already supports Promise syntax:
However, all async API calls support an optional callback as the final
argument, if a callback is provided a Promise will not be returned.
Then you can make getItemsFromCollection become:
getItemsFromCollection: async function (collectionName) {
// let itemArr = [];
const collection = client.db().collection(collectionName);
const items = await collection.find({}).toArray(); // don't pass callback param
console.log(items);
// I guess you want to get back an array of the collection item
return items;
},
I have this function which gets some data from my database but i'm having a trouble calling the function and getting the proper response
function getEvents()
{
var x = [];
var l = dbCollection['e'].find({}).forEach(function(y) {
x.push(y);
});
return x;
});
and another function which calls this function but it always returns undefined.
How can i make the function wait till mongoose has finished filling up the array?
Thanks for the help! My life
dbCollection['e'].find is called non-blocking way so you are returning x before filling. You need to use callbacks or some mongoose promises. You can get all returning values from database like following snippet
function getEvents(callback) {
dbCollection['e'].find({}, function(error, results) {
// results is array.
// if you need to filter results you can do it here
return callback(error, results);
})
}
Whenever you need to call getEvents function you need to pass a callback to it.
getEvents(function(error, results) {
console.log(results); // you have results here
})
You should read mongoose docs for how queries work.
There is also support for promises in mongoose. You can check this url for more information about promises.
The solution proposed by #orhankutlu should work fine.
I will give another solution using promise. You can choose one between these two solutions depending on your style of programming.
Solution using promise:
function getEvents() {
return new Promise(function(resolve, reject){
dbCollection['e'].find({}, function(error, results) {
if (error) return reject(error);
var x = [];
results.forEach(function(y){
x.push(y);
});
// forEach() is a blocking call,
// so the promise will be resolved only
// after the forEach completes
return resolve(x);
});
});
};
Calling getEvents():
getEvents().then(function(result){
console.log(result); //should print 'x'
}).catch(function(err){
// Handle error here in case the promise is rejected
});
I will encourage you to try both the approaches, ie, using callbacks and using promises. Hope you find it useful!
I'm trying to find a way to send multiple requests (using Express) and process all the response in one function.
Here's my code :
// In router.js
app.get('/api/FIRST_PATH', CALLBACK_FUNCTION_A );
// In CALLBACK_FUNCTION_A file :
module.exports = function (req, response) {
CALLBACK_FUNCTION_TO_SERVICE_A();
CALLBACK_FUNCTION_TO_SERVICE_B();
CALLBACK_FUNCTION_TO_SERVICE_C();
}
My problem is to send the requests CALLBACK_FUNCTION_TO_SERVICE_A, CALLBACK_FUNCTION_TO_SERVICE_B and CALLBACK_FUNCTION_TO_SERVICE_C and then retrieve all the results in another function to process them.
Any help would be greatly appreciated.
You can learn more about the new js standard and use Promise.
// In CALLBACK_FUNCTION_A file :
module.exports = function (req, response) {
var promises = [CALLBACK_FUNCTION_TO_SERVICE_A(),
CALLBACK_FUNCTION_TO_SERVICE_B(),
CALLBACK_FUNCTION_TO_SERVICE_C()];
Promise.all(promises).then( function(results) {
//results is an array
//results[0] contains the result of A, and so on
});
}
Of course CALLBACK_FUNCTION_TO_SERVICE_A() and such need to return Promise objects. You form a function like this:
function asyncFunction(callback) {
//...
callback(result);
}
You can create a Promise like this:
var p = new Promise(asyncFunction);
It'll start running the function, and supports the Promise interface.
So for example, either use request-promise or you can do something like:
function CALLBACK_FUNCTION_TO_SERVICE_A() {
var worker = function(callback) {
app.get('/api/FIRST_PATH', callback);
};
return new Promise(worker);
}
You can read more about Promise and how to also easily handle errors.
You could use async parallel. You can keep all your API calls as async.parallel array or JSON(Example uses Array).
async.parallel(
[
function(done){
reqServcieA(..., funnction(err, response){
if(err) done(err,null);
done(null, response);
}
},
function(done){
reqServcieA(..., funnction(err, response){
if(err) done(err,null);
done(null, response);
}
},
...// You can keep as many request inside the array
], function(err,results){
// Will be called when all requests are returned
//results is an array which will contain all responses as in request arry
//results[0] will have response from requestA and so on
});
Let's say I have 3 files.
index.js makes a call to the backend like this
$.post('/test/', data, function(response) {
//handle success here
})
routes.js handles the route like this
app.post('/test/', function(req, res){
item.getItems(function(response){
res.json(response);
});
});
items.js is the model which accesses the database and makes a POST request for each item
function getItems(callback) {
database.query('SELECT * from items', function(result){
result.forEach(function(item){
request.post('/api/', item, function(req, res) {
//finished posting item
});
});
});
//callback here doesnt wait for calls to finish
}
where/when should I call the callback passed to getItems() to handle a success/failure in index.js?
Because your request.post() operations are async, you have to use some method of keeping track of when they are all done and then you can call your callback. There are multiple ways of doing that. I'll outline a few:
Manually Keeping Track of Count of Request Operations
function getItems(callback) {
database.query('SELECT * from items', function(result){
var remaining = result.length;
result.forEach(function(item){
request.post('/api/', item, function(err, res) {
--remaining;
//finished posting item
if (remaining === 0) {
callback();
}
});
});
});
}
The main problem with doing this manually is that propagating error in nested async operations is difficult when you get to actually figuring out how you're going to handle errors. This is much easier in the other methods shown here.
Using Promises
// load Bluebird promise library
var Promise = require('bluebird');
// promisify async operations
Promise.promisifyAll(request);
function queryAsync(query) {
return new Promise(function(resolve, reject) {
// this needs proper error handling from the database query
database.query('SELECT * from items', function(result){
resolve(result);
});
});
}
function getItems(callback) {
return queryAsync('SELECT * from items').then(function(result) {
return Promise.map(result, function(item) {
return request.postAsync('/api/', item);
});
});
}
getItems.then(function(results) {
// success here
}, function(err) {
// error here
})
It seems strange that you're making an API request in your server-side code, unless this is some sort of middle tier code that interacts with the API... but you're interacting with a database, so I'm still confused on why you can't just do a database insert, or have a bulk insert API call?
Anyway, if you must do it the way you're asking, I've done this in the past with a recursive method that trims down the result array... I really don't know if this is a good approach, so I'd like to hear any feedback. Something like this:
function recursiveResult(result, successfulResults, callback) {
var item = result.shift();
// if item is undefined, then we've hit the end of the array, so we'll call the original callback
if (item !== undefined) {
console.log(item, result);
// do the POST in here, and in its callback, call recursiveResult(result, successfulResults, callback);
successfulResults.push(item);
return recursiveResult(result, successfulResults, callback);
}
// make sure callback is defined, otherwise, server will crash
else if (callback) {
return callback(successfulResults);
}
else {
// log error... callback undefined
}
}
function getItems(callback) {
var successfulResults = [];
var result = [1, 2, 3, 4];
recursiveResult(result, successfulResults, callback);
}
console.log('starting');
getItems(function(finalResult) {
console.log('done', finalResult);
});
I have a loop that I need to run inside of a callback, unfortunately accessing the array outside of the callback leaves me with a blank array. I know why this happens, but I want to know the best solution to tackle this.
Gallery.prototype.getGallery = function(cb) {
self = this;
var cos = new pb.CustomObjectService();
var ms = new pb.MediaService();
var s = [];
cos.loadTypeByName('Gallery Image', function(err, gallery){
cos.findByType(gallery._id.toString(), function(err, rpy){
for(var i = 0; i < rpy.length; i++){
ms.loadById(rpy[i].Image, function(e,r){
s.push(r.location);
console.log(r.location); /* <-- logs expected data */
});
}
console.log(s[0]); /* <-- this is undefined */
});
});
};
Replace your for loop with a call to async.*; in this case async.map seems right. Pass a callback to async.map; it will be invoked when all the individual calls to ms.loadById are done, with the array of results.
async.map(
rpy,
function(elt, callback) {
ms.loadById(elt.Image, callback);
},
function(err, data) {
// comes here after all individual async calls have completed
// check errors; array of results is in data
}
);
If you want to go into the promises world, then wrap the calls to ms.loadById in a promise. Here's a roll-your-own version, but various versions of what is usually called promisify are also out there.
function loadByIdPromise(elt) {
return new Promise(function(resolve, reject) {
ms.loadById(elt.image, function(err, data) {
if (err) return reject(err);
resolve(data);
});
});
}
Then do a Promise.all on the resulting promises:
Promise.all(rpy.map(loadByIdPromise))
.then(function(data) {
// comes here when all individual async calls complete successfully
// data is your array of results
});
Using the promises style, your entire code would look like:
loadTypeByNamePromise('Gallery Image') .
then(function(gallery) { return findByTypePromise(gallery._id.toString(); }) .
then(function(rpy) { return Promise.all(rpy.map(loadByIdPromise)); }) .
then(function(results) { /* do something with [results] */ });