I am looping over a collection of Users in node using mongoose. I want to save the data of every User in a excel File.
Usr.findById(usrID, function(err, foundTagestipp){
sheet1.set(1, counter, 'some Userdata');
}
The problem is that I want to save the file after it went through the whole collection.
So when do I know when the last user is done? Can I somehow give that to a callback or can I somehow get information from mongoose when it is done?
I hope it is clear what I meant by that...
Sorry for the nooby question
you can wrap each async call inside a promise, collect a list of promises and resolve them outside the loop with a call to Promise.all() then execute your business logic.
let promises = [];
myCollection.forEach(elem => {
let promise = new Promise( (resolve, reject) => {
Usr.findById(elem.usrID, function(err, foundTagestipp){
if(err) {
return reject(err);
}
resolve({userId: elem.usrId, userData: foundTagestipp})
}
})
promises.push(promise);
})
Promise.all(promises)
.then(data => {
//here all the findById have already been called
//you can save your data to the csv
})
Related
How to use multi query like find , update , insert , delete in one service on mongodb
I can query this below
router.get('/userlist', function(req, res) {
User.find({},function (err, docs) {
res.json(docs);
});
});
but i want to query like this is an error
router.get('/userlist', function(req, res) {
var data = array();
User.find({},function (err, docs) {
data['data1'] = docs
})
Content.find({},function (err, docs) {
data['data2'] = docs
})
res.json(docs)
});
Can anyone help me or It is possible to use query like this?
Thank you
You can use async await to run multiple mongo queries inside a function like below:
router.get('/userlist', async function(req, res) {
var data = array();
try{
//first query
let user_data = await User.find({}).exec();
data['data1'] = user_data;
//second query
let content_data = await Content.find({}).exec();
data['data2'] = content_data;
return res.status(200).json(data)
}
catch(err){
return res.status(400).json({err})
}
});
My question is what is stopping you? Yes, you can do this. I did those kinds of works.
There remain two points.
If you need to perform multiple queries, based on the response of the previous query, you can use async/await or promises. If you don't wanna try those, there is still a way. You can check the response of the previous query, make conditions, and if right, execute the seconde one. And thus you can make a chain of queries.
And if you don't need to rely on the responses, do whatever you want. There is nothing wrong...
Happy Coding!
You have to wait for your database call response. So, You can use promise for this. MDN: Promise Make promise of database call and resolve that after you got data from database. Promise.all will wait until your both the promises resolved. MDN: Promise.all()
router.get('/userlist', function(req, res) {
var data = [];
let promise1 = new Promise((resolve, reject) => {
User.find({},function (err, docs) {
data['data1'] = docs;
resolve();
})
});
let promise2 = new Promise((resolve, reject) => {
Content.find({},function (err, docs) {
data['data2'] = docs
})
});
Promise.all([promise1, promise2]).then(result => res.json(docs))
});
NB:I am new to Node.js
I would like to call a function in the current module and it perform the data pull with a promise or async function. Then be able to access the data in a variable. Once it is a variable out of the promise maybe I can store/use/investigate the data. Any help would be much appreciated.
I manged to get the code below to run but I am not really sure if it is correct? Is there a cleaner/proper way to do this? Very confused.
I have tried to write a promise but I am going in circles.
const publicClient = new CoinbasePro.PublicClient();
const coinbasePro = require('coinbase-pro')
const p = new Promise((resolve, reject) => {
yourFunction()
async function yourFunction() {
try {
const products = await publicClient.getProducts();
resolve(products);
//console.log('ArrayName', products);
return products
} catch (error) {
reject(error);
}
}
});
p.then(result => console.log('Result', result))
p.catch(err => console.log('Error', err.message));
1) I would like to be able to invoke this promise from the same module and get the data in a variable.
2) I would like to be able to invoke this promise from another module and get the data in a variable.
Consider a promise-chained chunk of code for example:
return Promise.resolve()
.then(function () {
return createSomeData(...);
})
.then(function () {
return updateSomeData(...);
})
.then(function () {
return deleteSomeData(...);
})
.catch(function (error) {
return ohFishPerformRollbacks();
})
.then(function () {
return Promise.reject('something failed somewhere');
})
In the above code, let's say something went wrong in the function updateSomeData(...). Then one would have to revert the create operation that was executed before this.
In another case, if something went wrong in the function deleteSomeData(...), then one would want to revert the operations executed in createSomeData(...) and updateSomeData(...).
This would continue as long as all the blocks have some revert operations defined for themselves in case anything goes wrong.
Only if there was a way in either NodeJs or the database or somewhere in the middle, that would revert all the transactions happening under the same block of code.
One way I can think of this to happen is by flagging all the rows in database with a transactionId (ObjectID) and a wasTransactionSuccessful(boolean), so that CRUD operations could be clubbed together with their transactionIds, and in case something goes wrong, those transactions could be simply deleted from the database in the ending catch block.
I read about rolling back transactions in https://docs.mongodb.com/manual/tutorial/perform-two-phase-commits/. But I want to see if it can be done in a more simpler fashion and in a generic manner for NoSQL databases to adapt.
I am not sure if this would satisfy your use case, but I hope it would.
let indexArray = [1, 2, 3];
let promiseArray = [];
let sampleFunction = (index) => {
return new Promise((resolve, reject) => {
setTimeout(resolve, 100, index);
});
}
indexArray.map((element) => {
promiseArray.push(sampleFunction(element));
});
Promise.all(promiseArray).then((data) => {
// do whatever you want with the results
}).catch((err) => {
//Perform your entire rollback here
});
async.waterfall([
firstFunc,
secondFunc
], function (err, result) {
if (err) {
// delete the entire thing
}
});
Using the async library would give you a much elegant solution than going with chaining.
Suppose I have the following source code in NodeJS using promise (connecting to mongodb with 3 collections, well, this is not important)
function getData(){
return new promise((resolve, reject) => {
var dataArray = [];
getCollection1().then((rows1) => {
getCollection2().then((rows2) => {
var val = someDataProcessingFunction(rows1, rows2);
dataArray.push(val);
resolve(dataArray);
}, (err) => {
reject(err);
}
}, (err) => {
reject(err);
});
getCollection3().then((rows3) => {
rows3.forEach((row3) => {
dataArray.push(row3);
});
resolve(dataArray);
}, (err) => {
reject(err);
});
});
}
The problem is, there is multiple places where dataArray is pushed, but I don't know when all the pushing is completed. so that I don't know when to resolve, to return the data to user, for example, by response.send in express. As to the code above while I am using resolves several times, I will only get part of the whole data returned.
I can think out two workarounds, neither is good. one is to surround all resolves in setTimeout, which means I must delay the response to user.
another is to count all the push times at the beginning, every time subtract 1 when one item is pushed. when the times goes 0, do resolve. but it is often difficult to calculate the number before doing the real loops.
Is there better solutions? Thank you and love you :)
First of all, avoid the Promise constructor antipattern and use return values instead! For multiple layers of asynchronous functions, just use multiple layers of promises. To await multiple promises at once, use Promise.all:
function getData(){
return Promise.all([
getCollection1(),
getCollection2(),
getCollection3()
]).then(([rows1, rows2, rows3]) => {
return [someDataProcessingFunction(rows1, rows2)].concat(rows3);
});
}
I am experimenting with Express and MongoDB, and have a functional API server that can add, update, delete, and retrieve a single post. The issue I have run into is returning all of the documents from Mongo.
I have a GET route that outputs the results, except it does not behave as I imagined it would. When you run this, the first GET request to /notes returns and empty array, i.e. []
let notes =[];
app.get('/notes', (req, res) => {
async function getNotes() {
try {
await db.collection('notes').find().forEach(function (myDoc) {
notes.push(myDoc);
})
} catch(err) {
console.log(err)
}
console.log(notes);
res.send((notes));
}
getNotes();
});
On the second GET to /notes, however, the data that was pushed into notes[] is returned, and it is then overwritten by the newly pushed data.
Can anyone help me fill in the blank spot in my understanding of this? I imagine there is something that I just didn't understand along the way.
Edit***
I have experimented with this a bit, and am still running into the same issues.
let array= [];
async function getNotes() {
try {
await db.collection('notes').find().toArray(function (err, notesArray) {
array = notesArray;
})
} catch (err) {
console.log(err)
}
console.log(array);
return array;
}
app.get('/notes', (req, res) => {
getNotes();
res.send(array);
});
MongoDB's .toArray() won't both invoke a callback and return a Promise.
Returns:
Promise if no callback passed
And, await depends on the Promise, so you won't be able to use the callback as well.
Though, one isn't really necessary. When the promise resolves, await will return the array of documents, allowing you to assign that to array or any other variable.
try {
array = await db.collection('notes').find().toArray();
}