i have been trying to insert an object into an array in async function ,but it
return an empty array as output in nodejs ,mongoose
var data = [];
app.get("/api/post", async (req, res) => {
const post = await UserPost.find();
post.forEach(async element => {
const email = await element.userid;
const user = await Account.find({ email });
const usern = await user[0].username;
var userobject = {
element,
usern
};
//Promise.all(userobject)
data.push(userobject);
});
console.log(data);
res.send({ data });
});
It seems you are struggling with promises. In order to achieve this specific scenario, you can use Promise.all and Array.map.
Here is a code I edited for you:
(*please note that this is just a dummy code for the sake of explanation)
app.get("/api/post", async (req, res) => {
try {
const posts = await dummyPromiseResolver(); // first promise
const promises = posts.map(async element => {
const user = await dummyEmailReturn(element.userid); // second promise
const usern = user[0].username;
return {
usern,
...element
};
});
const fresult = await Promise.all(promises);
res.send(fresult);
} catch (error) {
console.error("error in posts fetch:" + error);
}
});
If I describe this code, posts.map is creating an Array of promises since we need to iterate through every object in the array and needs to add values from separate promises.
Then Promise.all can execute your promise array and return final results array with your desired results.
Note: You can also use for … of as well but when we need to happen things parallelly we use Promise.all. You can find more information from this thread.
here is a link for code sandbox: https://codesandbox.io/embed/serverless-cookies-nu4h0
Please note that I have added dummyPromiseResolver and dummyEmailReturn which would be equal to UserPost.find() and Account.find() functions respectively. In addition to that, I removed a few unnecessary awaits in your code. I added a try catch block to catch any exceptions. You can change that try catch as you please.
hope this will help you. let me know if you need more clarifications.
Related
I'v been searching around for a few hours (with no success) on how to have an async function return a result, store it in a variable outside the function and reuse that data.
My issue is that I fetch the same data over and over in a few of my functions which seems unnecessary.
Basically this is what I want, and right now it's returning a promise.
let leads;
try {
var result = await fetch('http://localhost:3000/lds');
leads = await result.json();
return leads;
} catch (e) {
// handle error
console.error(e)
}
}
var results = readDb();
console.log(results);
For example, initially I run a function fetch the data and create a table.
Secondly I run another function that fetches the same data to create pagination buttons.
Thirdly I run another function that fetches the same data, yet again, and listens for the pagination button click to show the data of the corresponding page number.
Ideally I would fetch the data only once.
Thanks in advance!
We can define leads outside the function and check if it's necessary to fetch it:
let leads;
const getLeads = async function() {
try {
if (!leads) { // fetch leads only they weren't already loaded
const result = await fetch('http://localhost:3000/lds');
leads = await result.json();
}
return leads;
} catch (e) {
// handle error
console.error(e)
}
}
Since you are using async/await you need to await the value, however it is only available in an async function. You can reuse a variable outside the function to store the value once it is loaded.
// reuse leads variable
let leads;
// async function to populate/return leads
async function readDb() {
// only populate if it is undefined
if (typeof leads === 'undefined') {
const result = await fetch('http://localhost:3000/lds');
leads = await result.json();
}
// return just loaded or referenced value
return leads;
}
// call the function asynchronously
(async function(){
// await the results here inside an async function
const results = await readDb();
console.log(results);
}());
I have a firebase callable function that does some batch processing on documents in a collection.
The steps are
Copy document to a separate collection, archive it
Run http request to third party service based on data in document
If 2 was successful, delete document
I'm having trouble with forcing the code to run synchronously. I can't figure out the correct await syntax.
async function archiveOrders (myCollection: string) {
//get documents in array for iterating
const currentOrders = [];
console.log('getting current orders');
await db.collection(myCollection).get().then(querySnapshot => {
querySnapshot.forEach(doc => {
currentOrders.push(doc.data());
});
});
console.log(currentOrders);
//copy Orders
currentOrders.forEach (async (doc) => {
if (something about doc data is true ) {
let id = "";
id = doc.id.toString();
await db.collection(myCollection).doc(id).set(doc);
console.log('this was copied: ' + id, doc);
}
});
}
To solve the problem I made a separate function call which returns a promise that I can await for.
I also leveraged the QuerySnapshot which returns an array of all the documents in this QuerySnapshot. See here for usage.
// from inside cloud function
// using firebase node.js admin sdk
const current_orders = await db.collection("currentOrders").get();
for (let index = 0; index < current_orders.docs.length; index++) {
const order = current_orders.docs[index];
await archive(order);
}
async function archive(doc) {
let docData = await doc.data();
if (conditional logic....) {
try {
// await make third party api request
await db.collection("currentOrders").doc(id).delete();
}
catch (err) {
console.log(err)
}
} //end if
} //end archive
Now i'm not familiar with firebase so you will have to tell me if there is something wrong with how i access the data.
You can use await Promise.all() to wait for all promises to resolve before you continue the execution of the function, Promise.all() will fire all requests simultaneously and will not wait for one to finish before firing the next one.
Also although the syntax of async/await looks synchronous, things still happen asynchronously
async function archiveOrders(myCollection: string) {
console.log('getting current orders')
const querySnapshot = await db.collection(myCollection).get()
const currentOrders = querySnapshot.docs.map(doc => doc.data())
console.log(currentOrders)
await Promise.all(currentOrders.map((doc) => {
if (something something) {
return db.collection(myCollection).doc(doc.id.toString()).set(doc)
}
}))
console.log('copied orders')
}
I am fairly new to Node.js and I am trying to pick it up with Koa.js framework.
I am struggling to understand why the index.js -> console.log run even if I have the await in the value.
Can anyone point me in the right direction?
index.js
router.get('getValue','/display',async (ctx) => {
var myUtilfunction = require('../util')
var result = await myUtilfunction.getData()
console.log(result)
}
util.js
async function getData(){
var customObject =[]
var standardObject =[]
conn.describeGlobal(function (err,res){
if (err){return console.error(err)}
console.log('No of Objects ' + res.sobjects.length)
res.sobjects.forEach(function(sobject){
if (sobject.custom){
customObject.push(sobject)
}else{
standardObject.push(sobject)
}
})
console.log("Done")
})
return [customObject, standardObject]
}
Try this one
await, works inside async functions
router.get('getValue','/display', async (ctx) => {
var myUtilfunction = require('../util')
var result = await myUtilfunction.getData();
console.log(result)
});
function getData(){
return new Promise((resolve, reject) => {
resolve('result goes here');
});
}
You need to specify the function is async for the await to work.
Something like this:
router.get('getValue','/display', async (ctx) => {
var myUtilfunction = require('../util')
var result = await myUtilfunction.getData()
console.log(result)
}
Hope this helps :)
In your function getData(), I think you've missplaced your return statement. The return statement should be placed inside the callback function used for conn.describeGlobal().
As you actually write your getData(), the conn.describeGlobal() call seems to be an asynchronous treatment, so the return statement placed outside is probably executed before you pushed something in your arrays.
The consequence is that your router get an empty response from your getData() function then the promise made by await keyword is resolved with an empty answer.
I have a function which contains a thousand of objects in an array:
function Alltransaction(transactionArray) {
transactionArray.map(async (transaction) => {
dataAgainsthash = await web3.eth.getTransaction(transaction)
TransactionObject = {
transactionHash : transaction,
from : dataAgainsthash.from
};
transactionArray.push(TransactionObject)
console.log("transaction array", transactionArray)
});
}
then i have another function which stores these thousands of object array into db
function saveTransactionToDb() {
console.log("after loop",transactionArray)
transactionss = new Transaction({
blockNumber : blockNumbers ,
transactions : transactionArray
})
// Now save the transaction to database
await transactionss.save();
// console.log("save to database")
}
then I call this in my router
await Alltransaction(transactionArray);
await saveTransactionToDb();
and I also try
Alltransaction(transactionArray).then(saveTransactionToDb())
But it always runs saveTransactionToDb() before the array of object populates the Alltransaction() method
have you try the async keyword before saveTransactionToDb and Alltransaction functions??
async function Alltransaction(transactionArray){
// your code
}
async function saveTransactionToDb(){
// your code logic*
await transactionss.save();
}
First, in Alltransaction the promise must be returned as well. In your code the function starts some processes but doesn't not await on it. Also, do not push the promises to the original array. I'm not sure what you were trying to accomplish there. Because mapping over the array gives you an array of promises, you can unify all of them with Promise.all().
function Alltransaction(transactionArray) {
const promises = transactionArray.map(async (transaction) => {
dataAgainsthash = await web3.eth.getTransaction(transaction)
const TransactionObject = {
transactionHash : transaction,
from : dataAgainsthash.from
};
return TransactionObject;
});
return Promise.all(promises);
}
Change saveTransactionToDb to receive an array instead of using the original array.
Then you'll be able to call it as:
const t = await Alltransaction(transactionArray);
await saveTransactionToDb(t);
Your second try it's not correct:
Alltransaction(transactionArray).then(saveTransactionToDb())
It's the same as:
const t = Alltransaction(transactionArray);
const s = saveTransactionToDb();
t.then(s)
That's why saveTransactionToDb doesn't way for transactions to complete. To use then, just pass the function without calling it:
Alltransaction(transactionArray).then(saveTransactionToDb)
I am working on an async problem. I'm making a web scraper and after I scrape the web, I need to put the data in my MongoDB database after putting it in. I need to send it into the frontend, but since I have a loop the elements I can't put the res.json() inside, as it'll gave an error (you can only send once after res.json()).
I'm stuck here. I've used promises before, but this is confusing.
router.get('/scrape', (req, res) => {
request('http://www.nytimes.com', function test(error, response, html) {
const $ = cheerio.load(html);
// An empty array to save the data that we'll scrape
const results = [];
$('h2.story-heading, p.summary').each(function(i, element) {
const link = $(element)
.children()
.attr('href');
const title = $(element)
.children()
.text();
const summary = $(element)
.children()
.text();
const data = {
title: title,
link: link,
summary: summary,
};
articles
.create(data)
.then((resp) => results.push(resp))
// .then((resp) => Promise.resolve(results)) //
// .then((jsonDta ) => res.json(jsonData)) // error you can only give response once.
.catch((err) => reject(err));
});
console.log(results); // empty array
res.json(results)// empty
});
});
My plan is:
to scrape a site (loop the elements)
then save into MongoDB (push the data into an array)
then after the loop pass it to the frontend.
I need to put the query method create... inside the loop because I need each data to have an id.
Instead of trying to accumulate results directly, you can map the elements contained in $('h2.story-heading, p.summary') to an array of promises, then aggregate with Promise.all(). The results you want will be delivered by Promise.all(...).then(...).
router.get('/scrape', (req, res) => {
request('http://www.nytimes.com', function test(error, response, html) {
const $ = cheerio.load(html);
const promises = $('h2.story-heading, p.summary')
.get() // as in jQuery, .get() unwraps Cheerio and returns Array
.map(function(element) { // this is Array.prototype.map()
return articles.create({
'title': $(element).children().text(),
'link': $(element).children().attr('href'),
'summary': $(element).children().text()
})
.catch(err => { // catch so any one failure doesn't scupper the whole scrape.
return {}; // on failure of articles.create(), inject some kind of default object (or string or whatever).
});
});
// At this point, you have an array of promises, which need to be aggregated with Promise.all().
Promise.all(promises)
.then(results => { // Promise.all() should accept whatever promises are returned by articles.create().
console.log(results);
res.json(results);
});
});
});
If you want any single failure to scupper the whole scrape, then omit the catch() and add catch() to the Promise.all().then() chain.
Notes:
For .get() (and most other methods), the jQuery documentation is better than the Cheerio documentation (but be careful because Cheerio is a lean version of jQuery).
At no point do you need new Promise(). All the promises you need are returned by articles.create().
Something like this might work (code not tested)
router.get('/scrape', (req, res) => {
request('http://www.nytimes.com', function test(error, response, html) {
const $ = cheerio.load(html);
// An empty array to save the data that we'll scrape
const results = [];
$('h2.story-heading, p.summary').each(function(i, element) {
const link = $(element)
.children()
.attr('href');
const title = $(element)
.children()
.text();
const summary = $(element)
.children()
.text();
const data = {
title: title,
link: link,
summary: summary,
};
const articleCreate = articles.create(data);
results.push(articleCreate);
});
console.log(results); // this is array of promise functions.
Promise.all(results).then(allResults => {
res.json(allResults)
});
// or you could use array.reduce for sequantial resolve instead of Promise.all
});
});
Use .map function to return all promises to Promise.all and then return the results.
request('http://www.nytimes.com', function test(error, response, html) {
const $ = cheerio.load(html);
var summary = $('h2.story-heading, p.summary')
Promise.all(summary.map((i, element) =>{
const data = {
title: $(element).children().text(),
link: $(element).children().attr('href'),
summary: $(element).children().text(),
};
return articles
.create(data)
}).get())
.then((result)=>{
console.log(result);
res.json(result);
});
})