I have two collections of products and categories and I have both collections that have data products(152), categories(10). So, I tried to connect the DB to retrieve the data. First I call products collection data next call categories collection data using async-await functionality. But it gets the first categories of data and next product data. How to solve this issue anyone can give the answer.
product.js
async function product_data(collection) {
let mongodb = await MongoDB.connect(collection)
let result = await mongodb.findAll()
return result
}
module.exports.product_data = product_data
category.js
async function category_data(collection) {
let mongodb = await MongoDB.connect(collection)
let result = await mongodb.findAll()
return result
}
module.exports.category_data = category_data
app.js
const {product_data} = require("./product")
const {category_data} = require("./category")
async function updatedb() {
let product_data = await product_data("ecomm_product")
console.log(product_data)
let category_data = await category_data("ecomm_category")
console.log(category_data)
}
I got result
Its first print category_data after print product_data
Expected result
Its first print product_data after print category_data
I can't reproduce this at all, not even with explicit delays in which products take longer to resolve than categories. Collapsing your code to a single file, and using proper JS conventions for naming and case:
function getProductData(collection) {
return new Promise(resolve => {
setTimeout(() => resolve('product'), 2000);
});
}
function getCategoryData(collection) {
return new Promise(resolve => {
setTimeout(() => resolve('category'), 1000);
});
}
async function updatedb() {
let product_data = await getProductData("ecomm_product")
console.log(product_data)
let category_data = await getCategoryData("ecomm_category")
console.log(category_data)
}
updatedb();
This simply yields the following output, every time:
$node test.js
product
category
Related
I have read a lot of posts on how to solve this problem, but I cannot understand it.
I have a database (psql) and a csv. I have a two functions. One to read a list of domains from psql. And another to read a different list of domains from the csv.
Both functions are async operations that live in separate modules.
Goal: to bring the results of both reader functions (which are arrays)into the same file and compare the files for duplicates.
Currently, I have made progress using Promise.all. However, I cannot seem to isolate the two separate arrays so I can use them.
Solution Function (not working):
This is where I am trying to read in both lists into two separate arrays.
The CSVList variable has a console.log that logs the array when the CSVList.filter is not present. Which leads me to believe that the array is actually there? Maybe?
const allData = async function () {
let [result1, result2] = await Promise.all([readCSV, DBList]);
const DBLists = result2(async (domainlist) => {
return domainlist;
});
const CSVList = result1(async (csv) => {
const csvArr = await csv.map((x) => {
return x[0];
});
console.log(csvArr);
return csvArr;
});
const main = await CSVList.filter((val) => !DBLists.includes(vals)); // this doesn't work. it says that filter is not a function. I understand why filter is not a function. What I do not understand is why the array is not being returned?
};
allData();
psql reader:
const { pool } = require("./pgConnect");
//
const DBList = async (callback) => {
await pool
.query(
`
SELECT website
FROM domains
limit 5
`
)
.then(async (data) => {
const domainList = await data.rows.map((x) => {
return x.website;
});
callback(domainList);
});
};
csv reader:
const { parseFile } = require("#fast-csv/parse");
const path = require("path");
const fs = require("fs");
const domainPath = path.join(__dirname, "domains.csv");
//reads initial domain list and pushes the domains to an array
//on end, calls a callback function with the domain data
const readCSV = async (callback) => {
let domainList = [];
let csvStream = parseFile(domainPath, { headers: false })
.on("data", (data) => {
//push csv data to domainList array
domainList.push(data);
// console.log(data);
})
.on("end", () => {
callback(domainList);
});
};
I took jFriend00 Advice and I updated my code a bit.
The biggest issue was the ReadCSV function. Fast-csv doesn't seem to be asynchronous. I wrapped it in a new promise manually. And then resolved that promise passing the domain list as an argument to resolve.
updated CSV Reader:
const readCSV2 = new Promise((resolve, reject) => {
let domainList = [];
let csvStream = parseFile(domainPath, { headers: false })
.on("data", (data) => {
//push csv data to domainList array
domainList.push(data[0]);
// console.log(data);
})
.on("end", () => {
resolve(domainList);
});
});
Updated Solution for comparing the two lists
const allData = async function () {
// get the values from the DB and CSV in one place
let [result1, result2] = await Promise.all([readCSV2, DBList]);
const CSVDomains = await result1;
const DBDomains = await result2();
//final list compares the two lists and returns the list of non duplicated domains.
const finalList = await CSVDomains.filter(
(val) => !DBDomains.includes(val)
);
console.log("The new list is: " + finalList);
};
Quick aside: I could have accomplished the same result by using psql ON CONFLICT DO NOTHING. This would have ignored duplicates when updating to the database because I have a UNIQUE constraint on the domain column.
I have an array of addons and I want to insert them into the db table.
var addons = [sample,sample,.....]
return new Promise((resolve,reject) => {
addons.foEach(async addon => {
// first check if the items is in db
const response = await Kinex.where({}).from('table_name');
if(response.length == 0){
// insert new record
const insertResp = kinex('table_name').insert(addon)
addon.system_id = insertResp[0];
}else{
addon.system_id = response[0].id;
}
})
})
What I expected is to have unique record in the database, but the above code produced duplicate record in the database. Please help to find out the issue with the code.
The problem is running async function inside a loop. As mentioned by #Felix, forEach doesn't know about async functions and doesn't wait for your where query to return. If you wanna do things in async manner inside loops, you can do it with for..of loops. Also make sure to always use try/catch blocks while using async/await. Below is the code in your case:
const addons = [sample,sample,.....];
return new Promise(async (resolve, reject) => {
try {
for (let addon of addons) {
// first check if the items is in db
const response = await Kinex.where({}).from('table_name');
if (response.length) {
const insertResp = await kinex('table_name').insert(addon)
addon.system_id = insertResp[0];
} else addon.system_id = response[0].id;
resolve(); // resolve with whatever you wants to return
}
} catch (e) {
reject(e)
}
});
You can read more on for..of with async/await here.
As pointed by #Sándor, here's the code using Promise.all:
var addons = [sample, sample, .....]
return Promise.all(addons.map(async addon => {
// Do your async stuff here
// first check if the items is in db
const response = await Kinex.where({}).from('table_name');
if (response.length == 0) {
// insert new record
const insertResp = kinex('table_name').insert(addon)
addon.system_id = insertResp[0];
} else {
addon.system_id = response[0].id;
}
}))
There are many threads on this and using that i am trying to find solution for below problem.
getMethod() {
execDBQuery(sqlQuery1)
.then(productionQueryRows => {
prodResultFunc(resultSet) // Calling a function to get another set of DB values
.then (result) {
console.log(result) // Final result which will be sent to user
}
});
}
async function prodResultFunc(prodRowSet) {
const results = await Promise.all(
prodRowSet.map( async (objBatchRow) => {
execDBQuery(sqlQuery2)
.then(resultSet => {
read value-1
})
execDBQuery(sqlQuery3)
.then(resultSet => {
read value-2
})
// Create a object using Value-1 & Value-2 and return object to map
});
);
}
I tried implementing below (with one just SQL execution) and it works fine.
async function prodResultFunc(prodRowSet) {
const results = await Promise.all(
prodRowSet.map( async (objBatchRow) => {
execDBQuery(sqlQuery2)
.then(resultSet => {
read values using resultSet
Create object with resultSet values
})
return object
});
);
}
But i want use values for both SQLs (2 & 3) for creating object and this where i am struggling to find syntax for implementation. Any help/pointers to existing threads will be a great help.
Whenever you're dealing with complex Promise chains, I find that the async/await syntax is much easier to read. You end up with a lot less nested code.
The example below shows how you might combine the results of two different queries:
const prodRowSet = [1,2,3];
// Mock query function.
async function execDBQuery(sql) {
return [{ id: 1, query: sql }];
}
async function prodResultFunc(prodRowSet) {
const results = await Promise.all(
prodRowSet.map( async (objBatchRow) => {
let resultSet2 = await execDBQuery('sqlQuery2');
let resultSet3 = await execDBQuery('sqlQuery3');
// Do whatever you want to combine the result sets.
let combinedObj = { resultSet2, resultSet3}
return combinedObj;
})
);
return results;
}
(async() => {
let output = await prodResultFunc(prodRowSet);
console.log("Output:", output);
})();
You might be looking for Promise.All
async function prodResultFunc(prodRowSet) {
const results = await Promise.all(
prodRowSet.map( async (objBatchRow) => {
Promise.all([execDBQuery(sqlQuery2), execDBQuery(sqlQuery3)]).then(function(values) {
//TODO:: Impement you business logic here
console.log(values);
});
});
);
}
This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Closed 3 years ago.
I'm trying to do upsert inside forEach becase the request is an array of objects using Sequelize.upsert method. I do this:
async createInfo (req, res){
newsData = req.body.news,
newsList = [];
newsData.forEach(async values => {
var news = {};
news.inserted_id = values.inserted_id;
if(news.inserted_id == null){
do {
news.inserted_id = crypto.getRandom(5);
var check = await InstitutionNews.count({where: {inserted_id: news.inserted_id}});
} while (check > 0);
}
News.upsert({
institution_id: institution_id,
inserted_id: news.inserted_id,
news_date: values.news_date,
news_title: values.news_title,
description: values.news_description,
created_by: created_by
}).then(ResNews => {
news.news_date = values.news_date;
news.news_title = values.news_title;
news.description = values.news_description;
newsList.push(news);
})
})
console.log("TEST")
}
but the process stop at the then(). It didn't execute the next code like the console.log.
Is there any way to execute next line code after the then(). I need then() because I wanna push the news object into newsList array. Because I need newsList as the if else conditional to do the next process.
Thanks.
Since it sounds like you need to wait for forEach to complete before you do another step, so I'd suggest using something like Promise.all:
async function createInfo(req, res) {
const newsData = req.body.news
const newsList = []
try {
await Promise.all(
newsData.map(
async (values) => {
const news = {}
news.inserted_id = values.inserted_id
// ...
News.upsert({...})
.then(ResNews => {
// ...
newsList.push(news)
})
}
)
)
console.log('newsList', newList)
// do other work
} catch (error) {
// handle errors appropriately
}
}
This way, you're creating an array of promises and waiting for all of them to resolve/finish.
Your current approach with forEach won't work in this case since it won't wait for each asynchronous call to finish before executing the "next" step. Since Promise.all returns a single Promise that you can then "wait" for to resolve before continuing with your next step.
Here's a simple example that somewhat does what you're trying to do:
async function createInfo() {
const newsData = [1, 2, 3, 4, 5]
const newsList = []
await Promise.all(
newsData.map(
async (values) => {
const temp = await Promise.resolve('temp')
console.log('first async call inside values of', values)
Promise.resolve('resolved')
.then((result) => {
newsList.push(`resolved with ${values}`)
})
}
)
)
console.log('newsList after')
console.log(newsList)
}
createInfo()
EDIT
Here's an alternative solution as rightly pointed by #Bergi in the comments:
async function createInfo(req, res) {
const newsData = req.body.news
try {
const newList = await Promise.all(
newsData.map(
async (values) => {
const news = {}
news.inserted_id = values.inserted_id
// ...
// since it doesn't look like you're using any
// data that you'd get back from the `News.upsert` call, wait for
// it to finish and just simply return your `news` object
await News.upsert({...})
news.news_date = values.news_date
news.news_title = values.news_title
news.description = values.news_description
return news
}
)
)
console.log('newsList', newList)
// do other work
} catch (error) {
// handle errors appropriately
}
}
use await instead of then like
await News.upsert({
institution_id: institution_id,
inserted_id: news.inserted_id,
news_date: values.news_date,
news_title: values.news_title,
description: values.news_description,
created_by: created_by
});
news.news_date = values.news_date;
news.news_title = values.news_title;
news.description = values.news_description;
newsList.push(news);
I have a function which contains a thousand of objects in an array:
function Alltransaction(transactionArray) {
transactionArray.map(async (transaction) => {
dataAgainsthash = await web3.eth.getTransaction(transaction)
TransactionObject = {
transactionHash : transaction,
from : dataAgainsthash.from
};
transactionArray.push(TransactionObject)
console.log("transaction array", transactionArray)
});
}
then i have another function which stores these thousands of object array into db
function saveTransactionToDb() {
console.log("after loop",transactionArray)
transactionss = new Transaction({
blockNumber : blockNumbers ,
transactions : transactionArray
})
// Now save the transaction to database
await transactionss.save();
// console.log("save to database")
}
then I call this in my router
await Alltransaction(transactionArray);
await saveTransactionToDb();
and I also try
Alltransaction(transactionArray).then(saveTransactionToDb())
But it always runs saveTransactionToDb() before the array of object populates the Alltransaction() method
have you try the async keyword before saveTransactionToDb and Alltransaction functions??
async function Alltransaction(transactionArray){
// your code
}
async function saveTransactionToDb(){
// your code logic*
await transactionss.save();
}
First, in Alltransaction the promise must be returned as well. In your code the function starts some processes but doesn't not await on it. Also, do not push the promises to the original array. I'm not sure what you were trying to accomplish there. Because mapping over the array gives you an array of promises, you can unify all of them with Promise.all().
function Alltransaction(transactionArray) {
const promises = transactionArray.map(async (transaction) => {
dataAgainsthash = await web3.eth.getTransaction(transaction)
const TransactionObject = {
transactionHash : transaction,
from : dataAgainsthash.from
};
return TransactionObject;
});
return Promise.all(promises);
}
Change saveTransactionToDb to receive an array instead of using the original array.
Then you'll be able to call it as:
const t = await Alltransaction(transactionArray);
await saveTransactionToDb(t);
Your second try it's not correct:
Alltransaction(transactionArray).then(saveTransactionToDb())
It's the same as:
const t = Alltransaction(transactionArray);
const s = saveTransactionToDb();
t.then(s)
That's why saveTransactionToDb doesn't way for transactions to complete. To use then, just pass the function without calling it:
Alltransaction(transactionArray).then(saveTransactionToDb)