Updating a user concurrently in mongoose using model.save() throws error - node.js

I am getting a mongoose error when I attempt to update a user field multiple times.
What I want to achieve is to update that user based on some conditions after making an API call to an external resource.
From what I observe, I am hitting both conditions at the same time in the processUser() function
hence, user.save() is getting called almost concurrently and mongoose is not happy about that throwing me this error:
MongooseError [ParallelSaveError]: Can't save() the same doc multiple times in parallel. Document: 5ea1c634c5d4455d76fa4996
I know am guilty and my code is the culprit here because I am a novice. But is there any way I can achieve my desired result without hitting this error? Thanks.
function getLikes(){
var users = [user1, user2, ...userN]
users.forEach((user) => {
processUser(user)
})
}
async function processUser(user){
var result = await makeAPICall(user.url)
// I want to update the user based on the returned value from this call
// I am updating the user using `mongoose save()`
if (result === someCondition) {
user.meta.likes += 1
user.markModified("meta.likes")
try {
await user.save()
return
} catch (error) {
console.log(error)
return
}
} else {
user.meta.likes -= 1
user.markModified("meta.likes")
try {
await user.save()
return
} catch (error) {
console.log(error)
return
}
}
}
setInterval(getLikes, 2000)

There are some issues that need to be addressed in your code.
1) processUser is an asynchronous function. Array.prototype.forEach doesn't respect asynchronous functions as documented here on MDN.
2) setInterval doesn't respect the return value of your function as documented here on MDN, therefore passing a function that returns a promise (async/await) will not behave as intended.
3) setInterval shouldn't be used with functions that could potentially take longer to run than your interval as documented here on MDN in the Usage Section near the bottom of the page.
Hitting an external api for every user every 2 seconds and reacting to the result is going to be problematic under the best of circumstances. I would start by asking myself if this is absolutely the only way to achieve my overall goal.
If it is the only way, you'll probably want to implement your solution using the recursive setTimeout() mentioned at the link in #2 above, or perhaps using an async version of setInterval() there's one on npm here

Related

NodeJS await inside forEach function

I am working with MySQL server, building a NodeJS server.
SomeWhere in the code I have async code that is not waiting, I believe its has something to do with the forEach function.
Is there anything wrong with my implementation of my next function:
async addResponsesToTasks(input, response, isFromCache) {
if (!isFromCache) {
this.saveResponseToCache(input, response);
}
console.log("===7===");
await response.forEach(async (pokemon) => {
console.log("===8===", pokemon);
await this.addTaskToFile(pokemon, false);
console.log("===13===");
});
return true;
}
And this is the output shows that it is not waiting for creation and save in the DB to finish, and jump to do some other things:
Prints of the output - can see 10,12,13,14,15 and only then 11
Hope you could maybe spot what I am missing.
Thanks in advance!
What's happening here is more of a JavaScript thing than a Sequelize thing.
The await is being applied to the lines of code that follow Item.create() inside of this function, but outer functions are continuing to execute.
console.log('9')
saveTaskToDb(task)
console.log('12')
In this code, '12' is going to get logged out before your '11', because this outer code is going to continue to execute, even if the code in saveTaskToDb is awaiting the resolution of the create() method.
To solve this, you'll either need to move all of your asynchronous code into the same code block, or you'll need to return your own promise in the saveTaskToDb function, and await saveTaskToDb.
So Solution was not using forEach, rather using for of loop like that:
async addResponsesToTasks(input, response, isFromCache) {
if (!isFromCache) {
this.saveResponseToCache(input, response);
}
for (const pokemon of response) {
await this.addTaskToFile(pokemon, false);
}
return true;
}
Based on this question: Using async/await with a forEach loop

Nodejs loop through array of urls in a synchronous way

i've worked with node now for 2 years but cannot solve the following requirements:
I have an array of ~ 50.000 Parameters
I need to loop through the array and make a get request to always the same url with the parameter added
I need to write the result of the url-call back to the array
It's needed to do this one by one, as i can not call the api with several threads.
I'm sure there is a simple solution for that but everything i tried didn't make the code wait for the get request to return. I know that doing things synchronous in node is not the way we should to things, but in this special situation it is by design that the process shall not go on till the result comes back.
Any hint appreciated
Regards
Use a for loop, use a means of doing the GET request that returns a promise (such as the got() library) and then use await to pause the for loop until your response comes back.
const got = require('got');
const yourArray = [...];
async function run() {
for (let [index, item] of yourArray.entries()) {
try {
let result = await got(item.url);
// do something with the result
} catch(e) {
// either handle the error here or throw to stop further processing
}
}
}
run().then(() => {
console.log("all done");
}).catch(err => {
console.log(err);
});

Node.js waiting for an async Redis hgetall call in a chain of functions

I'm still somewhat new to working with Node and def new to working asynchronously and with promises.
I have an application that is hitting a REST endpoint, then calling a chain of functions. The end of this chain is calling hgetall and I need to wait until I get the result and pass it back. I'm testing with Postman and I'm getting {} back instead of the id. I can console.log the id, so I know that this is because some of the code isn't waiting for the result of hgetall before continuing.
I'm using await to wait for the result of hgetall, but that's only working for the end of the chain. do I need to do this for the entire chain of functions, or is there a way to have everything wait for the result before continuing on? Here's the last bit of the logic chain:
Note: I've removed some of the logic from the below functions and renamed a few things to make it a bit easier to see the flow and whats going on with this particular issue. So, some of it may look a bit weird.
For this example, it will call GetProfileById().
FindProfile(info) {
var profile;
var profileId = this.GenerateProfileIdkey(info); // Yes, this will always give me the correct key
profile = this.GetProfileById(profileId);
return profile;
}
This checks with the Redis exists, to verify if the key exists, then tries to get the id with that key. I am now aware that the Key() returns true instead of what Redis actually returns, but I'll fix that once I get this current issue resolved.
GetProfileById(profileId) {
if ((this.datastore.Key(profileId) === true) && (profileId != null)) {
logger.info('GetProfileById ==> Profile found. Returning the profile');
return this.datastore.GetId(profileId);
} else {
logger.info(`GetProfileById ==> No profile found with key ${profileId}`)
return false;
}
}
GetId() then calls the data_store to get the id. This is also where I started to use await and async to try and wait for the result to come through before proceeding. This part does wait for the result, but the functions prior to this don't seem to wait for this one to return anything. Also curious why it only returns the key and not the value, but when I print out the result in hgetall I get the key and value?
async GetId(key) {
var result = await this.store.RedisGetId(key);
console.log('PDS ==> Here is the GetId result');
console.log(result); // returns [ 'id' ]
return result;
}
and finally, we have the hgetall call. Again, new to promises and async, so this may not be the best way of handling this or right at all, but it is getting the result and waiting for the result before it returns anything
async RedisGetId(key) {
var returnVal;
var values;
return new Promise((resolve, reject) => {
client.hgetall(key, (err, object) => {
if (err) {
reject(err);
} else {
resolve(Object.keys(object));
console.log(object); // returns {id: 'xxxxxxxxxxxxxx'}
return object;
}
});
});
}
Am I going to need to async every single function that could potentially end up making a Redis call, or is there a way to make the app wait for the Redis call to return something, then continue on?
Short answer is "Yes". In general, if a call makes an asynchronous request and you need to await the answer, you will need to do something to wait for it.
Sometimes, you can get smart and issue multiple calls at once and await all of them in parallel using Promise.all.
However, it looks like in your case your workflow is synchronous, so you will need to await each step individually. This can get ugly, so for redis I typically use something like promisify and make it easier to use native promises with redis. There is even an example on how to do this in the redis docs:
const {promisify} = require('util');
const getAsync = promisify(client.get).bind(client);
...
const fooVal = await getAsync('foo');
Makes your code much nicer.

Promise { <pending> } error even after reaching to then() block [duplicate]

This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
Closed 3 years ago.
I am working on a small shopping site using nodejs and mongodb. I have been able to store and retrieve data from my database. However, I can't get this particular function to work which is supposed to retrieve products from user cart. The products are retrieved into the then() block, but when i try to return or print the products by performing some operations in the products, i get output as Promise { pending }.
Before marking this question as duplicate(which it is not, but if you think it is), atleast help me with solve this.
const productIds = this.cart.items.map(eachItem => {
return eachItem.pid;
}); //to get product IDs of all the products from the cart
const cartData = db.collection('products') //db is the database function and works fine
.find({_id: {$in: productIds}})
.toArray()
.then(products => {
console.log(products); //all the products are printed correctly
products.map(eachProduct => {
return {
...eachProduct,
quantity: this.cart.items.find(eachCP => {
return eachCP.pid.toString() === eachProduct._id.toString()
}).quantity //to get the quantity of that specific product (here eachProduct)
};
})
})
.catch(err => {
console.log(err);
});
console.log('cartData: ', cartData); //but here it prints Promise /{ pending /}
I can't understand why i get Promise { } as output although i get the data from the database successfully in the then() block.
Sorry for the messsy code btw. I'm new to mongodb and don't have so much knowledge about promises too.
Promise#then does not "wait" in the sense that the next statement in the program would be delayed until the promise completes.
It "waits" only in the sense that execution of the callback that you pass to then is delayed until the promise completes.
But your current function (the one that set up the then) does not block and continues to run immediately. Meaning that everything outside of the function you pass to then may see the promise in its still incompleted state.
You probably want to make use of the async/await construct, as described (for example) in the linked duplicate thread.

Unable to understand why the try and catch is not working as expected in mongoose

I am new to mongoose.I am using Sails js, Mongo DB and Mongoose in my project. My basic requirement was to find details of all the users from my user collection. My code is as follows:
try{
user.find().exec(function(err,userData){
if(err){
//Capture the error in JSON format
}else{
// Return users in JSON format
}
});
}
catch(err){
// Error Handling
}
Here user is a model which contains all the user details. I had sails lifted my app and then I closed my MongoDB connection. I ran the API on DHC and found the following:
When I ran the API for the first time on DHC, the API took more than 30 sec to show me an error that the MongoDB connection is not avaliable.
When I ran the API for the second time, The API timed out without giving an response.
My Question here why is the try and catch block unable to handle such an error exception effectively in mongoose or is it something that I am doing wrong?
EDIT
My Requirement is that mongoose should display the error immediately if the DB connection is not present.
First let’s take a look at a function that uses a synchronous usage pattern.
// Synchronous usage example
var result = syncFn({ num: 1 });
// do the next thing
When the function syncFn is executed the function executes in sequence until the function
returns and you’re free to do the next thing. In reality, synchronous functions should be
wrapped in a try/catch. For example the code above should be written like this:
// Synchronous usage example
var result;
try {
result = syncFn({ num: 1 });
// it worked
// do the next thing
} catch (e) {
// it failed
}
Now let’s take a look at an asynchronous function usage pattern.
// Asynchronous usage example
asyncFn({ num: 1 }, function (err, result) {
if (err) {
// it failed
return;
}
// it worked
// do the next thing
});
When we execute asyncFn we pass it two arguments. The first argument is the criteria to be used by the function. The second argument is a callback that will execute whenever asyncFn calls the callback. asyncFn will insert two arguments in the callback – err and result). We
can use the two arguments to handle errors and do stuff with the result.
The distinction here is that with the asynchronous pattern we do the next thing within the callback of the asynchronous function. And really that’s it.

Resources