recursive retry function in nodejs [duplicate] - node.js

This question already has answers here:
Promise Retry Design Patterns
(21 answers)
Closed 4 years ago.
I'm new to nodejs and ES6 and trying to get my head around promises. I have a requirement to retry a dynamodb query function for specific intervals (5 seconds in this case) if the the result is not acceptable! So I have a function like this:
const retryFunc = (ddbParams, numberOfRetry) => {
return new Promise((resolve, reject) => {
return DDBUtils.query(ddbParams).then(result => {
//need to return a specific number of rows from the ddb table
if(numberOfRetry > 0){
if(result.length !== 100){
numberOfRetry = numberOfRetry - 1
setTimeout(() => {
retryFunc(ddbParams, numberOfRetry)
}, 5000)
}
}
resolve(result)
}).catch(error => {
reject(error)
})
})
}
When the dynamodb query returning the acceptable result (100 records) in the first call then the function working fine and returning the result to the caller. But if the function needs to be retried to satisfied the 100 condition then it is not returning the result to the caller when it gets satisfied! Can anybody help me to understand what is happening?

First, avoid the explicit promise construction antipattern - .query already returns a Promise, so there's no need to construct another one. Then, you inside your if(result.length !== 100){, you need to be able to chain together recursive calls of retryFunc; you can't directly return from an (asynchronous, callback-based) setTimeout, as with your current code.
One option would be to create a delay function, which returns a Promise that resolves after the desired amount of time - then, you can use return delay(5000).then(() => retryFunc(ddbParams, numberOfRetry - 1)) to return the recursive call.
const delay = ms => new Promise(res => setTimeout(res, ms));
const retryFunc = (ddbParams, numberOfRetry) => {
return DDBUtils.query(ddbParams).then(result => {
if(numberOfRetry > 0 && result.length !== 100) {
return delay(5000).then(() => retryFunc(ddbParams, numberOfRetry - 1));
}
});
}

Related

Node.js how to .then after declaring a variable as a function that has a promise inside [duplicate]

This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
How can I access the value of a promise?
(14 answers)
Closed 2 months ago.
I am attempting to set a variable scannedCode to a scanned qr code result. However, the steps after that happens continues even before scannedCode is in use, due to the promises. How can I make it so nothing continues until the variable finishes and is usable?
function stuff(image) {
var scannedCode = qrRun(image);
console.log("I want to console.log scannedcode and it be the actual thing completed")
}
async function qrRun(image) {
const img = await jimp.read(fs.readFileSync(image));
const qr = new QRReader();
const value = await new Promise((resolve, reject) => {
qr.callback = (err, v) => err != null ? reject(err) : resolve(v);
qr.decode(img.bitmap);
});
console.log(value.result);
return value.result;
}
Or if there is no way to do that, how would I just remove the Promise from the function currently there so I don't need a .then without breaking the qrrun.
async function stuff(image) {
const scannedCode = await qrRun(image);
}

Trouble with Promise and Async functions

I am trying to use promises and async functions to send chunks of an array to an api call that inserts them into a DB. I am trying to get the top function to chunk the array and await for the backend to finsih then it will move on to the next chunk. It errors out after the first iteration. Any ideas??
async chunkArray(arr) {
let len = arr.length
let update_arr
let i = 1
for(i; i<=len; i++) {
if((i%125) === 0) {
update_arr = arr.slice(i-125,i)
await this.updateChuckArray(update_arr)
} else if(i === len) {
let div = (Math.floor(len/125) * 125)
update_arr = arr.slice(div, len)
await this.updateChuckArray(update_arr)
}
}
},
updateChuckArray(update) {
return new Promise(resolve => {
this.$http.put(`/route`, update).then(res => {
res.data.error ? this.$root.updateError(res.data.error) : this.$root.updateSuccess(res.data.message)
}).catch(error => {
this.$root.updateError(res.data.error)
})
})
}
First off your updateChuckArray() never resolves the promise it returns (you never call resolve()).
Instead of manually wrapping a new promise around your function call (that is a promise anti-pattern), you can just return the promise you already have and write it like this:
updateChuckArray(update) {
return this.$http.put(`/route`, update).then(res => {
res.data.error ? this.$root.updateError(res.data.error) : this.$root.updateSuccess(res.data.message);
}).catch(error => {
this.$root.updateError(error);
})
}
FYI, it's unclear what your error handling strategy is. The way you wrote the code (which is followed above), you catch an error from this.$http.put() and handle it and let the loop continue. If that's what you want, this will work. If you want the for loop to abort on error, then you need to rethrow the error in the .catch() handler so the error gets back to the await.
Also, not that in your .catch() handler, you were doing this:
this.$root.updateError(res.data.error)
but there is no res defined there. The error is in error. You would need to use that in order to report the error. I'm not sure what the structure of the error object is here or what exactly you pass to $.root.updateError(), but it must be something that comes from the error object, not an object named res.

Node.js - for loop index is update to last value (surprisingly) [duplicate]

This question already has answers here:
Asynchronous Process inside a javascript for loop [duplicate]
(6 answers)
Closed 4 years ago.
For loop in node.js is behaving quite weirdly. code is as below:
for(var i = 0; i < req.body.stages.length; i++){
if (req.body.current_stage == req.body.stages[i].stage_short_desc){
console.log('value of i :'+ i)
Application.findOne({ application_num: req.params.id }, (error, application) => {
if (error) {
logger.error('Application not found' + error)
} else {
console.log('reached here : ' + i)
console.log(req.body.stages[i].requisites.length)
....
}
})
}
}
And result is:
value of i :0
reached here : 8
error: uncaughtException: Cannot read property 'requisites' of undefined.
req.body.current_stage == req.body.stages[i].stage_short_desc ---> happens at i = 0
I am surprised, when index i was 0, it is entering into if loop and satisfies the condition, hence first line of result. Then find the application from database, if successful then if we notice value of index i is 8 (which is maximum value of index). Can anyone please suggest how this is possible?
Thanks!
The for loop continues to run instantly, while the findOne is executed asynchronously for each step of the loop. When the callback for each findOne is called, the loop has already finished, hence i is 8 at that point, every time.
This happens because you're trying to execute an asynchronous code involving callbacks inside a synchronous piece of code. For your code to work the way intended, you may promisify the mongoose methods, and then push that into an array of promises that you may choose to resolve later :
const promises = [];
for (let i = 0; i < req.body.stages.length; i++) {
if (req.body.current_stage == req.body.stages[i].stage_short_desc) {
console.log('value of i :' + i);
// Don't forget to promisify mongoose methods
promises.push(Application.findOne({ application_num: req.params.id }));
}
}
Promise.all(promises).then((application) => {
// Do your magic here
}).catch((err) => {
// Again your magic
})

How to stop rethink database call if cannot find result?

I run RethinkDB command with Node.js (babel) asynchronous call:
let user = await r.table('users').filter({key: key}).limit(1).run();
How can I stop the asynchronous call, if database cannot find result?
Using the await function means that node will wait for the asynchronous r.table(... command to return before continuing to the next line of code, meaning that it behaves logically as if it were synchronous code.
Your specific command should return when RethinkDB finds the first 'user' document with the specified key. There is no need to "stop" it if it cannot find a result, it will stop as soon as it (a) finds a result or (b) finished scanning the entire table.
In general "stopping" asynchronous code in node/javascript is not possible but you can limit the amount of time you'll wait for an async method. Here is an example using the Promise.race() function.
/*
* toy async function
*
* returns a promise that resolves to the specified number `n`
* after the specified number of seconds `s` (default 2)
*/
const later = (n, s=2) => {
return new Promise(resolve => {
setTimeout(() => resolve(n), s*1000);
})
}
/*
* returns a promise that rejects with `TIMEOUT_ERROR` after the
* specified number of seconds `s`
*/
const timeout = (s) => {
return new Promise((resolve, reject) => {
setTimeout(() => reject("TIMEOUT_ERROR"), s*1000)
})
}
/*
* Example 1: later finished before timeout
* later resolves after 1 second, timeout function rejects after 3 seconds
* so we end up in the `.then` block with `val == 42`
*/
Promise.race([later(42, 1), timeout(3)])
.then(val => {
// do somethign with val...
console.log(val)
}).catch(err => {
if (err === "TIMEOUT_ERROR") {
console.log("we timed out!")
} else {
consle.log("something failed (but it was not a timeout)")
}
});
/*
* Example 2 (using async/await syntax): we timeout before later returns.
* later resolves after 3 seconds, timeout function rejects after 2 seconds
* so we end up in the `.catch` block with `err == "TIMEOUT_ERROR"`
*/
try {
const val = await Promise.race([later(11, 3), timeout(2)]);
// do something with val...
} catch (err) {
if (err === "TIMEOUT_ERROR") {
console.error("we timed out!")
} else {
console.error("something failed (but it was not a timeout)")
}
}

Node promises: use first/any result (Q library)

I understand using the Q library it's easy to wait on a number of promises to complete, and then work with the list of values corresponding to those promise results:
Q.all([
promise1,
promise2,
.
.
.
promiseN,
]).then(results) {
// results is a list of all the values from 1 to n
});
What happens, however, if I am only interested in the single, fastest-to-complete result? To give a use case: Say I am interested in examining a big list of files, and I'm content as soon as I find ANY file which contains the word "zimbabwe".
I can do it like this:
Q.all(fileNames.map(function(fileName) {
return readFilePromise(fileName).then(function(fileContents) {
return fileContents.contains('zimbabwe') ? fileContents : null;
}));
})).then(function(results) {
var zimbabweFile = results.filter(function(r) { return r !== null; })[0];
});
But I need to finish processing every file even if I've already found "zimbabwe". If I have a 2kb file containing "zimbabwe", and a 30tb file not containing "zimbabwe" (and suppose I'm reading files asynchronously) - that's dumb!
What I want to be able to do is get a value the moment any promise is satisfied:
Q.any(fileNames.map(function(fileName) {
return readFilePromise(fileName).then(function(fileContents) {
if (fileContents.contains('zimbabwe')) return fileContents;
/*
Indicate failure
-Return "null" or "undefined"?
-Throw error?
*/
}));
})).then(function(result) {
// Only one result!
var zimbabweFile = result;
}).fail(function() { /* no "zimbabwe" found */ });
With this approach I won't be waiting on my 30tb file if "zimbabwe" is discovered in my 2kb file early on.
But there is no such thing as Q.any!
My question: How do I get this behaviour?
Important note: This should return without errors even if an error occurs in one of the inner promises.
Note: I know I could hack Q.all by throwing an error when I find the 1st valid value, but I'd prefer to avoid this.
Note: I know that Q.any-like behavior could be incorrect, or inappropriate in many cases. Please trust that I have a valid use-case!
You are mixing two separate issues: racing, and cancelling.
Racing is easy, either using Promise.race, or the equivalent in your favorite promise library. If you prefer, you could write it yourself in about two lines:
function race(promises) {
return new Promise((resolve, reject) =>
promises.forEach(promise => promise.then(resolve, reject)));
}
That will reject if any promise rejects. If instead you want to skip rejects, and only reject if all promises reject, then
function race(promises) {
let rejected = 0;
return new Promise((resolve, reject) =>
promises.forEach(promise => promise.then(resolve,
() => { if (++rejected === promises.length) reject(); }
);
}
Or, you could use the promise inversion trick with Promise.all, which I won't go into here.
Your real problem is different--you apparently want to "cancel" the other promises when some other one resolves. For that, you will need additional, specialized machinery. The object that represents each segment of processing will need some way to ask it to terminate. Here's some pseudo-code:
class Processor {
promise() { ... }
terminate() { ... }
}
Now you can write your version of race as
function race(processors) {
let rejected = 0;
return new Promise((resolve, reject) =>
processors.forEach(processor => processor.promise().then(
() => {
resolve();
processors.forEach(processor => processor.terminate());
},
() => { if (++rejected === processors.length) reject(); }
);
);
}
There are various proposals to handle promise cancellation which might make this easier when they are implemented in a few years.

Resources