Waiting in a while loop on an async function (Node.js/ES6) - node.js

I'm writing a Windows Node.js server app (using ES6 btw).
The first thing I want to do - in the top-level code - is sit in a while loop, calling an async function which searches for a particular registry key/value. This function is 'proven' - it returns the value data if found, or else throws:
async GetRegValue(): Promise<string> { ... }
I need to sit in a while loop until the registry item exists, and then grab the value data. (With a delay between retries).
I think I know how to wait for an async call to complete (one way or the other) before progressing with the rest of the start-up, but I can't figure out how to sit in a loop waiting for it to succeed.
Any advice please on how to achieve this?
(I'm fairly new to typescript, and still struggling to get my head round all async/await scenarios!)
Thanks
EDIT
Thanks guys. I know I was 'vague' about my code - I didn't want to put my real/psuedo code attempts, since they have all probably overlooked the points you can hopefully help me understand.
So I just kept it as a textual description... I'll try though:
async GetRegValue(): Promise<string> {
const val: RegistryItem = await this.GetKeyValue(this.KEY_SW, this.VAL_CONN);
return val.value
}
private async GetKeyValue(key: string, name: string): Promise<RegistryItem> {
return await new Promise((resolve, reject) => {
new this.Registry({
hive: this.Hive, key
}).get(name, (err, items) => {
if (err) {
reject(new Error('Registry get failed'));
}
else {
resolve( items );
}
});
})
.catch(err => { throw err });
}
So I want to do something like:
let keyObtained = false
let val
while (keyObtained == false)
{
// Call GetRegValue until val returned, in which case break from loop
// If exception then pause (e.g. ~100ms), then loop again
}
}
// Don't execute here till while loop has exited
// Then use 'val' for the subsequent statements
As I say, GetRegValue() works fine in other places I use it, but here I'm trying to pause further execution (and retry) until it does come back with a value

You can probably just use recursion. Here is an example on how you can keep calling the GetRegValue function until is resolves using the retryReg function below.
If the catch case is hit, it will just call GetRegValue over and over until it resolves successfully.
you should add a counter in the catch() where if you tried x amount of times you give up.
Keep in mind I mocked the whole GetRegValue function, but given what you stated this would still work for you.
let test = 0;
function GetRegValue() {
return new Promise((resolve, reject) => {
setTimeout(function() {
test++;
if (test === 4) {
return resolve({
reg: "reg value"
});
}
reject({
msg: "not ready"
});
}, 1000);
});
}
function retryReg() {
GetRegValue()
.then(registryObj => {
console.log(`got registry obj: ${JSON.stringify(registryObj)}`)
})
.catch(fail => {
console.log(`registry object is not ready: ${JSON.stringify(fail)}`);
retryReg();
});
}
retryReg();

I don't see why you need this line:
.catch(err => { throw err });
The loop condition of while isn't much use in this case, as you don't really need a state variable or expression to determine if the loop should continue:
let val;
while (true)
{
try {
val = await GetRegValue(/* args */);
break;
} catch (x) {
console.log(x); // or something better
}
await delay(100);
}
If the assignment to val succeeds, we make it to the break; statement and so we leave the loop successfully. Otherwise we jump to the catch block and log the error, wait 100 ms and try again.
It might be better to use a for loop and so set a sensible limit on how many times to retry.
Note that delay is available in an npm package of the same name. It's roughly the same as:
await new Promise(res => setTimeout(res, 100));

Related

Wait for loop is finished to run the rest of the program

I want to push on my array all the data which are on the feed variable. I need to access to the data array variable outside of the foreach loop.
But the console.log is execute before the loop.
I have try to change the promise to an async await and it is the same.
let data = []
sources.forEach((src) => {
parser.parseURL(src.rss, (err, feed) => {
data.push(feed)
})
})
console.log(data)
Thank you to you help.
I could make an ad-hoc debounce function.. debounce as in it'll run n milliseconds AFTER it's finished calling.. surprisingly, 0 milliseconds would still occur after an instruction written below it.. using that logic, here's an example
var debObj={}; //empty object SOLELY for temporary storage of the debounce function
function debounce(fn,n,obj){ //debounce function
function doIt(){
obj[fn.toString()]=[fn,setTimeout(()=>{
fn();delete(obj[fn.toString()])
},n)]
}
if(obj[fn.toString()]){
clearTimeout(obj[fn.toString()][1])
doIt(); return;
}
doIt(); return;
}
function debouncer(fn,n){return function(){debounce(fn,n,debObj)}} //debounce function that imitates the functionality of the real debounce
//now for your code snippet...............................................................
let data = []
let logIt=debouncer(()=>console.log(data),0)
sources.forEach((src) => {
parser.parseURL(src.rss, (err, feed) => {
data.push(feed)
})
logIt()
})
I know it's extremely adhoc.. but does it work?

Model.findOne() function not working when passed a variable as the value

I am new to node.js.
I am trying to create function, where a randomly generated String is queried to check if it exists or not. If it already exists, the String is randomly generated till it is unique.
let validID = false;
console.log(temp); //temp is the randomly generated String.
while(!validID){
Website.findOne({shortcut: temp},function(err,docs){
if(docs==null){
validID = true;
console.log("The shortcut for the url is" + temp);
}else{
console.log("FOUND");
temp = generateId();
}
});
}
When run, the code is stuck in an infinite while loop.
I tried to see whether the code works with a String value ( not a variable ) passed in as the query inside findOne(). It worked. I am assuming that the fact that temp is a variable is causing the problem. Can variables be passed in as a value in a query? If so, what is the correct way?
Website.findOne operates asynchronously, i.e. the callback-function you passed to it, will be run once the results from the mongodb are fetched. However, node will not be able to actually process this callback, since your callstack never gets emptied due to your while-loop. If you're interested, you can find out more about this here.
One way to solve this is to wrap your Mongo-DB call in a promise, wait for it to resolve, then return if the ID is unique and continue by calling it recursively otherwise (note that this can be highly simplified by using async/await but for understanding how this works using promised are beneficial imo):
function findIdPromise(temp) {
return new Promise((resolve, reject) => {
Website.findOne({
shortcut: temp
}, function (err, docs) {
if (err) {
return reject(err);
}
resolve(docs);
});
});
}
function getNextIsUniqueIdPromise(shortcut) {
return findIdPromise()
.then(docs => {
if (docs == null) {
return shortcut;
}
return getNextIsUniqueIdPromise(generateId());
});
}
// call it initially with
getNextIsUniqueIdPromise(firstShortcutToCheck)
.then(shortcut => {
console.log("The shortcut for the url is" + shortcut):
})
.catch(err => {
console.log("an error occured", err):
});

Node Synchronous loops

I've got a function.
function async extractTars (tarList) {
try {
for (let i = 0; i < tarList.length; i ++) {
// I need this loop to be sync but it isn't hitting the next iteration
return new Promise((resolve, reject) => {
fs.createReadStream(`${PATHS.TAR}/${tarList[i]}`)
.pipe(tar.extract(PATHS.GZ))
.on('error', err => reject(err))
.on('finish', () => resolve())
})
}
// What is the correct way to resolve this async fn? Should i just return or will it resolve anyway after the loop?
} catch (e) {
// handle error
}
}
For some reason it never hits the next iteration of the loop. What am i missing here? How can i make these type of loops synchronous. In this particular project i've got many many loops that need to complete before the next. I've tried several methods that i've see here on SO but i'm obviously missing something.
Any help would be much appreciated!
You're using return, that's why you're not hitting the next iteration, use await instead to wait until the promise is done before going to the next iteration.
return inside a for will stop the loop & end the function returning the new Promise
function async extractTars (tarList) {
try {
for (let i = 0; i < tarList.length; i ++) {
// I need this loop to be sync but it isn't hitting the next iteration
await new Promise((resolve, reject) => {
fs.createReadStream(`${PATHS.TAR}/${tarList[i]}`)
.pipe(tar.extract(PATHS.GZ))
.on('error', err => reject(err))
.on('finish', () => resolve())
})
}
// What is the correct way to resolve this async fn? Should i just return or will it resolve anyway after the loop?
} catch (e) {
// handle error
}
}
And just to clear things up:
I need this loop to be sync but it isn't hitting the next iteration
That loop won't be sync, it will look like it's sync, when using async/await but you can't make asynchronous code be synchronous. And createReadStream is asnychronous.
And regarding:
What is the correct way to resolve this async fn? Should i just
return or will it resolve anyway after the loop?
When all iterations are done or an error is thrown (since you're catching it and not rejecting), meaning that you read all the files in tarList, the function will be resolved. Since you are not returning anything, undefined will be the resolved value.
From MDN:
When a return statement is used in a function body, the execution of the function is stopped. If specified, a given value is returned to the function caller. For example, the following function returns the square of its argument, x, where x is a number.

Node promises: use first/any result (Q library)

I understand using the Q library it's easy to wait on a number of promises to complete, and then work with the list of values corresponding to those promise results:
Q.all([
promise1,
promise2,
.
.
.
promiseN,
]).then(results) {
// results is a list of all the values from 1 to n
});
What happens, however, if I am only interested in the single, fastest-to-complete result? To give a use case: Say I am interested in examining a big list of files, and I'm content as soon as I find ANY file which contains the word "zimbabwe".
I can do it like this:
Q.all(fileNames.map(function(fileName) {
return readFilePromise(fileName).then(function(fileContents) {
return fileContents.contains('zimbabwe') ? fileContents : null;
}));
})).then(function(results) {
var zimbabweFile = results.filter(function(r) { return r !== null; })[0];
});
But I need to finish processing every file even if I've already found "zimbabwe". If I have a 2kb file containing "zimbabwe", and a 30tb file not containing "zimbabwe" (and suppose I'm reading files asynchronously) - that's dumb!
What I want to be able to do is get a value the moment any promise is satisfied:
Q.any(fileNames.map(function(fileName) {
return readFilePromise(fileName).then(function(fileContents) {
if (fileContents.contains('zimbabwe')) return fileContents;
/*
Indicate failure
-Return "null" or "undefined"?
-Throw error?
*/
}));
})).then(function(result) {
// Only one result!
var zimbabweFile = result;
}).fail(function() { /* no "zimbabwe" found */ });
With this approach I won't be waiting on my 30tb file if "zimbabwe" is discovered in my 2kb file early on.
But there is no such thing as Q.any!
My question: How do I get this behaviour?
Important note: This should return without errors even if an error occurs in one of the inner promises.
Note: I know I could hack Q.all by throwing an error when I find the 1st valid value, but I'd prefer to avoid this.
Note: I know that Q.any-like behavior could be incorrect, or inappropriate in many cases. Please trust that I have a valid use-case!
You are mixing two separate issues: racing, and cancelling.
Racing is easy, either using Promise.race, or the equivalent in your favorite promise library. If you prefer, you could write it yourself in about two lines:
function race(promises) {
return new Promise((resolve, reject) =>
promises.forEach(promise => promise.then(resolve, reject)));
}
That will reject if any promise rejects. If instead you want to skip rejects, and only reject if all promises reject, then
function race(promises) {
let rejected = 0;
return new Promise((resolve, reject) =>
promises.forEach(promise => promise.then(resolve,
() => { if (++rejected === promises.length) reject(); }
);
}
Or, you could use the promise inversion trick with Promise.all, which I won't go into here.
Your real problem is different--you apparently want to "cancel" the other promises when some other one resolves. For that, you will need additional, specialized machinery. The object that represents each segment of processing will need some way to ask it to terminate. Here's some pseudo-code:
class Processor {
promise() { ... }
terminate() { ... }
}
Now you can write your version of race as
function race(processors) {
let rejected = 0;
return new Promise((resolve, reject) =>
processors.forEach(processor => processor.promise().then(
() => {
resolve();
processors.forEach(processor => processor.terminate());
},
() => { if (++rejected === processors.length) reject(); }
);
);
}
There are various proposals to handle promise cancellation which might make this easier when they are implemented in a few years.

What is a sensible way to structure my control flow (promises and looping)?

I'm not sure of how to adequately achieve my desired control flow using promises/bluebird.
Essentially I have a database with X 'tasks' stored and each needs to be loaded and executed sequentially. I don't want to run more than one task concurrently and the entire code must continue executing indefinitely.
I have achieved this with the following code so far:
export default function syncLoop() {
getNextTaskRunner().then((taskRunner) => {
if (taskRunner) {
taskRunner.startTask()
.then(syncLoop)
.catch((error) => {
throw new Error(error);
});
} else {
syncLoop();
}
});
}
getNextTaskRunner() simply loads and resolves with the next task from the database (calc'd based on timestamps). Or it resolves with null (no task avail).
taskRunner.startTask() resolves with null when the full task has completed.
I've been advised that the way it is structured (recursive /w promises) could lead to stack issues after it has been running for some time.
What I've thought about doing is to restructure it to something like:
let running = false;
setInterval(() => {
if (!running) {
running = true;
getNextTaskRunner().then((taskRunner) => {
if (taskRunner) {
taskRunner.startTask()
.then(() => {
running = false;
})
.catch((error) => {
log.error(error);
});
} else {
running = false;
}
});
}
}, 5000);
Or as yet another possibility, using event emitters in some form?
task.on('complete', nextTask());
Thoughts and advice will be greatly appreciated!
What stack issues? The way you've written your code is perfectly fine as long as getNextTaskRunner is truly async (i.e. it gives control back to the main loop at some point, e.g. if it does async io). There is no recursion in your code in that case. Whoever told you that is mistaken.
Though you might want to add a setTimeout somewhere so you won't flood your db with requests. Plus it will help you if getNextTaskRunner will no longer be sync (due to for example in memory caching):
export default function syncLoop() {
setTimeout(() => {
getNextTaskRunner().then((taskRunner) => {
if (taskRunner) {
taskRunner.startTask()
.then(syncLoop)
.catch((error) => {
throw new Error(error);
});
} else {
syncLoop();
}
});
}, 2000);
}

Resources