How to handle for loop with promises conditionally - node.js

I am still trying to find a way to handle promises in a loop conditionally with the ability to break out of the loop.
Here is a short example
return new Promise(function (resolve, reject) {
if (ipAddresses.length > 0) {
let currentServer, agentOptions;
for (let i = 0; i < ipAddresses.length; i++) {
currentServer = ipAddresses[i];
agentOptions = {
};
// We need to block here
let isReachable = NetworkUtils.checkIfReachable(agentOptions, ip);
if (isReachable) {
resolve(currentServer);
// Break out of the loop
return currentServer;
}
else {
// Continue looping and trying to find a working server
}
}
reject(new Error("No working servers found"));
}
else {
resolve(new Error("No servers ips provided"));
}
})
The problem is that I don't need to run all requests in parallel lie Promise.all or async.foreach, but I rather need to call each promise sequentually and if condition is true I need to break out the loop and do not make any further requests as far as I have found a reachable server.
Please suggest what is the correct way to handle this use case. I have been searching for a long time, but still haven't found any good solution.
Thanks
EDIT
Sorry, yeah NetworkUntils.checkIfReachable() returns promise

Assuming NetworkUntils.checkIfReachable() is actually asynchronous (which is the only context in which this question makes sense and seems likely in nodejs) and assuming that NetworkUntils.checkIfReachable() returns a promise or can easily be changed to return a promise, then you can do the following:
findFirstReachableServer(ipAddresses) {
if (!ipAddresses || !ipAddresses.length) {
return Promise.reject(new Error("No servers ips provided"));
}
let agentOptions = {...};
let index = 0;
function next() {
if (index < ipAddresses.length) {
let ipAddress = ipAddresses[index++];
return NetworkUtils.checkIfReachable(agentOptions, ipAddress).then(function(isReachable) {
if (!isReachable) {
return next();
} else {
return ipAddress;
}
})
} else {
return new Error("No working servers found");
}
}
return Promise.resolve().then(next);
}
This function returns a promise that resolves with the ipAddress if a reachable one was found. It rejects if either no addresses were passed in, no reachable address was found or if NetworkUtils.checkIfReachable() rejects for any internal reason.
Note, to run non-blocking async operations in sequence, you can't use a normal for loop because each individual operation won't block so the for loop won't wait for them (it will just run to completion before any operations have finished). As such, you have to use a different method of sequencing. There are lots of different ways to do that. Since you don't necessarily want to run the entire sequence, I've chosen a manual sequencing where you control whether the next iteration is called or not.

Given you tagged the question async-await, using that is not a bad idea indeed:
async function findWorkingServer(ipAddresses) {
if (ipAddresses.length > 0) {
for (const currentServer of ipAddresses) {
const agentOptions = { … };
const isReachable = await NetworkUtils.checkIfReachable(agentOptions, ip);
// We do "block" here ^^^^^
if (isReachable) {
return currentServer;
}
}
throw new Error("No working servers found");
} else {
return new Error("No servers ips provided"); // I think you meant to `throw` here
}
}

Related

Returning a value from a mix of async/sync function from a provider (different script) to express server

Apologies for asking this question - I know there are tons of information about async functions out there but I seem to have tried everything and cannot find a solution..
First of all let me outline the architecture of my program. There are two scripts: a main server script (node.js, express), which processes GET requests and provider script, which deals with the blockchain in the background to return some values. The server script is responsible for invoking a method that returns a value from the provider. The provider does all the work.
The snippet of the provider script:
getInfo(index, account, key) {
//Waiting on an asynchronous method, which does some work in the blockchain in the background; everything functions as it should be
try {
await this.getBlockchain
(
index
, account
, key
).then(result => {
// Here instead I invoke a SYNCHRONOUS method, which simply formats the response in a correct way
const reply = this.sendReply(result)
console.log(reply) //Logs the correct reply in the format in which the server is expecting it
return reply;
});
}
catch (error) {
return { error: 003, result: false };
}
}
The snippet of the server script:
server.get("/getAccount", async (req, res) => {
let index = req.query.index;
let account = req.query.account;
let key = req.query.key;
// Here I also check for the validity of the query values, irrelevant to this issue
// The provider class is imported as provider, hence, the provider.method (this has been tested many times before)
try {
await provider.getInfo(index, account, key).then(reply => {
const { error: infoError, result: infoValue } = reply
if (infoError == false) {
res.send(`${infoValue}`);
} else {
res.send(`${infoError}`);
};
});
}
catch (error) {
res.send("008");
}
}
);
I honestly have no idea how to approach this; I tried self-contained async function on the server side as well as different syntax but the reply is always undefined even though the reply from a synchronous call in the provider is correct.
Could someone help me to understand what I'm doing wrong? This is my first time working with async with numerous scripts and functions and I'm finding it very confusing.
Thank you so much!
With your current structure, you need to return the result of the await so that the top level of your function is returning something from the async function.
async getInfo(index, account, key) {
try {
let retVal = await this.getBlockchain(index, account, key).then(result => {
return this.sendReply(result);
});
return retVal;
} catch (error) {
return { error: 003, result: false };
}
}
But, really, it's a better coding style to not mix await and .then() and to just go with one style like this:
async getInfo(index, account, key) {
try {
let result = await this.getBlockchain(index, account, key);
return this.sendReply(result);
} catch (error) {
return { error: 003, result: false };
}
}
Note, this function never rejects because it's catching its own rejections and turning it into a resolved value. So, the caller cannot use .catch() to see errors. The caller must always check for the error property in the resolved object. This is not usually how you program with promises. It can be made to work, but often does not meet the expectations of the caller (as errors are usually communicated back via rejected promises).
This has to be a dup. but... Don't mix await and .then.
You simply try/catch around await.
try {
const reply = await provider.getInfo(index, account, key);
const { error: infoError, result: infoValue } = reply
if (infoError == false) {
res.send(`${infoValue}`);
} else {
res.send(`${infoError}`);
};
} catch (error) {
res.send(500);
}

Waiting in a while loop on an async function (Node.js/ES6)

I'm writing a Windows Node.js server app (using ES6 btw).
The first thing I want to do - in the top-level code - is sit in a while loop, calling an async function which searches for a particular registry key/value. This function is 'proven' - it returns the value data if found, or else throws:
async GetRegValue(): Promise<string> { ... }
I need to sit in a while loop until the registry item exists, and then grab the value data. (With a delay between retries).
I think I know how to wait for an async call to complete (one way or the other) before progressing with the rest of the start-up, but I can't figure out how to sit in a loop waiting for it to succeed.
Any advice please on how to achieve this?
(I'm fairly new to typescript, and still struggling to get my head round all async/await scenarios!)
Thanks
EDIT
Thanks guys. I know I was 'vague' about my code - I didn't want to put my real/psuedo code attempts, since they have all probably overlooked the points you can hopefully help me understand.
So I just kept it as a textual description... I'll try though:
async GetRegValue(): Promise<string> {
const val: RegistryItem = await this.GetKeyValue(this.KEY_SW, this.VAL_CONN);
return val.value
}
private async GetKeyValue(key: string, name: string): Promise<RegistryItem> {
return await new Promise((resolve, reject) => {
new this.Registry({
hive: this.Hive, key
}).get(name, (err, items) => {
if (err) {
reject(new Error('Registry get failed'));
}
else {
resolve( items );
}
});
})
.catch(err => { throw err });
}
So I want to do something like:
let keyObtained = false
let val
while (keyObtained == false)
{
// Call GetRegValue until val returned, in which case break from loop
// If exception then pause (e.g. ~100ms), then loop again
}
}
// Don't execute here till while loop has exited
// Then use 'val' for the subsequent statements
As I say, GetRegValue() works fine in other places I use it, but here I'm trying to pause further execution (and retry) until it does come back with a value
You can probably just use recursion. Here is an example on how you can keep calling the GetRegValue function until is resolves using the retryReg function below.
If the catch case is hit, it will just call GetRegValue over and over until it resolves successfully.
you should add a counter in the catch() where if you tried x amount of times you give up.
Keep in mind I mocked the whole GetRegValue function, but given what you stated this would still work for you.
let test = 0;
function GetRegValue() {
return new Promise((resolve, reject) => {
setTimeout(function() {
test++;
if (test === 4) {
return resolve({
reg: "reg value"
});
}
reject({
msg: "not ready"
});
}, 1000);
});
}
function retryReg() {
GetRegValue()
.then(registryObj => {
console.log(`got registry obj: ${JSON.stringify(registryObj)}`)
})
.catch(fail => {
console.log(`registry object is not ready: ${JSON.stringify(fail)}`);
retryReg();
});
}
retryReg();
I don't see why you need this line:
.catch(err => { throw err });
The loop condition of while isn't much use in this case, as you don't really need a state variable or expression to determine if the loop should continue:
let val;
while (true)
{
try {
val = await GetRegValue(/* args */);
break;
} catch (x) {
console.log(x); // or something better
}
await delay(100);
}
If the assignment to val succeeds, we make it to the break; statement and so we leave the loop successfully. Otherwise we jump to the catch block and log the error, wait 100 ms and try again.
It might be better to use a for loop and so set a sensible limit on how many times to retry.
Note that delay is available in an npm package of the same name. It's roughly the same as:
await new Promise(res => setTimeout(res, 100));

Empty AWS S3 bucket of arbitrary cardinality with NodeJS & TypeScript

My removeObjects function has me stummped.The function is suppose to syncronoulsy get a list of objects in an S3 bucket then asyncronously removes the objects. Repeat if the list was truncated, until the there are no more objects to remove. (AWS doesn't provide the total count of objects in the bucket and listObjects pages the results.)
What am I doing wrong / why doesn't my function work? The solution should exploit single thread and async nature of JS. For the bounty I am hoping for an answer specific to the module. The git repo is public if you want to see the entire module.
export function removeObjects(params: IS3NukeRequest): Promise<S3.Types.DeleteObjectsOutput> {
const requests: Array<Promise<S3.Types.DeleteObjectsOutput>> = [];
let isMore;
do {
listObjectsSync(params)
.then((objectList: S3.Types.ListObjectsV2Output) => {
isMore = objectList.ContinuationToken = objectList.IsTruncated ? objectList.NextContinuationToken : null;
requests.push(params.Client.deleteObjects(listObjectsV2Output2deleteObjectsRequest(objectList)).promise());
})
.catch((err: Error) => { Promise.reject(err); });
} while (isMore);
return Promise.all(requests);
}
export async function listObjectsSync(params: IS3NukeRequest): Promise<S3.Types.ListObjectsV2Output> {
try {
return await params.Client.listObjectsV2(s3nukeRequest2listObjectsRequest(params)).promise();
} catch (err) {
return Promise.reject(err);
}
}
Thanks.
The thing is that listObjectsSync function returns a Promise, so you need to treat it as an async function and can't just use a loop with it. What you need to do is to create a chain of promises while your isMore is true, I've done it using a recursive approach (I'm not pro in TS, so please check the code before using it). I also haven't tried the code live, but logically it should work :)
const requests: Array<Promise<S3.Types.DeleteObjectsOutput>> = [];
function recursive(recursiveParams) {
return listObjectsSync(recursiveParams).then((objectList: S3.Types.ListObjectsV2Output) => {
let isMore = objectList.ContinuationToken = objectList.IsTruncated ? objectList.NextContinuationToken : null;
requests.push(params.Client.deleteObjects(listObjectsV2Output2deleteObjectsRequest(objectList)).promise());
if (isMore) {
//do we need to change params here?
return recursive(recursiveParams)
}
//this is not necessary, just to indicate that we get out of the loop
return true;
});
}
return recursive(params).then(() => {
//we will have all requests here
return Promise.all(requests);
});

Nested asynchronous mongoDB calls in node js

I have quite a simple problem, but I can't find an elegant solution to fix this.
In the following code, I have two nested calls to a mongo DB. I use Monk to manage my calls.
The problem is : the for loop (1) loops before the nested insertion can happen. So the next find (2) instruction does not find the last inserted action.
The call order is 1-2-2-2-3-3-3 (for an actionList of size 3). So all my data is inserted.
My objective is to have the call order 1-2-3-2-3-2-3
Do you have any clue of how to manage such a problem, without making a big find on my database and manage my list server-side ? (Get all data, make myself the search, that is quite horrible to do, insert elements I want, then push it all to the db...)
for (var action of actionList)//(1)
{
collectionActions.find(//(2)
{eventid : action.eventid},
function(e,actionsFound)
{
if (actionsFound.length == 0)
{
collectionActions.insert(action, function(err, result)//(3)
{
console.log("insert action : " + action._id);
})
}
}
)
}
The native Promise object has an all method that could be leveraged to help.
Assuming find is a compliant promise, the following code would queue up all of the actions in an array through map and which would return a promise for each action that eventually returns messages to the final then for all.
A couple of notes: your code as it stands swallows all of the errors that might occur (I'm not sure that is want you want); this also assumes that insert returns a promise.
Promise.all([
// Iterate over actionList
actionList.map(function(action) {
// returns a promise with a then already attached
return collectionActions.find({
eventid: action.eventid
}).then(function(e, actionsFound) {
if (actionsFound.length == 0) {
// returns another promise that will resolve up to outer promises
return collectionActions.insert(action, function(err, result) {
// Finally resolve a value for outer promises
return 'insert action : ' + action._id;
});
} else {
// A different value to resolve with if the above promise
// is not required
return 'some other message for ' + action._id;
}
});
})
]).then(function(results) {
// Log out all values resolved by promises
console.log(results);
});
UPDATE: After the clarification of the question it sounds like you just need to chain the promises together rather than run them in parallel.
// Iterate over actionList
actionList.reduce(function(promise, action) {
// Chain promises together
return promise.then(function(results) {
return collectionActions.find({
eventid: action.eventid
}).then(function(e, actionsFound) {
if (actionsFound.length == 0) {
// returns another promise that will resolve up to outer promises
return collectionActions.insert(action, function(err, result) {
// Finally resolve a value for outer promises
return results.push('insert action : ' + action.sourceName);
});
} else {
// A different value to resolve with if the above promise
// is not required
return results.push('some other message for ' + action.sourceName);
}
});
});
}, Promise.resolve([])).then(function(results) {
// Log out all values resolved by promises
console.log(results);
});
I finally got my solution, by using a recursive function.
var currentIndex = 0;
var searchAndInsert = function(actionList)
{
var action = actionList[currentIndex];
if (typeof actionList[currentIndex] != "undefined")
{
collectionActions.find(
{eventid : action.eventid},
function(e,actions)
{
console.log("find ended")
if (actions.length == 0)
{
collectionActions.insert(action, function(err, result)
{
console.log("insert action : " + action.sourceName);
currentIndex++;
if (typeof actionList[currentIndex] != "undefined")
searchAndInsert(actionList);
})
}
else
{
currentIndex++;
if (typeof actionList[currentIndex] != "undefined")
searchAndInsert(actionList);
}
}
)
}
};

different redirects during different promise fails causes Error: cannot call http.ServerResponse.end()

I have a chain of promises that could fail at different points. Depending on where it fails, I might want it to do different things. In some places I might want to render a page, other places redirect. The problem I'm finding is when it runs through all the fail functions and then has errors with http.ServerResponse.end() being called numerous times.
Example:
Parse.Promise.as(1).then(function() {
if (apples) {
return apples.fetch().fail(function() { res.redirect('/somewhere') } );
} else {
return {};
}
}).then(function() {
// doing other stuff
},function() {
res.redirect('/elsewhere');
}).fail(function() {
res.render('error.ejs');
});
Should I be doing this a different way?
( I start with as(1) just to get into the promise chain since there are two different starting cases of apples and !apples that both need to continue to the next part of the chain, but only one of them could start the chain since {} can't. Not sure if that's the best way to do that either. )
Just like exceptions, you can throw different errors and get to different places.
Parse.Promise.as(1).then(function() {
if (apples) {
return apples.fetch();
} else {
throw new Error("No Apples");
}
}).then(function() {
// doing other stuff
}).then(null,function(e) {
if(e.message !== "No Apples"){ // can also subclass Error
res.render("error.ejs");
} else {
res.redirect("/elsewhere");
}
});
Note, like I said in your different question - what you really want is .then(null,function(){ rather than .fail usually. It's a poor name choice for .fail on their side imo.
Because I like them, here is the synchronous analogy:
try {
if(apples){
var a = apples.fetch(); // might throw too
} else{
throw new Error("No Apples");
}
}catch(e){
if(e.message !== "No Apples"){ // can also subclass Error
res.render("error.ejs");
} else {
res.redirect("/elsewhere");
}
}

Resources