I am trying to use promises and async functions to send chunks of an array to an api call that inserts them into a DB. I am trying to get the top function to chunk the array and await for the backend to finsih then it will move on to the next chunk. It errors out after the first iteration. Any ideas??
async chunkArray(arr) {
let len = arr.length
let update_arr
let i = 1
for(i; i<=len; i++) {
if((i%125) === 0) {
update_arr = arr.slice(i-125,i)
await this.updateChuckArray(update_arr)
} else if(i === len) {
let div = (Math.floor(len/125) * 125)
update_arr = arr.slice(div, len)
await this.updateChuckArray(update_arr)
}
}
},
updateChuckArray(update) {
return new Promise(resolve => {
this.$http.put(`/route`, update).then(res => {
res.data.error ? this.$root.updateError(res.data.error) : this.$root.updateSuccess(res.data.message)
}).catch(error => {
this.$root.updateError(res.data.error)
})
})
}
First off your updateChuckArray() never resolves the promise it returns (you never call resolve()).
Instead of manually wrapping a new promise around your function call (that is a promise anti-pattern), you can just return the promise you already have and write it like this:
updateChuckArray(update) {
return this.$http.put(`/route`, update).then(res => {
res.data.error ? this.$root.updateError(res.data.error) : this.$root.updateSuccess(res.data.message);
}).catch(error => {
this.$root.updateError(error);
})
}
FYI, it's unclear what your error handling strategy is. The way you wrote the code (which is followed above), you catch an error from this.$http.put() and handle it and let the loop continue. If that's what you want, this will work. If you want the for loop to abort on error, then you need to rethrow the error in the .catch() handler so the error gets back to the await.
Also, not that in your .catch() handler, you were doing this:
this.$root.updateError(res.data.error)
but there is no res defined there. The error is in error. You would need to use that in order to report the error. I'm not sure what the structure of the error object is here or what exactly you pass to $.root.updateError(), but it must be something that comes from the error object, not an object named res.
Related
I'm writing a Windows Node.js server app (using ES6 btw).
The first thing I want to do - in the top-level code - is sit in a while loop, calling an async function which searches for a particular registry key/value. This function is 'proven' - it returns the value data if found, or else throws:
async GetRegValue(): Promise<string> { ... }
I need to sit in a while loop until the registry item exists, and then grab the value data. (With a delay between retries).
I think I know how to wait for an async call to complete (one way or the other) before progressing with the rest of the start-up, but I can't figure out how to sit in a loop waiting for it to succeed.
Any advice please on how to achieve this?
(I'm fairly new to typescript, and still struggling to get my head round all async/await scenarios!)
Thanks
EDIT
Thanks guys. I know I was 'vague' about my code - I didn't want to put my real/psuedo code attempts, since they have all probably overlooked the points you can hopefully help me understand.
So I just kept it as a textual description... I'll try though:
async GetRegValue(): Promise<string> {
const val: RegistryItem = await this.GetKeyValue(this.KEY_SW, this.VAL_CONN);
return val.value
}
private async GetKeyValue(key: string, name: string): Promise<RegistryItem> {
return await new Promise((resolve, reject) => {
new this.Registry({
hive: this.Hive, key
}).get(name, (err, items) => {
if (err) {
reject(new Error('Registry get failed'));
}
else {
resolve( items );
}
});
})
.catch(err => { throw err });
}
So I want to do something like:
let keyObtained = false
let val
while (keyObtained == false)
{
// Call GetRegValue until val returned, in which case break from loop
// If exception then pause (e.g. ~100ms), then loop again
}
}
// Don't execute here till while loop has exited
// Then use 'val' for the subsequent statements
As I say, GetRegValue() works fine in other places I use it, but here I'm trying to pause further execution (and retry) until it does come back with a value
You can probably just use recursion. Here is an example on how you can keep calling the GetRegValue function until is resolves using the retryReg function below.
If the catch case is hit, it will just call GetRegValue over and over until it resolves successfully.
you should add a counter in the catch() where if you tried x amount of times you give up.
Keep in mind I mocked the whole GetRegValue function, but given what you stated this would still work for you.
let test = 0;
function GetRegValue() {
return new Promise((resolve, reject) => {
setTimeout(function() {
test++;
if (test === 4) {
return resolve({
reg: "reg value"
});
}
reject({
msg: "not ready"
});
}, 1000);
});
}
function retryReg() {
GetRegValue()
.then(registryObj => {
console.log(`got registry obj: ${JSON.stringify(registryObj)}`)
})
.catch(fail => {
console.log(`registry object is not ready: ${JSON.stringify(fail)}`);
retryReg();
});
}
retryReg();
I don't see why you need this line:
.catch(err => { throw err });
The loop condition of while isn't much use in this case, as you don't really need a state variable or expression to determine if the loop should continue:
let val;
while (true)
{
try {
val = await GetRegValue(/* args */);
break;
} catch (x) {
console.log(x); // or something better
}
await delay(100);
}
If the assignment to val succeeds, we make it to the break; statement and so we leave the loop successfully. Otherwise we jump to the catch block and log the error, wait 100 ms and try again.
It might be better to use a for loop and so set a sensible limit on how many times to retry.
Note that delay is available in an npm package of the same name. It's roughly the same as:
await new Promise(res => setTimeout(res, 100));
I've got a function.
function async extractTars (tarList) {
try {
for (let i = 0; i < tarList.length; i ++) {
// I need this loop to be sync but it isn't hitting the next iteration
return new Promise((resolve, reject) => {
fs.createReadStream(`${PATHS.TAR}/${tarList[i]}`)
.pipe(tar.extract(PATHS.GZ))
.on('error', err => reject(err))
.on('finish', () => resolve())
})
}
// What is the correct way to resolve this async fn? Should i just return or will it resolve anyway after the loop?
} catch (e) {
// handle error
}
}
For some reason it never hits the next iteration of the loop. What am i missing here? How can i make these type of loops synchronous. In this particular project i've got many many loops that need to complete before the next. I've tried several methods that i've see here on SO but i'm obviously missing something.
Any help would be much appreciated!
You're using return, that's why you're not hitting the next iteration, use await instead to wait until the promise is done before going to the next iteration.
return inside a for will stop the loop & end the function returning the new Promise
function async extractTars (tarList) {
try {
for (let i = 0; i < tarList.length; i ++) {
// I need this loop to be sync but it isn't hitting the next iteration
await new Promise((resolve, reject) => {
fs.createReadStream(`${PATHS.TAR}/${tarList[i]}`)
.pipe(tar.extract(PATHS.GZ))
.on('error', err => reject(err))
.on('finish', () => resolve())
})
}
// What is the correct way to resolve this async fn? Should i just return or will it resolve anyway after the loop?
} catch (e) {
// handle error
}
}
And just to clear things up:
I need this loop to be sync but it isn't hitting the next iteration
That loop won't be sync, it will look like it's sync, when using async/await but you can't make asynchronous code be synchronous. And createReadStream is asnychronous.
And regarding:
What is the correct way to resolve this async fn? Should i just
return or will it resolve anyway after the loop?
When all iterations are done or an error is thrown (since you're catching it and not rejecting), meaning that you read all the files in tarList, the function will be resolved. Since you are not returning anything, undefined will be the resolved value.
From MDN:
When a return statement is used in a function body, the execution of the function is stopped. If specified, a given value is returned to the function caller. For example, the following function returns the square of its argument, x, where x is a number.
I'm writing a script that is intended to load some stuff from .txt files and then perform multiple ( in a loop) requests to a website with node.js` browser emulator nightmare.
I have no problem with reading from the txt files and so no, but managing to make it run sync and without exceptions.
function visitPage(url, code) {
new Promise((resolve, reject) => {
Nightmare
.goto(url)
.click('.vote')
.insert('input[name=username]', 'testadmin')
.insert('.test-code-verify', code)
.click('.button.vote.submit')
.wait('.tag.vote.disabled,.validation-error')
.evaluate(() => document.querySelector('.validation -error').innerHTML)
.end()
.then(text => {
return text;
})
});
}
async function myBackEndLogic() {
try {
var br = 0, user, proxy, current, agent;
while(br < loops){
current = Math.floor(Math.random() * (maxLoops-br-1));
/*...getting user and so on..*/
const response = await visitPage('https://example.com/admin/login',"code")
br++;
}
} catch (error) {
console.error('ERROR:');
console.error(error);
}
}
myBackEndLogic();
The error that occurs is:
UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'webContents' of undefined
So the questions are a few:
1) How to fix the exception
2) How to make it actually work sync and emulate everytime the address ( as in a previous attempt, which I didn't save, I fixed the exception, but the browser wasn't actually openning and it was basically skipped
3) (Not so important) Is it possible to select a few objects with
.wait('.class1,.class2,.validation-error')
and save each value in different variables or just get the text from the first that occured? ( if no any of these has occurred, then return 0 for example )
I see a few issues with the code above.
In the visitPage function, you are returning a Promise. That's fine, except you don't have to create the wrapping promise! It looks like nightmare returns a promise for you. Today, you're dropping an errors that promise returns by wrapping it. Instead - just use an async function!
async function visitPage(url, code) {
return Nightmare
.goto(url)
.click('.vote')
.insert('input[name=username]', 'testadmin')
.insert('.test-code-verify', code)
.click('.button.vote.submit')
.wait('.tag.vote.disabled,.validation-error')
.evaluate(() => document.querySelector('.validation -error').innerHTML)
.end();
}
You probably don't want to wrap the content of this method in a 'try/catch'. Just let the promises flow :)
async function myBackEndLogic() {
var br = 0, user, proxy, current, agent;
while(br < loops){
current = Math.floor(Math.random() * (maxLoops-br-1));
const response = await visitPage('https://example.com/admin/login',"code")
br++;
}
}
When you run your method - make sure to include a catch! Or a then! Otherwise, your app may exit early.
myBackEndLogic()
.then(() => console.log('donesies!'))
.catch(console.error);
I'm not sure if any of this will help with your specific issue, but hopefully it gets you on the right path :)
I understand using the Q library it's easy to wait on a number of promises to complete, and then work with the list of values corresponding to those promise results:
Q.all([
promise1,
promise2,
.
.
.
promiseN,
]).then(results) {
// results is a list of all the values from 1 to n
});
What happens, however, if I am only interested in the single, fastest-to-complete result? To give a use case: Say I am interested in examining a big list of files, and I'm content as soon as I find ANY file which contains the word "zimbabwe".
I can do it like this:
Q.all(fileNames.map(function(fileName) {
return readFilePromise(fileName).then(function(fileContents) {
return fileContents.contains('zimbabwe') ? fileContents : null;
}));
})).then(function(results) {
var zimbabweFile = results.filter(function(r) { return r !== null; })[0];
});
But I need to finish processing every file even if I've already found "zimbabwe". If I have a 2kb file containing "zimbabwe", and a 30tb file not containing "zimbabwe" (and suppose I'm reading files asynchronously) - that's dumb!
What I want to be able to do is get a value the moment any promise is satisfied:
Q.any(fileNames.map(function(fileName) {
return readFilePromise(fileName).then(function(fileContents) {
if (fileContents.contains('zimbabwe')) return fileContents;
/*
Indicate failure
-Return "null" or "undefined"?
-Throw error?
*/
}));
})).then(function(result) {
// Only one result!
var zimbabweFile = result;
}).fail(function() { /* no "zimbabwe" found */ });
With this approach I won't be waiting on my 30tb file if "zimbabwe" is discovered in my 2kb file early on.
But there is no such thing as Q.any!
My question: How do I get this behaviour?
Important note: This should return without errors even if an error occurs in one of the inner promises.
Note: I know I could hack Q.all by throwing an error when I find the 1st valid value, but I'd prefer to avoid this.
Note: I know that Q.any-like behavior could be incorrect, or inappropriate in many cases. Please trust that I have a valid use-case!
You are mixing two separate issues: racing, and cancelling.
Racing is easy, either using Promise.race, or the equivalent in your favorite promise library. If you prefer, you could write it yourself in about two lines:
function race(promises) {
return new Promise((resolve, reject) =>
promises.forEach(promise => promise.then(resolve, reject)));
}
That will reject if any promise rejects. If instead you want to skip rejects, and only reject if all promises reject, then
function race(promises) {
let rejected = 0;
return new Promise((resolve, reject) =>
promises.forEach(promise => promise.then(resolve,
() => { if (++rejected === promises.length) reject(); }
);
}
Or, you could use the promise inversion trick with Promise.all, which I won't go into here.
Your real problem is different--you apparently want to "cancel" the other promises when some other one resolves. For that, you will need additional, specialized machinery. The object that represents each segment of processing will need some way to ask it to terminate. Here's some pseudo-code:
class Processor {
promise() { ... }
terminate() { ... }
}
Now you can write your version of race as
function race(processors) {
let rejected = 0;
return new Promise((resolve, reject) =>
processors.forEach(processor => processor.promise().then(
() => {
resolve();
processors.forEach(processor => processor.terminate());
},
() => { if (++rejected === processors.length) reject(); }
);
);
}
There are various proposals to handle promise cancellation which might make this easier when they are implemented in a few years.
I have a function,
asdf() {
var a = fooController.getOrCreateFooByBar(param);
console.log("tryna do thing");
console.log(a); //undefined
if (!a.property) {
//blah
}
that dies. getOrCreateFooByBar does a
Model.find({phoneNumber : number}).exec()
.then(function(users) {})
and finds or creates the model, returning it at the end:
.then(function(foo) { return foo}
How can I use the result of this in asdf()? I feel like this is a fairly easy question but I am getting stuck. If I try to do a.exec() or a.then() I get 'a cannot read property of undefined' error.
The main idea about Promises (as opposed to passed callbacks) is that they are actual objects you can pass around and return.
fooController.getOrCreateFooByBar would need to return the Promise it gets from Model.find() (after all of the processing done on it). Then, you'll be able to access it in a in your asdf function.
In turn, asdf() should also return a Promise, which would make asdf() thenable as well. As long as you keep returning Promises from asynchronous functions, you can keep chaining them.
// mock, you should use the real one
const Model = { find() { return Promise.resolve('foo'); } };
function badExample() {
Model.find().then(value => doStuff(value));
}
function goodExample() {
return Model.find().then(value => doStuff(value));
}
function asdf() {
var a = badExample();
var b = goodExample();
// a.then(whatever); // error, a is undefined because badExample doesn't return anything
return b.then(whatever); // works, and asdf can be chained because it returns a promise!
}
asdf().then(valueAfterWhatever => doStuff(valueAfterWhatever));