I have read some topics related to this but did not help, or I could not make them work. I have:
exports.findBookRating = (book) => {
var average;
Review.findAll({
where: {
bookId: book.id
},
attributes: [[sequelize.fn('avg', sequelize.col('rating')), 'avg_rated']],
raw: true
})
.then(number => {
average = number[0].avg_rated;
console.log(average); //this works
})
.catch(err => {
console.log(err);
});
console.log(average) // this does not work
}
I wanted to return number[0].avg_rated, and that is why I created that average variable but it did not work, can you please help me? Thank you!
"Promise.findA" is async, while the rest is sync. Your last line will execute before your Promise resolved and thus returning "undefined".
your Review.findAll is asynchronous which means right after it's called, it'll proceed to your console.log(average) // this does not work
And since it is asynchronous, by the time the console.log(average) // this does not work is called, average might not have yet been set.
Continue your code inside the .then part or use await if you need the value from an async call.
Example of using await:
const number = await Review.findAll()
const average = number[0].avg_rated
Related
I have a function with multiple forEach loops:
async insertKpbDocument(jsonFile) {
jsonFile.doc.annotations.forEach((annotation) => {
annotation.entities.forEach(async (entity) => {
await this.addVertex(entity);
});
annotation.relations.forEach(async (relation) => {
await this.addRelation(relation);
});
});
return jsonFile;
}
I need to make sure that the async code in the forEach loop calling the this.addVertex function is really done before executing the next one.
But when I log variables, It seems that the this.addRelation function is called before the first loop is really over.
So I tried adding await terms before every loops like so :
await jsonFile.doc.annotations.forEach(async (annotation) => {
await annotation.entities.forEach(async (entity) => {
await this.addVertex(entity);
});
await annotation.relations.forEach(async (relation) => {
await this.addRelation(relation);
});
});
But same behavior.
Maybe it is the log function that have a latency? Any ideas?
As we've discussed, await does not pause a .forEach() loop and does not make the 2nd item of the iteration wait for the first item to be processed. So, if you're really trying to do asynchronous sequencing of items, you can't really accomplish it with a .forEach() loop.
For this type of problem, async/await works really well with a plain for loop because they do pause the execution of the actual for statement to give you sequencing of asynchronous operations which it appears is what you want. Plus, it even works with nested for loops because they are all in the same function scope:
To show you how much simpler this can be using for/of and await, it could be done like this:
async insertKpbDocument(jsonFile) {
for (let annotation of jsonFile.doc.annotations) {
for (let entity of annotation.entities) {
await this.addVertex(entity);
}
for (let relation of annotation.relations) {
await this.addRelation(relation);
}
}
return jsonFile;
}
You get to write synchronous-like code that is actually sequencing asynchronous operations.
If you are really avoiding any for loop, and your real requirement is only that all calls to addVertex() come before any calls to addRelation(), then you can do this where you use .map() instead of .forEach() and you collect an array of promises that you then use Promise.all() to wait on the whole array of promises:
insertKpbDocument(jsonFile) {
return Promise.all(jsonFile.doc.annotations.map(async annotation => {
await Promise.all(annotation.entities.map(entity => this.addVertex(entity)));
await Promise.all(annotation.relations.map(relation => this.addRelation(relation)));
})).then(() => jsonFile);
}
To fully understand how this works, this runs all addVertex() calls in parallel for one annotation, waits for them all to finish, then runs all the addRelation() calls in parallel for one annotation, then waits for them all to finish. It runs all the annotations themselves in parallel. So, this isn't very much actual sequencing except within an annotation, but you accepted an answer that has this same sequencing and said it works so I show a little simpler version of this for completeness.
If you really need to sequence each individual addVertex() call so you don't call the next one until the previous one is done and you're still not going to use a for loop, then you can use the .reduce() promise pattern put into a helper function to manually sequence asynchronous access to an array:
// helper function to sequence asynchronous iteration of an array
// fn returns a promise and is passed an array item as an argument
function sequence(array, fn) {
return array.reduce((p, item) => {
return p.then(() => {
return fn(item);
});
}, Promise.resolve());
}
insertKpbDocument(jsonFile) {
return sequence(jsonFile.doc.annotations, async (annotation) => {
await sequence(annotation.entities, entity => this.addVertex(entity));
await sequence(annotation.relations, relation => this.addRelation(relation));
}).then(() => jsonFile);
}
This will completely sequence everything. It will do this type of order:
addVertex(annotation1)
addRelation(relation1);
addVertex(annotation2)
addRelation(relation2);
....
addVertex(annotationN);
addRelation(relationN);
where it waits for each operation to finish before going onto the next one.
foreach will return void so awaiting it will not do much. You can use map to return all the promises you create now in the forEach, and use Promise.all to await all:
async insertKpbDocument(jsonFile: { doc: { annotations: Array<{ entities: Array<{}>, relations: Array<{}> }> } }) {
await Promise.all(jsonFile.doc.annotations.map(async(annotation) => {
await Promise.all(annotation.entities.map(async (entity) => {
await this.addVertex(entity);
}));
await Promise.all(annotation.relations.map(async (relation) => {
await this.addRelation(relation);
}));
}));
return jsonFile;
}
I understand you can run all the addVertex concurrently. Combining reduce with map splitted into two different set of promises you can do it. My idea:
const first = jsonFile.doc.annotations.reduce((acc, annotation) => {
acc = acc.concat(annotation.entities.map(this.addVertex));
return acc;
}, []);
await Promise.all(first);
const second = jsonFile.doc.annotations.reduce((acc, annotation) => {
acc = acc.concat(annotation.relations.map(this.addRelation));
return acc;
}, []);
await Promise.all(second);
You have more loops, but it does what you need I think
forEach executes the callback against each element in the array and does not wait for anything. Using await is basically sugar for writing promise.then() and nesting everything that follows in the then() callback. But forEach doesn't return a promise, so await arr.forEach() is meaningless. The only reason it isn't a compile error is because the async/await spec says you can await anything, and if it isn't a promise you just get its value... forEach just gives you void.
If you want something to happen in sequence you can await in a for loop:
for (let i = 0; i < jsonFile.doc.annotations.length; i++) {
const annotation = jsonFile.doc.annotations[i];
for (let j = 0; j < annotation.entities.length; j++) {
const entity = annotation.entities[j];
await this.addVertex(entity);
});
// code here executes after all vertix have been added in order
Edit: While typing this a couple other answers and comments happened... you don't want to use a for loop, you can use Promise.all but there's still maybe some confusion, so I'll leave the above explanation in case it helps.
async/await does not within forEach.
A simple solution: Replace .forEach() with for(.. of ..) instead.
Details in this similar question.
If no-iterator linting rule is enabled, you will get a linting warning/error for using for(.. of ..). There are lots of discussion/opinions on this topic.
IMHO, this is a scenario where we can suppress the warning with eslint-disable-next-line or for the method/class.
Example:
const insertKpbDocument = async (jsonFile) => {
// eslint-disable-next-line no-iterator
for (let entity of annotation.entities) {
await this.addVertex(entity)
}
// eslint-disable-next-line no-iterator
for (let relation of annotation.relations) {
await this.addRelation(relation)
}
return jsonFile
}
The code is very readable and works as expected. To get similar functionality with .forEach(), we need some promises/observables acrobatics that i think is a waste of effort.
I have a list of promises and currently I am using promiseAll to resolve them
Here is my code for now:
const pageFutures = myQuery.pages.map(async (pageNumber: number) => {
const urlObject: any = await this._service.getResultURL(searchRecord.details.id, authorization, pageNumber);
if (!urlObject.url) {
// throw error
}
const data = await rp.get({
gzip: true,
headers: {
"Accept-Encoding": "gzip,deflate",
},
json: true,
uri: `${urlObject.url}`,
})
const objects = data.objects.filter((object: any) => object.type === "observed-data" && object.created);
return new Promise((resolve, reject) => {
this._resultsDatastore.bulkInsert(
databaseName,
objects
).then(succ => {
resolve(succ)
}, err => {
reject(err)
})
})
})
const all: any = await Promise.all(pageFutures).catch(e => {
console.log(e)
})
So as you see here I use promise all and it works:
const all: any = await Promise.all(pageFutures).catch(e => {
console.log(e)
})
However I noticed it affects the database performance wise so I decided to resolve every 3 of them at a time.
for that I was thinking of different ways like cwait, async pool or wrting my own iterator
but I get confused on how to do that?
For example when I use cwait:
let promiseQueue = new TaskQueue(Promise,3);
const all=new Promise.map(pageFutures, promiseQueue.wrap(()=>{}));
I do not know what to pass inside the wrap so I pass ()=>{} for now plus I get
Property 'map' does not exist on type 'PromiseConstructor'.
So whatever way I can get it working(my own iterator or any library) I am ok with as far as I have a good understanding of it.
I appreciate if anyone can shed light on that and help me to get out of this confusion?
First some remarks:
Indeed, in your current setup, the database may have to process several bulk inserts concurrently. But that concurrency is not caused by using Promise.all. Even if you had left out Promise.all from your code, it would still have that behaviour. That is because the promises were already created, and so the database requests will be executed any way.
Not related to your issue, but don't use the promise constructor antipattern: there is no need to create a promise with new Promise when you already have a promise in your hands: bulkInsert() returns a promise, so return that one.
As your concern is about the database load, I would limit the work initiated by the pageFutures promises to the non-database aspects: they don't have to wait for eachother's resolution, so that code can stay like it was.
Let those promises resolve with what you currently store in objects: the data you want to have inserted. Then concatenate all those arrays together to one big array, and feed that to one database bulkInsert() call.
Here is how that could look:
const pageFutures = myQuery.pages.map(async (pageNumber: number) => {
const urlObject: any = await this._service.getResultURL(searchRecord.details.id,
authorization, pageNumber);
if (!urlObject.url) { // throw error }
const data = await rp.get({
gzip: true,
headers: { "Accept-Encoding": "gzip,deflate" },
json: true,
uri: `${urlObject.url}`,
});
// Return here, don't access the database yet...
return data.objects.filter((object: any) => object.type === "observed-data"
&& object.created);
});
const all: any = await Promise.all(pageFutures).catch(e => {
console.log(e);
return []; // in case of error, still return an array
}).flat(); // flatten it, so all data chunks are concatenated in one long array
// Don't create a new Promise with `new`, only to wrap an other promise.
// It is an antipattern. Use the promise returned by `bulkInsert`
return this._resultsDatastore.bulkInsert(databaseName, objects);
This uses .flat() which is rather new. In case you have no support for it, look at the alternatives provided on mdn.
First, you asked a question about a failing solution attempt. That is called X/Y problem.
So in fact, as I understand your question, you want to delay some DB request.
You don't want to delay the resolving of a Promise created by a DB request... Like No! Don't try that! The promise wil resolve when the DB will return a result. It's a bad idea to interfere with that process.
I banged my head a while with the library you tried... But I could not do anything to solve your issue with it. So I came with the idea of just looping the data and setting some timeouts.
I made a runnable demo here: Delaying DB request in small batch
Here is the code. Notice that I simulated some data and a DB request. You will have to adapt it. You also will have to adjust the timeout delay. A full second certainly is too long.
// That part is to simulate some data you would like to save.
// Let's make it a random amount for fun.
let howMuch = Math.ceil(Math.random()*20)
// A fake data array...
let someData = []
for(let i=0; i<howMuch; i++){
someData.push("Data #"+i)
}
console.log("Some feak data")
console.log(someData)
console.log("")
// So we have some data that look real. (lol)
// We want to save it by small group
// And that is to simulate your DB request.
let saveToDB = (data, dataIterator) => {
console.log("Requesting DB...")
return new Promise(function(resolve, reject) {
resolve("Request #"+dataIterator+" complete.");
})
}
// Ok, we have everything. Let's proceed!
let batchSize = 3 // The amount of request to do at once.
let delay = 1000 // The delay between each batch.
// Loop through all the data you have.
for(let i=0;i<someData.length;i++){
if(i%batchSize == 0){
console.log("Splitting in batch...")
// Process a batch on one timeout.
let timeout = setTimeout(() => {
// An empty line to clarify the console.
console.log("")
// Grouping the request by the "batchSize" or less if we're almost done.
for(let j=0;j<batchSize;j++){
// If there still is data to process.
if(i+j < someData.length){
// Your real database request goes here.
saveToDB(someData[i+j], i+j).then(result=>{
console.log(result)
// Do something with the result.
// ...
})
} // END if there is still data.
} // END sending requests for that batch.
},delay*i) // Timeout delay.
} // END splitting in batch.
} // END for each data.
I'm accessing google API for converting coordinates into detailed objects using node-geocoder library from npmjs. Everything went well and I'm getting the expected object from geocoder API. The problem started the moment when I thought of using the data outside the promise function. I want to use the values outside the promise/async-await function.
Below is the code I've tried, Pls take a look and help me. TIA...
function goecoderPromiseFunction() {
return new Promise(function (resolve, reject) {
geocoder.reverse({ lat: 45.767, lon: 4.833 })
.then(data => {
cityName = data[0].city;
resolve(cityName);
})
.catch(err => {
console.log(err);
});
});
}
async function app() {
var a = await goecoderPromiseFunction();
return a;
}
var a = app();
console.log("a->", a);
I expect the variable "a" should print the city name "Lyon", but it prints
a-> Promise { < pending > }
The promise returned by the app function is never consumed, that is why it remains in a pending state.
Call then on the app function to get the result :
app().then(a => console.log("a->", a));
You can also use async/await :
(async function() {
var a = await app();
console.log("a->", a);
})();
An asynchronous function actually returns a promise that 'resolves' to the function's return value. You are therefore assigning a promise to the value of a. If you are in the global scope, you obviously cannot use async/await so you need to use either a self-executing async function or you need to run
a.then(data => console.log('a->', data));
to get what you are looking for.
Find out more about async functions here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
It prints because console.log("a->", a); runs while promise haven't returned answer for a variable "a"
Note: you haven't used reject function, if there is an error you wont notice and may be that error was the required answer to be carried by variable "a" that's why it still pending i.e still waiting.
For more idea try to use reject function inside the catch block example reject(err)
instead of console it out as you've done
Imagine you are working with a database of national teams and athletes (like for the Olympic games). When testing your app, you create a new database every time. Here are two models:
const Team = sequelize.define('team', {
name: {type: Sequelize.TEXT, allowNull: false}
})
const Athlete = sequelize.define('athlete', {
name: {type: Sequelize.TEXT, allowNull: false}
})
Team.hasMany(Athlete)
Athlete.belongsTo(Team)
Now, you have a function that creates the teams for you:
function mock_database() {
let team0 = Team.create({
name: 'TeamGB'
}).then(team => {
generate_athletes(team)
// Or maybe I need to return the nested promises?
// return generate_athletes(team)
})
let team1 = Team.create({
name: 'TeamFr'
}).then(team => {
generate_athletes(team)
// Or maybe I need to return the nested promises?
// return generate_athletes(team)
})
return [team0, team1]
}
Now, the generate_athletes creates nested database objects (athletes in the team):
function generate_athletes(team) {
let athlete0 = Athlete.create({
name: 'John Smith'
}).then(athlete => {
team.setAthletes(athlete)
athlete.setTeam(team)
})
let athlete1 = Athlete.create({
name: 'Joanna Smith'
}).then(athlete => {
team.setAthletes(athlete)
athlete.setTeam(team)
})
return [athlete0, athlete1]
}
At this point, I would like to wait until the database has all the data. I can do that with Promise.all(mock_database()).then(() => {/*my application starts here*/}), which would be waiting on the two promises team0 and team1. But does that guarantee that I will be waiting on athlete0 and athlete1 implicitly too? Or do I need to return those promises back up, incorporate them all into a big array of [team0, team1, promises0, promises1] and wait on that?
You would need to return the nested promises. If you don't return the nested promise then your top-level promises will just resolve immediately after creating the teams (since creating the athletes is async and you aren't waiting for it to finish).
Essentially you would want something like this...
function mock_database() {
const team0 = Team.create({ // <-- team0::Promise
// ...
});
const team1 = Team.create({ // <-- team1::Promise
// ...
})
.then(team => {
return generate_athletes(team); // <-- Will only resolve team1 when this resolves
});
return Promise.all([team0, team1]); // Wait for both promises to resolve
}
function generate_athletes(team) {
const athlete1 = Athlete.create({ // athlete1::Promise
// ...
});
const athlete2 = Athlete.create({ // athlete2::Promise
// ...
});
return Promise.all([athlete1, athlete2]); // Wait for both promises to resolve
}
Unless you explicitly return the promises your program does not have access to the then and catch properties of the promise. So always remember, if you want to wait for a promise you will definitely need access to it! So always return it!
Good luck :)
EDIT: You can find a Promise.all explanation here. Essentially though what is happening is that each database action returns a promise. Those promises will resolve if and only if the database action is successful. Since promises are just regular JavaScript objects, they can be saved to a variable.
What I ended up doing was saving a reference to all of the promises I wanted to resolve and then used a builtin promise function to wait for the completion of all of the promises. Since the athletes depend on the team being created, for each team we also need to wait for the list of athletes to be generated.
I marked up the code with some more comments!
had following code which worked:
let sdk = new SDK({ name: "somevalue"});
module.exports= ()=> {
sdk.dothing();
}
I then needed to change the parameter to use data from an async function:
let asyncfunc = require('asyncfunc');
let sdk = new SDK({ name: (()=>{
return asyncfunc()
.then((data) => {
return data.value
})
});
module.exports= ()=> {
sdk.dothing();
}
Following the change, the call to new SDK is failing because the parameter passed is {} as the asyncFunc promise has not yet resolved.
I'm getting back into node after a year and new to promises. what is the proper way to do this?
As you've found, you can't pass in a promise to something that's expecting a string. You need to wait for the asynchronous operation to complete.
This means that your SDK won't be ready right away, so you have two options:
Change your module so it returns a promise for the needed value. Anyone who needs to use your module would need to use the returned promise.
Example:
let pSdk = asyncFunc()
.then(data => new SDK({ name: data.value }));
module.exports = () => pSdk.then(sdk => sdk.dothing());
Store an sdk value that's not populated immediately. Users of your module can obtain the SDK instance directly, but it might not be ready when they need it.
Example:
let sdk;
asyncFunc()
.then(data => sdk = new SDK({ name: data.value }));
module.exports = () => {
if(!sdk) { throw new Error("The SDK is not ready yet!"); }
return sdk.dothing();
};
if any bit of code, in node, is asynchronous then immediately next bit of code will be executed. it doesn't matter if the asynchronous code is wrapped in promise or not.( For codes wrapped in the promise the compiler will return a pending promise to be resoled or rejected and proceed to the next bit of code.) When you are creating an object using new SDK({ }) the name is having reference to a pending promise which is yet to be settled that's why your code is failing to fulfill your requirement. You can do it this way to resolve your problem.
asyncfunc()
.then((data) => {
return new SDK({ name: data.value });
}).then(function(sdk){
//do your work here using sdk
})
One important point to be noted here is you can't return from .then() to assign the value to any variable as you are doing. The value returned from .then() will be accessible from the next chained .then() not by outside global variable.Since you are exporting sdk.dothing() so you need to export it inside the last .then()