I want to force Node to wait for the promise to complete (either by failure or success). So far, I'm not seeing what I wanted. I'm trying to wait for two web services to complete before merging, and especially before the function ends.
I've tried these two approaches below. In both cases, Node refuses to wait.
Approach 1:
function getStrategy() {
// this is a web service that takes a few ms to run,
// but so far I haven't seen any evidence that it bothers.
}
function getConfig() {
// Both strategy and jwt are set to Promises
const strategy = getStrategy();
const jwt = getJwt();
const lodash = require('lodash');
var config = {};
try {
Promise.all([strategy,jwt])
.then(data => { config = lodash.merge(config,data)})
} catch(error) {
console.log(' Print error message');
}
return config;
}
Approach 2:
function getStrategy() {
// this is a web service that takes a few ms to run,
// but so far I haven't seen any evidence that it bothers.
}
async function getConfig() {
// Both strategy and jwt are set to Promises
const strategy = getStrategy();
const jwt = getJwt();
const lodash = require('lodash');
var config = {};
try {
var promiseResult = await Promise.all([strategy, jwt]);
const lodash = require('lodash');
config = lodash.merge(config, strategy[0]);
config = lodash.merge(config, jwt);
} catch (reason) {
console.error('------------------------------------------------------------------------------------');
console.error(reason);
console.error("in getConfig(): Could not fetch strategy or jwt");
console.error('------------------------------------------------------------------------------------');
}
return config;
}
I wish approach 2 worked, but it does not. It will not print any console.log statements after the call to Promise.all. So within that function it does wait. Except that it because I told it to "await", I have to make the function async. That allows Node to say, "oh, it's an async function, I can just go off and do something else and completely ignore the await keyword." It does this by returning to the function calling getConfig().
In the first approach, neither the "then" handler, nor any exceptions are thrown. It just impatiently leaves the function and goes back to the caller.
How do I get the thread that calls the getConfig() function to wait for the result. I mean really wait, not like, partially "await". Or, throw an exception and let me handle that. I'm finding that in Node, as soon as something because asynchronous, I have no idea how to get the await, or the then handler to work.
Updated attempt:
I separated the two service calls to individually control each service. I now have
async function getSynchronousStrategy(isVaultAvailable) {
const secretsVaultReader = require('./src/configuration/secretsVaultReader');
const secretsConfigReader = require('./src/configuration/secretsConfigReader');
const strategy = isVaultAvailable
? secretsVaultReader.getStrategy()
: secretsConfigReader.getStrategy();
await strategy;
console.log('## strategy after await=' + strategy);
return strategy; ''
}
...
const strategy = await getSynchronousStrategy(isVaultAvailable);
console.log('### strategy =' + JSON.stringify(strategy));
...
In the above case, Node sees the await strategy, but then prints
"## strategy after await=[object Promise]"
However, the second await seems to work, and it prints the desired strategy. I'm guessing the promise was eventually settled and it was able to print the result. I don't mind the time, I just want it to wait.
Obviously, it did not wait. It
I would suggest simply using await with both the function calls directly, in this manner it will wait for both the functions to return responses before proceeding further.
Related
Building a node.js CLI application. Users should choose some tasks to run and based on that, tasks should work and then spinners (using ora package) should show success and stop spin.
The issue here is spinner succeed while tasks are still going on. Which means it doesn't wait.
Tried using typical Async/Await as to have an async function and await each function under condition. Didn't work.
Tried using promise.all. Didn't work.
Tried using waterfall. Same.
Here's the code of the task runner, I create an array of functions and pass it to waterfall (Async-waterfall package) or promise.all() method.
const runner = async () => {
let tasks = [];
spinner.start('Running tasks');
if (syncOptions.includes('taskOne')) {
tasks.push(taskOne);
}
if (syncOptions.includes('taskTwo')) {
tasks.push(taskTwo);
}
if (syncOptions.includes('taskThree')) {
tasks.push(taskThree);
}
if (syncOptions.includes('taskFour')) {
tasks.push(taskFour);
}
// Option One
waterfall(tasks, () => {
spinner.succeed('Done');
});
// Option Two
Promise.all(tasks).then(() => {
spinner.succeed('Done');
});
};
Here's an example of one of the functions:
const os = require('os');
const fs = require('fs');
const homedir = os.homedir();
const outputDir = `${homedir}/output`;
const file = `${homedir}/.file`;
const targetFile = `${outputDir}/.file`;
module.exports = async () => {
await fs.writeFileSync(targetFile, fs.readFileSync(file));
};
I tried searching concepts. Talked to the best 5 people I know who can write JS properly. No clue.. What am I doing wrong ?
You don't show us all your code, but the first warning sign is that it doesn't appear you are actually running taskOne(), taskTwo(), etc...
You are pushing what look like functions into an array with code like:
tasks.push(taskFour);
And, then attempting to do:
Promise.all(tasks).then(...)
That won't do anything useful because the tasks themselves are never executed. To use Promise.all(), you need to pass it an array of promises, not an array of functions.
So, you would use:
tasks.push(taskFour());
and then:
Promise.all(tasks).then(...);
And, all this assumes that taskOne(), taskTwo(), etc... are function that return a promise that resolves/rejects when their asynchronous operation is complete.
In addition, you also need to either await Promise.all(...) or return Promise.all() so that the caller will be able to know when they are all done. Since this is the last line of your function, I'd generally just use return Promise.all(...) and this will let the caller get the resolved results from all the tasks (if those are relevant).
Also, this doesn't make much sense:
module.exports = async () => {
await fs.writeFileSync(targetFile, fs.readFileSync(file));
};
You're using two synchronous file operations. They are not asynchronous and do not use promises so there's no reason to put them in an async function or to use await with them. You're mixing two models incorrectly. If you want them to be synchronous, then you can just do this:
module.exports = () => {
fs.writeFileSync(targetFile, fs.readFileSync(file));
};
If you want them to be asynchronous and return a promise, then you can do this:
module.exports = async () => {
return fs.promises.writeFile(targetFile, await fs.promises.readFile(file));
};
Your implementation was attempting to be half and half. Pick one architecture or the other (synchronous or asynchronous) and be consistent in the implementation.
FYI, the fs module now has multiple versions of fs.copyFile() so you could also use that and let it do the copying for you. If this file was large, copyFile() would likely use less memory in doing so.
As for your use of waterfall(), it is probably not necessary here and waterfall uses a very different calling model than Promise.all() so you certainly can't use the same model with Promise.all() as you do with waterfall(). Also, waterfall() runs your functions in sequence (one after the other) and you pass it an array of functions that have their own calling convention.
So, assuming that taskOne, taskTwo, etc... are functions that return a promise that resolve/reject when their asynchronous operations are done, then you would do this:
const runner = () => {
let tasks = [];
spinner.start('Running tasks');
if (syncOptions.includes('taskOne')) {
tasks.push(taskOne());
}
if (syncOptions.includes('taskTwo')) {
tasks.push(taskTwo());
}
if (syncOptions.includes('taskThree')) {
tasks.push(taskThree());
}
if (syncOptions.includes('taskFour')) {
tasks.push(taskFour());
}
return Promise.all(tasks).then(() => {
spinner.succeed('Done');
});
};
This would run the tasks in parallel.
If you want to run the tasks in sequence (one after the other), then you would do this:
const runner = async () => {
spinner.start('Running tasks');
if (syncOptions.includes('taskOne')) {
await taskOne();
}
if (syncOptions.includes('taskTwo')) {
await taskTwo();
}
if (syncOptions.includes('taskThree')) {
await taskThree();
}
if (syncOptions.includes('taskFour')) {
await taskFour();
}
spinner.succeed('Done');
};
I'm still somewhat new to working with Node and def new to working asynchronously and with promises.
I have an application that is hitting a REST endpoint, then calling a chain of functions. The end of this chain is calling hgetall and I need to wait until I get the result and pass it back. I'm testing with Postman and I'm getting {} back instead of the id. I can console.log the id, so I know that this is because some of the code isn't waiting for the result of hgetall before continuing.
I'm using await to wait for the result of hgetall, but that's only working for the end of the chain. do I need to do this for the entire chain of functions, or is there a way to have everything wait for the result before continuing on? Here's the last bit of the logic chain:
Note: I've removed some of the logic from the below functions and renamed a few things to make it a bit easier to see the flow and whats going on with this particular issue. So, some of it may look a bit weird.
For this example, it will call GetProfileById().
FindProfile(info) {
var profile;
var profileId = this.GenerateProfileIdkey(info); // Yes, this will always give me the correct key
profile = this.GetProfileById(profileId);
return profile;
}
This checks with the Redis exists, to verify if the key exists, then tries to get the id with that key. I am now aware that the Key() returns true instead of what Redis actually returns, but I'll fix that once I get this current issue resolved.
GetProfileById(profileId) {
if ((this.datastore.Key(profileId) === true) && (profileId != null)) {
logger.info('GetProfileById ==> Profile found. Returning the profile');
return this.datastore.GetId(profileId);
} else {
logger.info(`GetProfileById ==> No profile found with key ${profileId}`)
return false;
}
}
GetId() then calls the data_store to get the id. This is also where I started to use await and async to try and wait for the result to come through before proceeding. This part does wait for the result, but the functions prior to this don't seem to wait for this one to return anything. Also curious why it only returns the key and not the value, but when I print out the result in hgetall I get the key and value?
async GetId(key) {
var result = await this.store.RedisGetId(key);
console.log('PDS ==> Here is the GetId result');
console.log(result); // returns [ 'id' ]
return result;
}
and finally, we have the hgetall call. Again, new to promises and async, so this may not be the best way of handling this or right at all, but it is getting the result and waiting for the result before it returns anything
async RedisGetId(key) {
var returnVal;
var values;
return new Promise((resolve, reject) => {
client.hgetall(key, (err, object) => {
if (err) {
reject(err);
} else {
resolve(Object.keys(object));
console.log(object); // returns {id: 'xxxxxxxxxxxxxx'}
return object;
}
});
});
}
Am I going to need to async every single function that could potentially end up making a Redis call, or is there a way to make the app wait for the Redis call to return something, then continue on?
Short answer is "Yes". In general, if a call makes an asynchronous request and you need to await the answer, you will need to do something to wait for it.
Sometimes, you can get smart and issue multiple calls at once and await all of them in parallel using Promise.all.
However, it looks like in your case your workflow is synchronous, so you will need to await each step individually. This can get ugly, so for redis I typically use something like promisify and make it easier to use native promises with redis. There is even an example on how to do this in the redis docs:
const {promisify} = require('util');
const getAsync = promisify(client.get).bind(client);
...
const fooVal = await getAsync('foo');
Makes your code much nicer.
it's the first time for me using async/await. I've got problems to use it in the context of a database request inside a dialogflow intent. How can I fix my code?
What happens?
When I try to run use my backend - this is what I get: "Webhook call failed. Error: Request timeout."
What do I suspect?
My helper function getTextResponse() waits for a return value of airtable, but never get's one.
What do I want to do?
"GetDatabaseField-Intent" gets triggered
Inside it sends a request to my airtable database via getTextResponse()
Because I use"await" the function will wait for the result before continuing
getTextResponse() will return the "returnData"; so the var result will be filled with "returnData"
getTextResponse() has finished; so the response will be created with it's return value
'use strict';
const {
dialogflow
} = require('actions-on-google');
const functions = require('firebase-functions');
const app = dialogflow({debug: true});
const Airtable = require('airtable');
const base = new Airtable({apiKey: 'MyKey'}).base('MyBaseID');
///////////////////////////////
/// Helper function - reading Airtable fields.
const getTextResponse = (mySheet, myRecord) => {
return new Promise((resolve, reject) => {
// Function for airtable
base(mySheet).find(myRecord, (err, returnData) => {
if (err) {
console.error(err);
return;
}
return returnData;
});
}
)};
// Handle the Dialogflow intent.
app.intent('GetDatabaseField-Intent', async (conv) => {
const sheetTrans = "NameOfSheet";
const recordFirst = "ID_OF_RECORD";
var result = await getTextResponse(sheetTrans, recordFirst, (callback) => {
// parse the record => here in the callback
myResponse = callback.fields.en;
});
conv.ask(myResponse);
});
// Set the DialogflowApp object to handle the HTTPS POST request.
exports.dialogflowFirebaseFulfillment = functions.https.onRequest(app);
As #Kolban pointed out, you are not accepting or rejecting the Promise you create in getTextResponse().
It also looks like the var result = await getTextResponse(...) call is incorrect. You have defined getTextResponse() to accept two parameters, but you are passing it three (the first two, plus an anonymous arrow function). But this extra function is never used/referenced.
I would generally avoid mixing explicit promises with async/await and definitely avoid mixing async/await with passing callbacks.
I don't know the details of the API you are using, but if the API already supports promises, then you should be able to do something like this:
const getTextResponse = async (mySheet, myRecord) => {
try {
return await base(mySheet).find(myRecord)
}
catch(err) {
console.error(err);
return;
}
)};
...
app.intent('GetDatabaseField-Intent', async (conv) => {
const sheetTrans = "NameOfSheet";
const recordFirst = "ID_OF_RECORD";
var result = await getTextResponse(sheetTrans, recordFirst)
myResponse = result.fields.en;
conv.ask(myResponse);
});
...
Almost all promised based libraries or APIs can be used with async/await, as they simply use Promises under the hood. Everything after the await becomes a callback that is called when the awaitted method resolves successfully. Any unsuccessful resolution throws a PromiseRejected error, which you handle by use of a try/catch block.
Looking at the code, it appears that you may have a misunderstanding of JavaScript Promises. When you create a Promise, you are passed two functions called resolve and reject. Within the body of your promise code (i.e. the code that will complete sometime in the future). You must invoke either resolve(returnData) or reject(returnData). If you don't invoke either, your Promise will never be fulfilled. Looking at your logic, you appear to be performing simple returns without invoking resolve or reject.
Let me ask you to Google again on JavaScript Promises and study them again with respect to the previous comments just made and see if that clears up the puzzle.
i tried to add a trigger function on my cloud functions.
When a document changed i want to perform some work. I have already the code for the work to do in another file (i separate my work into files, so it's get easier for me), so i have a function that performs work and when it is finished the callback is called.
This is what my index.js look like:
// requires...
// Express, i also use express for simulating an API
const app = express()
// So there is my Listener
const listenerForChange = functions.firestore
.document('myPath/documentName')
.onWrite((change, context) => {
const document = change.after.exists ? change.after.data() : null;
if (document != null) {
const oldDocument = change.before.data();
const newDocument = change.after.data()
// Here i call my function that is in the worker-listeners.js
listenerWorker.performStuff(newDocument, function() {
return 0
})
}
else {
console.error("Listener for classic was triggered but doc does not exist")
return 0
}
});
const api = functions.https.onRequest(app)
module.exports = {
api,
listenerForChange
}
...
There is my listenerWorker.js :
module.exports = {
performStuff: function(data, complete) {
performStuff(data, complete)
}
};
function performStuff(data, complete) {
// do stuff here ...
complete()
}
Problem is that i always an error in the Firebase console saying : Function returned undefined, expected Promise or value
Even if i do not do anything inside my worker and calling the callback as soon as i can i get the error.
So i understand the functions needs a response, promise or value. But it's like i'm not in the scope anymore but i do not want to return before the work is finished !
Any help ? thank you guys
All asynchronous work that you perform should be represented by a promise. If you have a utility object that does async work, it should return a promise so that the caller can decide what to do next with it. Right now you're just using callbacks, and that's going to make things much more difficult than necessary (because you'll have to "promisify" those callbacks).
Just use promises everywhere and it'll be much more clear how your function should work.
Setting up a node.js app to retrieve order(s) information from shopify. I'm trying to store that data and run some logic, but I can't seem to get a hold of it.
THIS WORKS:
shopify.order.list()
.then(function(orders){
console.log(orders);
});
THIS DOES NOT WORK:
var orders_json;
shopify.order.list()
.then(function(orders){
orders_json=orders;
//console.log(orders);
});
console.log(orders_json); //returns undefined
Let me introduce you to the world of async/await. As long as you declare your function as async and the function you are "awaiting" returns a promise, you can handle this in a more synchronous way. Have a look at the docs linked above. Notice how I called the async function after it was declared. You can't call await outside the scope of an async function.
async function fetchOrders() {
try {
const orders_json = await shopify.order.list();
// do stuff with orders_json
return orders_json;
} catch(err) {
// handle err
}
}
const orders = fetchOrders();