Note to would-be closers or markers-as-duplicate : this question is not answered in How do I convert an existing callback API to promises?, as all answers there treat of calls in isolation and do not explain how to deal with successive, dependent calls.
This question is basically the same as in combining results from several async/await calls, but because of my slightly different context I fail to see how the solution therein can be adapted/mimicked.
I have two successive calls to a database, using an old API which only knows about callbacks. The second call needs objects/values returned by the first.
I have a working callback-version of the code, as follows :
connection.query(sql1,function(error1,results1,fields1) {
if(error1) {
console.log("Error during first query",error1);
}
let sql2=computeUsingFirstResult(result1);
connection.query(sql2,function(error2,results2,fields2) {
if(error2) {
console.log("Error during second query",error2);
}
doSomething(connection,results1,results2);
})
});
Here is my unsuccessful attempt to do it in async/await-style :
const util = require('util');
async function firstQuery(connection) {
return util.promisify(connection.query).call(sql1).catch(
error1 => console.log("Error during first query : ", error1)
);;
}
async function secondQuery(connection, result1) {
let sql2 = computeUsingFirstResult(result1);
return util.promisify(connection.query).call(sql2).catch(
error2 => console.log("Error during second query : ", error2)
);
}
let finalResult = {};
async function main() {
const results1 = await firstQuery(connection);
const results2 = await secondQuery(connection, results1);
doSomething(connection, results1, results2);
finalResult = [results1,results2];
console.log("Here is the finalResult : ", finalResult);
}
main().catch(
err => console.log("Something went wrong towards the end", err)
);
My async/await version fails as all the intermediate results are undefined. Yet, as far as I can see it should be equivalent to the non-async version above.
What is the correct way to do this ?
There are 2 approaches I am hoping you can try and see if either of them help:
(1) Bind the util.promisify to connection for both firstQuery and secondQuery something like so:
async function firstQuery(connection) {
return util.promisify(connection.query).bind(connection,sql1)().catch(
error1 => console.log("Error during first query : ", error1)
);;
}
// bind makes sure that internal bindings of connection object are in tact
If the above approach doesn't yield any result, try the next approach:
(2) Try using Promises-Wrapper instead of util.promisify for both firstQuery and secondQuery like so:
function firstQuery(connection,sql1){
return new Promise((resolve,reject)=>{
connection.query(sql1,function(error1,results1,fields1){
return error1 ? reject(error1):resolve(results1);
})
});
}
And now call them as you were in your code using Async/Await in the main function.
Few Potential bugs/typos I noticed in the code:
(1) firstQuery doesn't seem to be passed sql1 as one of its arguments, maybe this is intentional if sql1 exists in global scope.
(2) If you attach a catch block to both your queries, they will have an impact on your 2 calls (see comments in the code below):
async function main() {
const results1 = await firstQuery(connection); //<-- if this call fails, results1 will be undefined and the call will go to next step
const results2 = await secondQuery(connection, results1); // same thing here
doSomething(connection, results1, results2);
finalResult = [results1,results2];
console.log("Here is the finalResult : ", finalResult);
}
A possible solution is to remove the catch blocks from individual functions and just return the respective promises.
Then you can deal with them via a single try/catch block like this:
async function main() {
try{
const results1 = await firstQuery(connection);
const results2 = await secondQuery(connection, results1);
doSomething(connection, results1, results2);
finalResult = [results1,results2];
console.log("Here is the finalResult : ", finalResult);
}catch(err){
console.log("Error",err);
}
}
This will ensure that as soon as the first call fails, function goes straight to catch block without executing any other lines.
Related
I'm having some difficulty with arangodb.beginTransaction(). In my tests, creating a transaction, calling a function via trx.run() API and then aborting the trx does not roll back the data changes on the database. I'm unclear what is happening and the documentation is extremely sparse. There is this documentation which is quite ominous, but also very vague:
If the given function contains asynchronous logic, only the synchronous part of the function will be run in the transaction. E.g. when using async/await only the code up to the first await will run in the transaction.
What about nested async/await inside the async function being called? For example, if I have this:
async runQuery(query) {
try {
const cursor = await this.arangodb.query(query)
return cursor.all()
} catch (error) {
logger.error('ArangoDB query failed', { stack: error })
throw error
}
}
async touchDocument(id, revision) {
const type = await this.getObjectTypeFromId(id)
const collection = await this.getCollection(type, false, true)
const idCollection = await this.getCollection(ID_COLLECTION, false, true)
const touchAql = aql`
LET permanentDocId = DOCUMENT(${idCollection}, ${id}).docId
LET permanentDoc = MERGE( DOCUMENT(permanentDocId), { _rev : ${revision} })
UPDATE permanentDoc WITH permanentDoc in ${collection} OPTIONS { ignoreRevs: false }
RETURN NEW
`
return this.runQuery(touchAql)
}
trx.run(() => this.touchDocument(parentId, parentRevision))
Will this.arangodb.query() execute inside the transaction or not?
I'm the author of arangojs. For posterity I'd like to clarify that this answer is about arangojs 6, which is the current release of arangojs at the time of this writing and the version that first added support for streaming transactions. The API may change in future versions although there are currently no plans to do so.
Because of how transactions work (as of arangojs 6), you need to pay special attention to the caveat mentioned in the documentation for the run method:
If the given function contains asynchronous logic, only the synchronous part of the function will be run in the transaction. E.g. when using async/await only the code up to the first await will run in the transaction. Pay attention to the examples below.
In other words, async functions will likely not behave correctly if you just wrap them in trx.run. I'd recommend passing the transaction object to each function and wrapping the method calls in those functions in trx.run.
For example:
async runQuery(query, trx) {
try {
const cursor = await trx.run(() => this.arangodb.query(query))
return trx.run(() => cursor.all())
} catch (error) {
logger.error('ArangoDB query failed', { stack: error })
throw error
}
}
async touchDocument(id, revision, trx) {
const type = await this.getObjectTypeFromId(id, trx)
const collection = await this.getCollection(type, false, true, trx)
const idCollection = await this.getCollection(ID_COLLECTION, false, true, trx)
const touchAql = aql`
LET permanentDocId = DOCUMENT(${idCollection}, ${id}).docId
LET permanentDoc = MERGE( DOCUMENT(permanentDocId), { _rev : ${revision} })
UPDATE permanentDoc WITH permanentDoc in ${collection} OPTIONS { ignoreRevs: false }
RETURN NEW
`
return this.runQuery(touchAql, trx)
}
this.touchDocument(parentId, parentRevision, trx)
The reason behind this is that trx.run sets the entire driver into "transaction mode", executes the given function and then disables "transaction mode" after executing it so unrelated code doesn't accidentally run in the transaction.
The drawback of this approach is that if the function is async and contains multiple await statements, only the code leading up to and including the first await will be run in the transaction. In your code that means "transaction mode" is disabled after this.getObjectTypeFromId(id) returns. If that method itself contains multiple await expressions, again only the first one will be part of the transaction (and so forth).
I hope this clears up some of the confusion.
I am adding user validation an data modification page on a node.js application.
In a synchronous universe, in a single function I would:
Lookup the original record in the database
Lookup the user in LDAP to see if they are the owner or admin
Do the logic and write the record.
In an asynchronous universe that won't work. To solve it I've built a series of hand-off functions:
router.post('/writeRecord', jsonParser, function(req, res) {
post = req.post;
var smdb = new AWS.DynamoDB.DocumentClient();
var params = { ... }
smdb.query(params, function(err,data){
if( err == null ) writeRecordStep2(post,data);
}
});
function writeRecord2( ru, post, data ){
var conn = new LDAP();
conn.search(
'ou=groups,o=amazon.com',
{ ... },
function(err,resp){
if( err == null ){
writeRecordStep3( ru, post, data, ldap1 )
}
}
}
function writeRecord3( ru, post, data ){
var conn = new LDAP();
conn.search(
'ou=groups,o=amazon.com',
{ ... },
function(err,resp){
if( err == null ){
writeRecordStep4( ru, post, data, ldap1, ldap2 )
}
}
}
function writeRecordStep4( ru, post, data, ldap1, ldap2 ){
// Do stuff with collected data
}
Additionally, because the LDAP and Dynamo logic are in their own source documents, these functions are scattered tragically around the code.
This strikes me as inefficient, as well as inelegant. I'm eager to find a more natural asynchronous pattern to achieve the same result.
Any promise library should sort your issue out. My preferred choice is bluebird. In summary they help you in performing blocking operations.
If you haven't heard about bluebird then just use it. It converts all function of a module and return promise which is then-able. Simply put, it promisifies all functions.
Here is the mechanism:
Module1.someFunction() \\do your job and finally pass the return object to next call
.then() \\Use that object which is return from the first call, do your job and return the updated value
.then() \\same goes on
.catch() \\do your job when any error occurs.
Hope you understand. Here is an example:
var readFile = Promise.promisify(require("fs").readFile);
readFile("myfile.js",
"utf8").then(function(contents) {
return eval(contents);
}).then(function(result) {
console.log("The result of evaluating
myfile.js", result);
}).catch(SyntaxError, function(e) {
console.log("File had syntax error", e);
//Catch any other error
}).catch(function(e) {
console.log("Error reading file", e);
});
I could not tell from your pseudo-code exactly which async operations depend upon results from with other ones and knowing that is key to the most efficient way to code a series of asynchronous operations. If two operations do not depend upon one another, they can run in parallel which generally gets to an end result faster. I also can't tell exactly what data needs to be passed on to later parts of the async requests (too much pseudo-code and not enough real code to show us what you're really attempting to do).
So, without that level of detail, I'll show you two ways to approach this. The first runs each operation sequentially. Run the first async operation, when it's done, run the next one and accumulates all the results into an object that is passed along to the next link in the chain. This is general purpose since all async operations have access to all the prior results.
This makes use of promises built into the AWS.DynamboDB interface and makes our own promise for conn.search() (though if I knew more about that interface, it may already have a promise interface).
Here's the sequential version:
// promisify the search method
const util = require('util');
LDAP.prototype.searchAsync = util.promisify(LDAP.prototype.search);
// utility function that does a search and adds the result to the object passed in
// returns a promise that resolves to the object
function ldapSearch(data, key) {
var conn = new LDAP();
return conn.searchAsync('ou=groups,o=amazon.com', { ... }).then(results => {
// put our results onto the passed in object
data[key] = results;
// resolve with the original object (so we can collect data here in a promise chain)
return data;
});
}
router.post('/writeRecord', jsonParser, function(req, res) {
let post = req.post;
let smdb = new AWS.DynamoDB.DocumentClient();
let params = { ... }
// The latest AWS interface gets a promise with the .promise() method
smdb.query(params).promise().then(dbresult => {
return ldapSearch({post, dbresult}, "ldap1");
}).then(result => {
// result.dbresult
// result.ldap1
return ldapSearch(result, "ldap2")
}).then(result => {
// result.dbresult
// result.ldap1
// result.ldap2
// doSomething with all the collected data here
}).catch(err => {
console.log(err);
res.status(500).send("Internal Error");
});
});
And, here's a parallel version that runs all three async operations at once and then waits for all three of the to be done and then has all the results at once:
// if the three async operations you show can be done in parallel
// first promisify things
const util = require('util');
LDAP.prototype.searchAsync = util.promisify(LDAP.prototype.search);
function ldapSearch(params) {
var conn = new LDAP();
return conn.searchAsync('ou=groups,o=amazon.com', { ... });
}
router.post('/writeRecord', jsonParser, function(req, res) {
let post = req.post;
let smdb = new AWS.DynamoDB.DocumentClient();
let params = { ... }
Promise.all([
ldapSearch(...),
ldapSearch(...),
smdb.query(params).promise()
]).then(([ldap1Result, ldap2Result, queryResult]) => {
// process ldap1Result, ldap2Result and queryResult here
}).catch(err => {
console.log(err);
res.status(500).send("Internal Error");
});
});
Keep in mind that due to the pseudo-code nature of the code in your question, this is also pseudo-code where implementation details (exactly what parameters you're searching for, what response you're sending, etc...) have to be filled in. This should be illustrative of promise chaining to serialize operations and the use of Promise.all() for parallelizing operations and promisifying a method that didn't have promises built in.
This question already has an answer here:
How to chain and share prior results with Promises [duplicate]
(1 answer)
Closed 6 years ago.
I'm new to promise. I am trying to use promise to send queries to mysql db. After some queries I will use the result from the query, do some calculations and then use the output as some parameters of next query. Looks like the following:
firstQuery(). then(secondQuery). then(thirdQuery). then(fourthQuery). ...
Say, in the fourthQuery, I need to use results coming from firstQuery and secondQuery, and will have some additional calculations. How should I do that?
I know I can get the result from the previous promise by passing a parameter to the function:
then(thirdQuery). then(cal("I can only get output from thirdQuery here")). then(fourthQuery("pass output from cal"))
In this case, I don't any advantages of Promise over callbacks, because I can always write a function to simplify the repeated callbacks.
If you can rewrite firstQuery, secondQuery etc you can do something like this
function firstQuery(allResult = {}) {
return doTheQuery()
.then(result => {
allResult.first = result;
return result;
});
}
function secondQuery(allResult = {}) {
return doTheQuery()
.then(result => {
allResult.second = result;
return result;
});
}
function thirdQuery(allResult = {}) {
return doTheQuery()
.then(result => {
allResult.third = result;
return result;
});
}
function fourthQuery(allResult = {}) {
return doTheQuery(allRessult.first, allResult.second)
.then(result => {
allResult.fourth = result;
return result;
});
}
then you can write use
firstQuery()
.then(secondQuery)
.then(thirdQuery)
.then(fourthQuery)
.then ...
the final result will be an object with the values of all the queries in {first, second, third, fourth} properties
Of course, if you want just the fourth query result
firstQuery()
.then(secondQuery)
.then(thirdQuery)
.then(fourthQuery)
.then(result => result.fourth)
.then ...
In that quite common cases you can move results of each then outside promise, and then they will be accessible in the remaining then blocks.
For example:
function first() { ... }
function second() { ... }
function third() { ... }
function fourth(firstValue, secondValue) { ... }
var firstResponse, secondResponse;
first()
.then(function(_firstResponse) {
firstResponse = _firstResponse;
return second();
})
.then(function(_secondResponse) {
secondResponse = _secondResponse;
return third();
})
.then(function() {
return fourth(firstResponse, secondResponse);
})
Usually you should think of promises as values and dependencies rather than a way to control the execution flow. One solid option is organizing your code something like this:
var firstResult = firstQuery();
var secondResult = firstResult.then(secondQuery);
var thirdResult = secondResult.then(thirdQuery);
var fourthResult = Promise.join(firstResult, secondResult, fourthQuery);
Basically I guess the key here is knowing of the join method of the bluebird library (others exist that could be used for the same effect), but an additional benefit is I find this kind of code much less error prone than one mixing raw variables and promise-variables.
Also note that this is possible because it's totally fine to call then on the same promise multiple times.
I am new to Node and want to call a function and determine whether or not to call the next function based on the results of the first function. The functions are asynchronous so the results are not immediately known. The following code works fine but now I want to add more asynchronous functions and chain them all together.
var doSomething = require('./custom-module1.js'); //returns a boolean
var doAnotherThing = require('./custom-module2.js'); //returns a double
var var1 = process.argv[2];
var var2 = process.argv[3];
var var3 = process.argv[4];
doSomething(var1, var3, function(data) {
if (data) {
doAnotherThing(var1,var2, function(data){
console.log(var1 + ' is value: ' + data);
});
}
});
I want to make the code something like this using promises.
getListOfStuff()
.then(
for(i=0;i<data.length;i++) {
doSomething(data[i], var3)
.then(
if (doSomething returns true) {
doAnotherThing(data[i], var2)
.then(doYetAnotherThing(data[i]))
.then(andEvenMoreThingsToBeDone(data[i]))
}
);
}
);
I read about q and using denodefiy but I can't understand the syntax to make it work. Would it be best to modify my custom modules to return promises so they would be inherently thenable? I get the concept but not the syntax, how do I actually do it?
I tried to denodeify using the following 2 different notations.
var doSomethingDN = Q.denodeify(doSomething);
var doAnotherThingDN= Q.denodeify(doAnotherThing);
doSomethingDN(var1, var3)
.then(
if(!data) {
doAnotherThingDN(var1,var2)
}
);
And this one
var doSomethingDN = Q.nfbind(doSomething);
var doAnotherThingDN= Q.nfbind(doAnotherThing);
doSomethingDN (var1, var3)
.then(
if(!data) {
doAnotherThingDN(var1,var2)
}
);
The if get's an unexpected token error. If I remove then both run but the boolean result from doSomething isn't able to control the program flow.
You still need to use a callback function with promises:
doSomethingDN (var1, var3)
.then(function(data) {
// ^^^^^^^^^^^^^^^^
if (!data) {
return doAnotherThingDN(var1,var2)
}
}); /*
^
You were getting a syntax error because you put the function body right as the argument to then.
I am new to node.js and I thought I was beginning to understand asynchronous functions, but this code made me think that I did not understand it correctly anyway.
I am preparing for an insert to mongoDB with mongoose, and the object to insert is post. What made me wonder is that not always post.kunde or post.leverandor is set before the insert.
I thought that as long there is no asynchronous function, the code should execute line by line.
function create_post(account, dato, fakturanummer, bilag, bilagstype, supplier, customer,
descr, moms, amount, saldo, comId ) {
return new Promise(function(resolve, reject) {
var post = new Posteringer;
post.konto = account;
post.dato = dato;
post.fakturanummer = fakturanummer;
post.bilag = bilag;
post.bilagstype = bilagstype;
if (Object.keys(supplier).length) {
post.leverandor = supplier;
}
if (Object.keys(customer).length) {
post.kunde = customer;
}
post.tekst = descr;
post.moms = moms;
post.belob = amount;
post.saldo = saldo;
post.companyId = comId;
var promise = Accounts.findSingle(comId, account).exec();
promise.then(function(acc) {
post.navn = acc.Navn;
console.log(post);
post.save(function(err) {
if (err) {console.log(err.message);}
resolve(true);
});
});
});
}
So there are a few things wrong here. A promise is basically saying "hey, I'm not done doing my stuff yet, but I'll promise to get this to you when I figure out all the things I need to do."
I am not entirely sure how you are calling this, but think of this like a big callback function. So after this you'd have something like,
create_post(......).then(
function(post){
post.save(function(err){
if (err) {console.log(err.message);}
});
});
The real issue I see is you are only resolving one promise. I am not 100% sure if you have to resolve the Mongo promise, unless you are using bluebird promises with it. Your first promise never gets returned though.
I will redo your code, I think this should work:
function create_post(account, dato, fakturanummer, bilag, bilagstype, supplier, customer,
descr, moms, amount, saldo, comId ) {
return new Promise(function(resolve, reject) {
var post = new Posteringer;
post.konto = account;
post.dato = dato;
post.fakturanummer = fakturanummer;
post.bilag = bilag;
post.bilagstype = bilagstype;
if (Object.keys(supplier).length) {
post.leverandor = supplier;
}
if (Object.keys(customer).length) {
post.kunde = customer;
}
post.tekst = descr;
post.moms = moms;
post.belob = amount;
post.saldo = saldo;
post.companyId = comId;
var promise = Accounts.findSingle(comId, account).exec();
promise.then(function(acc) {
post.navn = acc.Navn;
console.log(post);
});
return (err ? reject(err) : resolve(post));
});
}
Then when you call this function call it like in my first example!
create_post(......).then(
function(post){
post.save(function(err){
if (err) {console.log(err.message);}
});
});
Literally what you are saying is if this resolves and I get my data without any errors, send back post. Then pass post into my callback function and do whatever you want to do with it.
Edit:
Make sure you always return something from your promises, as far as I know the async call will never resolve as it never receives anything. Although someone might have an example where this isn't true.