I'm implementing my service using Node.js with AWS DynamoDB (aws-sdk).
It's unclear for me how to implement the following scenario with promises:
I get a request to modify an entity with specified attributes
I'm trying to find the entity in the DB (async call find)
If the entity not found then create one with initial state (async call createInitialStateObject)
Modify the entity (which was in the DB before or just created on step 3) according to specific rules (async call applyModifications)
This is my first attempt:
function scenario(params) {
find(params).then((data) => {
let objectExists = checkExistense(data);
if (!objectExists) {
createInitialStateObject(params).then((data) => {
console.log("Object created");
// OK
}).catch((err) => {
console.error("Object not created");
// exit and return error
});
}
applyModifications(params).then((data) => {
// OK, return data
}).catch((err) => {
// exit and return error
});
}).catch((err) => {
// exit and return error
});
}
But the flaw here is that creation could happen before the modification, it's not bound to happen one after another.
The other attempt works, but looks a bit weird. I create an empty promise to call in case the object already exists:
function scenario(params) {
find(params).then((data) => {
let conditionalPromise = new Promise((resolve) => {
resolve(null);
});
let objectExists = checkExistense(data);
if (!objectExists) {
conditionalPromise = createInitialStateObject(params);
}
conditionalPromise.then((data) => {
applyModifications(params).then((data) => {
// OK, return data
}).catch((err) => {
// exit and return error
});
}).catch((err) => {
// exit and return error
});
}).catch((err) => {
// exit and return error
});
}
How it should be implemented in a right way?
Creating 'empty' or sync. Promises isn't unusual. There is even a short way of doing that: Promise.resolve(value) creates and resolves a Promise immediately.
Besides that you should make use of proper chaining and stop nesting things so much. Once you are in a chain, you don't even need to resolve an empty promise as a return value of a non thenable object is interpreted as exactly this.
function scenario(params) {
return find(params)
.then(data => {
let objectExists = checkExistense(data);
if (!objectExists) {
return createInitialStateObject(params);
}
// if we return nothing (or null in your case) this will be the same as return Promise.resolve()
return null;
})
.then(data => applyModifications(params))
.then(data => console.log(data))
.catch(err => console.log(err));
// exit and return error
}
Related
I have a set of functions in Node.js that I would like to load in a certain order. I will provide some mockup code abstracted and simplified:
function updateMyApp() {
loadDataToServer()
.then(() => useData())
.then(() => saveData())
.then(() => { console.log("updateMyApp done") })
}
function loadDataToServer() {
return new Promise( (resolve, reject) {
...preparing data and save file to cloud...
resolve()})
}
function handleDataItem(item) {
// Function that fetches data item from database and updates each data item
console.log("Name", item.name)
}
function saveData() {
// Saves the altered data to some place
}
useData is a bit more complex. In it I would like to, in order:
console.log('Starting alterData()')
Load data, as json, from the cloud data source
Iterate through every item in the json file and do handleDataItem(item) on it.
When #2 is done -> console.log('alterData() done')
Return a resolved promise back to updateMyApp
Go on with saveData() with all data altered.
I want the logs to show:
Starting useData()
Name: Adam
Name: Ben
Name: Casey
useData() done
my take on this is the following:
function useData() {
console.log('Starting useData()')
return new Promise( function(resolve, reject) {
readFromCloudFileserver()
.then(jsonListFromCloud) => {
jsonListFromCloud.forEach((item) => {
handleDataItem(item)
}
})
.then(() => {
resolve() // I put resolve here because it is not until everything is finished above that this function is finished
console.log('useData() done')
}).catch((error) => { console.error(error.message) })
})
}
which seems to work but, as far as I understand this is not how one is supposed to do it. Also, this seems to do the handleDataItem outside of this chain so the logs look like this:
Starting useData()
useData() done
Name: Adam
Name: Ben
Name: Casey
In other words. It doesn't seem like the handleDataItem() calls are finished when the chain has moved on to the next step (.then()). In other words, I can not be sure all items have been updated when it goes on to the saveData() function?
If this is not a good way to handle it, then how should these functions be written? How do I chain the functions properly to make sure everything is done in the right order (as well as making the log events appear in order)?
Edit: As per request, this is handleDataItem less abstracted.
function handleDataItem(data) {
return new Promise( async function (resolve) {
data['member'] = true
if (data['twitter']) {
const cleanedUsername = twitterApi.cleanUsername(data['twitter']).toLowerCase()
if (!data['twitter_numeric']) {
var twitterId = await twitterApi.getTwitterIdFromUsername(cleanedUsername)
if (twitterId) {
data['twitter_numeric'] = twitterId
}
}
if (data['twitter_numeric']) {
if (data['twitter_protected'] != undefined) {
var twitterInfo = await twitterApi.getTwitterGeneralInfoToDb(data['twitter_numeric'])
data['twitter_description'] = twitterInfo.description
data['twitter_protected'] = twitterInfo.protected
data['twitter_profile_pic'] = twitterInfo.profile_image_url.replace("_normal", '_bigger')
data['twitter_status'] = 2
console.log("Tweeter: ", data)
}
} else {
data['twitter_status'] = 1
}
}
resolve(data)
}).then( (data) => {
db.collection('people').doc(data.marker).set(data)
db.collection('people').doc(data.marker).collection('positions').doc(data['report_at']).set(
{
"lat":data['lat'],
"lon":data['lon'],
}
)
}).catch( (error) => { console.log(error) })
}
The twitterAPI functions called:
cleanUsername: function (givenUsername) {
return givenUsername.split('/').pop().replace('#', '').replace('#', '').split(" ").join("").split("?")[0].trim().toLowerCase()
},
getTwitterGeneralInfoToDb: async function (twitter_id) {
var endpointURL = "https://api.twitter.com/2/users/" + twitter_id
var params = {
"user.fields": "name,description,profile_image_url,protected"
}
// this is the HTTP header that adds bearer token authentication
return new Promise( (resolve,reject) => {
needle('get', endpointURL, params, {
headers: {
"User-Agent": "v2UserLookupJS",
"authorization": `Bearer ${TWITTER_TOKEN}`
}
}).then( (res) => {
console.log("result.body", res.body);
if (res.body['errors']) {
if (res.body['errors'][0]['title'] == undefined) {
reject("Twitter API returns undefined error for :'", cleanUsername, "'")
} else {
reject("Twitter API returns error:", res.body['errors'][0]['title'], res.body['errors'][0]['detail'])
}
} else {
resolve(res.body.data)
}
}).catch( (error) => { console.error(error.message) })
})
},
// Get unique id from Twitter user
// Twitter API
getTwitterIdFromUsername: async function (cleanUsername) {
const endpointURL = "https://api.twitter.com/2/users/by?usernames="
const params = {
usernames: cleanUsername, // Edit usernames to look up
}
// this is the HTTP header that adds bearer token authentication
const res = await needle('get', endpointURL, params, {
headers: {
"User-Agent": "v2UserLookupJS",
"authorization": `Bearer ${TWITTER_TOKEN}`
}
})
if (res.body['errors']) {
if (res.body['errors'][0]) {
if (res.body['errors'][0]['title'] == undefined) {
console.error("Twitter API returns undefined error for :'", cleanUsername, "'")
} else {
console.error("Twitter API returns error:", res.body['errors'][0]['title'], res.body['errors'][0]['detail'])
}
} else {
console.error("Twitter API special error:", res.body)
}
} else {
if (res.body['data']) {
return res.body['data'][0].id
} else {
//console.log("??? Could not return ID, despite no error. See: ", res.body)
}
}
},
You have 3 options to deal with your main issue of async methods in a loop.
Instead of forEach, use map and return promises. Then use Promise.all on the returned promises to wait for them to all complete.
Use a for/of loop in combination with async/await.
Use a for await loop.
It sounds like there's a problem in the implementation of handleDataItem() and the promise that it returns. To help you with that, we need to see the code for that function.
You also need to clean up useData() so that it properly returns a promise that propagates both completion and errors.
And, if handleDataItem() returns a promise that is accurate, then you need to change how you do that in a loop here also.
Change from this:
function useData() {
console.log('Starting useData()')
return new Promise( function(resolve, reject) {
readFromCloudFileserver()
.then(jsonListFromCloud) => {
jsonListFromCloud.forEach((item) => {
handleDataItem(item)
}
})
.then(() => {
resolve() // I put resolve here because it is not until everything is finished above that this function is finished
console.log('useData() done')
}).catch((error) => { console.error(error.message) })
})
}
to this:
async function useData() {
try {
console.log('Starting useData()')
const jsonListFromCloud = await readFromCloudFileserver();
for (let item of jsonListFromCloud) {
await handleDataItem(item);
}
console.log('useData() done');
} catch (error) {
// log error and rethrow so caller gets the error
console.error(error.message)
throw error;
}
}
The structural changes here are:
Switch to use async/await to more easily handle the asynchronous items in a loop
Remove the promise anti-pattern that wraps new Promise() around an existing promise - no need for that AND you weren't capturing or propagating rejections from readFromCloudFileServer() which is a common mistake when using that anti-pattern.
rethrow the error inside your catch after logging the error so the error gets propagated back to the caller
I'm looking to write some code that chains together some promises. I have a condition that based on the result of one promise I either call the next function that returns a promise and continue chaining another few functions, or I do nothing (effectively end the promise chain).
I have the following three possible solutions, I kinda think they all are a bit messy though.
Here is my first approach, what I dislike here is the nested promises.
initalAsyncCall()
.then((shouldContinue) => {
if (shouldContinue) {
return nextStep()
.then(() => anotherStep())
}
})
.catch((error) => {
handleError(error);
})
Here is my second. This one seems a bit longer and possibly harder to read
const shouldContinuePromise = initialAsyncCall();
const nextStepPromise = shouldContinuePromise.then((shouldContinue) => {
if (shouldContinue) return nextStep();
});
Promise.all([shouldContinuePromise, nextStepPromise])
.spread((shouldContinue) => {
if (shouldContinue) return anotherStep();
})
.catch((error) => {
handleError(error);
});
And finally here is my last approach. What I don't like here is I am throwing an error when it's not really an error.
initalAsyncCall()
.then((shouldContinue) => {
if (!shouldContinue) throw new HaltException()
return nextStep();
})
.then(() => anotherStep())
.catch(HaltException, (ex) => {
// do nothing... maybe some logging
})
.catch((error) => {
handleError(error);
})
Your first approach seems fine, to avoid the nesting you can return the promise and add an extra then block for the nested part like this
initalAsyncCall()
.then((shouldContinue) => {
if (shouldContinue) {
return nextStep()
} else {
throw Error('skip next step')
}
})
.then(() => anotherStep())
.catch((error) => {
handleError(error);
})
If you don't like throwing an unnecessary error in the third approach, You could use async/await to have more control and get rid of the function scope/nesting problem, which is also recommended for the new nodejs versions due to better error stack traces.
try {
const shouldContinue = await initalAsyncCall()
if (shouldContinue) {
await nextStep()
await anotherStep()
// or await Promise.all([nextStep(), anotherStep()]) if they're not dependent
}
}
catch (error) {
handleError(error);
}
I am making a test app using systeminformation. I'm trying to make it so that each then waits for the previous function to finish. The problem I'm having is that the functions I am running inside are also promises, so the next then runs before the function finishes.
const si = require('systeminformation');
var cpuObj;
function initCPU() {
return new Promise(resolve => {
si.cpu()
.then(data => cpuObj = data)
.catch(err => console.log(err))
.then(() => {
setTimeout(() => console.log("timer"), 3000);
})
.then(() => {
si.cpuTemperature().then(data => console.log(data));
})
.then(() => {
console.log("here");
});
});
}
function test() {
console.log(cpuObj);
}
initCPU().then(() => {
test();
});
Output:
here
{ main: -1, cores: [], max: -1 }
timer
Expected Output:
{ main: -1, cores: [], max: -1 }
timer
here
A few points that need to be addressed:
setTimeout() does not return a promise, so you need to promisify and return it.
Flatten your chain by returning the promises from within each of the continuations rather than attempting to chain continuations within other continuations (i.e. then() inside of then()).
Do not wrap the continuation chain with a promise constructor, as the chain itself is already a promise, just return it directly instead. This is considered an antipattern.
Do not use globals, because it makes the initCPU() no longer re-entrant safe. Multiple calls to initCPU() before the promise returned by the first call resolves will result in unexpected behavior otherwise. Instead, use the appropriate scope to pass values along, which in this case is the function itself.
Allow errors to propagate to the caller and let the caller decide how to handle the error. Do not handle errors from within initCPU() unless you expect to use a fallback and continue to provide meaningful data to the caller.
const si = require('systeminformation');
const delay = ms => new Promise(resolve => { setTimeout(resolve, ms); });
function initCPU() {
// use local scope, not global
let cpuObj;
// return this promise chain directly
return si.cpu()
.then(data => {
cpuObj = data;
// return the promise to the chain
return delay(3000);
})
// let caller handle errors
// .catch(err => console.log(err))
// flatten your chain
.then(() => {
console.log('timer');
// return the promise to the chain
return si.cpuTemperature();
})
// flatten your chain
.then(data => {
console.log(data);
console.log('here');
// pass data to caller
return cpuObj;
});
}
function test(cpuObj) {
// received from last continuation of initCPU()
console.log(cpuObj);
}
initCPU()
.then(test)
// handle error from caller
.catch(err => {
console.log(err);
});
If you just want to query the cpu object immediately, and query cpuTemperature after 3 seconds, I'd do something like this using Promise.all():
// default to 3 seconds, allow it to be configurable
function initCPU(ms = 3000) {
return Promise.all([
si.cpu(),
delay(ms).then(() => si.cpuTemperature())
]).then(([cpu, cpuTemperature]) => ({
cpu,
cpuTemperature
}));
}
function test (obj) {
console.log(obj.cpu);
console.log(obj.cpuTemperature);
}
initCPU()
.then(test)
.catch(err => {
console.log(err);
});
I was reading some nodejs tutorial which talks about rejection in nodejs. They say that it's best practice to reject an error instead of a string or an plain text. Taking example of this code.
This is a example of rejecting a string
function cookMeat(chef){
grillMeat(chef)
.then(meat => {
if(chef.isTired){
return Promise.reject(chef.tiredReason);
}
return Promise.resolve(meat);
})
}
function cookNoodle(cheif){
boilNoodle(chef)
.then(noodle => {
if(chef.isTired){
return Promise.reject(chef.tiredReason);
}
return Promise.resolve(noodle);
})
}
function cook(){
let chef
prepareFood()
.then(c => {
chef = c;
return true;
})
.then(() => cookMeat(chef))
.then(() => cookNoodle(chef))
.catch(err => {
state: Fail,
reason: error
})
.then(res => {
state:Ready
})
}
cook()
.then((res) => serveCustomer(res))
And this is a example of rejecting an error
function cookMeat(chef){
grillMeat(chef)
.then(meat => {
if(chef.isTired){
return Promise.reject(new Error(chef.tiredReason));
}
return Promise.resolve(meat);
})
}
function cookNoodle(cheif){
boilNoodle(chef)
.then(noodle => {
if(chef.isTired){
return Promise.reject(new Error(chef.tiredReason));
}
return Promise.resolve(noodle);
})
}
function cook(){
let chef
prepareFood()
.then(c => {
chef = c;
return true;
})
.then(() => cookMeat(chef))
.then(() => cookNoodle(chef))
.catch(err => {
state: Fail,
reason: error.message
})
.then(res => {
state:Ready
})
}
cook()
.then((res) => serveCustomer(res))
Since I want to use reject to skip part of the promise chain. So I am wondering if there are any difference?
wPromise rejections are similar to throwing exceptions / error objects.
There are two rules that apply to throwing exceptions that apply here too:
In javascript, it's better to throw an Error object. Among other things, you will get stack information. It's also what most people expect when using a javascript code base.
Don't use exceptions for flow-control
The second one is such a common advice, you can google it verbatim and learn more. You're using Promise rejections as flow control and this is a bad idea.
Your functions can be rewritten a bit though. This is even better:
function cookMeat(){
grillMeat()
.then(meat => {
if(meat.isRaw){
throw new Error(meat.rawReason);
}
return meat;
});
}
function cookNoodle(){
boilNoodle()
.then(noodle => {
if(noodle.isRaw){
throw new Error(noodle.rawReason);
}
return noodle;
})
}
function cook(){
return prepareFood()
.then(() => cookMeat())
.then(() => cookNoodle())
.catch(err => {
state: Fail,
reason: error.message
})
.then(res => {
state:Ready
})
}
cook()
.then((res) => talkWithCustomer(res))
I got rid of your Promise.reject and Promise.resolve statements, because they are unneccary from within a then() function. The advice to use them only really applies 'outside' of then() chains.
I have a constructor which fetches data from a DynamoDB using promisified dynogels to populate part of the object's properties.
So after instantiating an instance of that object, the property is not populated, here is an extract of the code:
export class QueryAuthoriser {
authPerms: [AuthPerms];
constructor (data: string) {
AuthPermsDDB.scan().execAsync().then ( (perms) => {
perms.Items.forEach(element => {
this.authPerms[element.name] = <AuthPerms> element.attrs
})
}).catch (err => {
console.log ('%%%%%%%%%%%%%% Err loading authPerms: ', err)
})
}
authFieldAccess (fieldName: string, args?:any): Promise<boolean> {
return new Promise ((resolve, reject) => {
console.log ('________________ authFieldAccess called for: ', fieldName)
console.log ('________________ this.authPerms entry: ', this.authPerms[fieldName])
resolve (true)
})
[...]
}
So when authFieldAccess method is called, the field this.authPerms is undefined. How can I fix this?
Thanks, I'm learning node and typescript the hard way :O
You generally don't want to carry out an asynchronous operation in a constructor because it complicates creating an object and then knowing when the async operation is done or if it had an error because you need to allow the constructor to return the object, not a promise that would tell you when the async operation was done.
There are several possible design options:
Option #1: Don't do any async operation in the constructor. Then, add a new method with an appropriate name that does the async operation and returns a promise.
In your case, you could make the new method be scan() that returns a promise. Then, you'd use your object by creating it and then calling scan and then using the returned promise to know when the data is valid.
I don't know TypeScript myself, so I'll give a modified version of your code, but the concept is the same either way whether it's TypeScript or plain Javascript:
export class QueryAuthoriser {
authPerms: [AuthPerms];
constructor (data: string) {
}
scan () {
return AuthPermsDDB.scan().execAsync().then ( (perms) => {
perms.Items.forEach(element => {
this.authPerms[element.name] = <AuthPerms> element.attrs
})
}).catch (err => {
console.log ('%%%%%%%%%%%%%% Err loading authPerms: ', err)
})
}
}
// usage
let obj = new QueryAuthoriser(...);
obj.scan(...).then(() => {
// the object is full initialized now and can be used here
}).catch(err => {
// error here
})
Option #2: Initiate the async operation in the constructor and use a promise in the instance data for the caller to know when everything is done.
export class QueryAuthoriser {
authPerms: [AuthPerms];
constructor (data: string) {
this.initialScan = AuthPermsDDB.scan().execAsync().then ( (perms) => {
perms.Items.forEach(element => {
this.authPerms[element.name] = <AuthPerms> element.attrs
})
}).catch (err => {
console.log ('%%%%%%%%%%%%%% Err loading authPerms: ', err)
})
}
}
// usage
let obj = new QueryAuthoriser(...);
obj.initialScan.then(() => {
// the object is full initialized now and can be used here
}).catch(err => {
// error here
});
Option #3: Use a factory function that returns a promise that resolves to the object itself.
export createQueryAuthorizer;
function createQueryAuthorizer(...) {
let obj = new QueryAuthorizer(...);
return obj._scan(...).then(() => {
// resolve with the object itself
return obj;
})
}
class QueryAuthoriser {
authPerms: [AuthPerms];
constructor (data: string) {
}
_scan () {
return AuthPermsDDB.scan().execAsync().then ( (perms) => {
perms.Items.forEach(element => {
this.authPerms[element.name] = <AuthPerms> element.attrs
})
}).catch (err => {
console.log ('%%%%%%%%%%%%%% Err loading authPerms: ', err)
})
}
}
// usage
createQueryAuthorizer(...).then(obj => {
// the object is fully initialized now and can be used here
}).catch(err => {
// error here
});
My preference is for option #3 for several reasons. It captures some shared code in the factory function that every caller has to do in the other schemes. It also prevents access to the object until it is properly initialized. The other two schemes just require documentation and programming discipline and can easily be misused.