I am using twitter's API to receive recent tweets by querying specific hash tags. The first GET request is to search/tweets which takes the hashtag and others queries in it's parameters. The response for this returns an array of tweets (objects). I push each of these id's into an external array. Now I want to loop through this array and make another call to statuses/show for each of these IDs so that I can get the location data of the user posting the tweet. The statuses/show end-point only takes a single tweet id but I have an array. How would I go about using the same getRequest function for an array of IDs?
I tried to implement it by reading online about Promises but it's not working.
Here's my code:
function getRequest(url, params) {
return new Promise(function (success, failure) {
twitterSearch.get(url, params, function (error, body, response) {
if (!error) {
success(body);
} else {
failure(error);
}
});
});
}
app.post('/twitter', (req, res) => {
console.log("body", req.body);
var tweetIDs = [];
var bounds = [];
getRequest('search/tweets', {q: req.body.tag, result_type:'recent', count:'100'})
.then((tweets) => {
tweets.statuses.map((status) => {
if(status.user.geo_enabled) {
console.log("ID: ", status.id);
tweetIDs.push(status.id);
}
});
return getRequest('statuses/show', tweetIDs); //I know tweetIDs is wrong, dont know what to do from here.
}).then((data) => {
console.log("User data for the ID")
console.log(data);
}).catch((error) => {
console.log(error)
});
res.send(bounds);
bounds.length = 0;
});
You nearly got it right! Map all tweets to Promises returned by your getRequest() function. Then, the Promise.all() method will return you a single Promise that resolves when all of the promises in your array resolves.
By the way, you can use Array.reduce instead of map to both filter and map your array at the same time, this factor your code a little better.
getRequest('search/tweets', {q: req.body.tag, result_type:'recent', count:'100'})
.then( tweets => {
let requests = tweets.statuses.reduce( (ret,status) => {
if(status.user.geo_enabled)
// The following is still wrong though, I guess. Format the params of getRequest accordingly.
ret.push( getRequest('statuses/show', status.id) );
return ret;
}, []);
return Promise.all(requests);
})
.then( results => {
results.forEach( res => {
console.log("User data for the ID");
console.log(res);
})
})
EDIT : Related jsfiddle
Replace the line
return getRequest('statuses/show', tweetIDs); //I know tweetIDs is wrong, dont know what to do from here.
with
return Promisel.all( tweetIDs.map( (tId) => getRequest('statuses/show', tId) );
Related
I am working on a project and on the newer side of Node. This is someone else's original implementation and I'm having an issue with promises and I know it most likely has to do with async/await or promise.all but I've spent alot time trying and still not getting the desired result.
I am going to the PlaceStore and returning a promise to get a list of people from a sql database.
After I have the list of people. I map over the the list so I can have each person's id. Inside the map (which may be the problem because maps are not asynchronous) I am trying to find the most recent thing done by that person by calling the getMostRecentThingByPerson function that goes to the ThingStore to get the most recent thing.
When I log the resent in the result of the getMostRecentThingByPerson it does log the correct values but I don't know how to make the value available in the map function so I can update and return the object containing the 'Most Recent Thing' and export it to Csv.
I've read alot about how this is an important topic and thank you so much for your guidance with this issue.
exports.exportPeople = (req, res, next) => {
const id = req.body.id;
const place = req.place;
PlaceStore.findPerson(place, async (err, people) => {
if (err) {
return new Error(err);
}
const peopleForCsv = people.map((person) => {
const mostRecentThing = this.getMostRecentThingByPerson(person.id) // tried adding await here but does not work because of the map.
return {
'First Name': person.first_name,
'Last Name': person.last_name,
'Most Recent Thing': mostRecentThing
};
});
const json2csvParser = new Parser({ delimiter: ',' });
const csv = json2csvParser.parse(peopleForCsv);
return res
.status(200)
.attachment('people_' + id + '.csv')
.set('Content-Type', 'text/csv')
.send(csv);
});
};
exports.getMostRecentThingByPerson= (id) => {
ThingStore.getThings({"person_id": id}, (err, things) => {
if (err) {
return new Error(err);
}
result = things.rows.length > 0 ? things.rows[0].capture_date : ''
console.log(`Recent thing result for ${id}`, result) //example date
return result
})
return result
}
As mentioned in the comments below your question, you need to promisify your callback-based code. This is a fundamental pattern with which you need to get familiar.
To promisify your function getMostRecentThingByPerson, you need to create and return a promise which resolves on the result or rejects on the error (if any):
exports.getMostRecentThingByPerson = (id) => {
return new Promise((resolve, reject) => {
ThingStore.getThings({"person_id": id}, (err, things) => {
if (err) {
reject(err);
} else {
const result = things.rows.length > 0 ? things.rows[0].capture_date : ''
console.log(`Recent thing result for ${id}`, result) // example date
resolve(result)
}
})
})
}
Now, getMostRecentThingByPerson returns a promise, so you need to await its resolution when calling it. If you do that inside a loop, you need to await the completion of each promise that will be created by each iteration of the loop:
exports.exportPeople = (req, res, next) => {
const id = req.body.id;
const place = req.place;
PlaceStore.findPerson(place, async (err, people) => {
if (err) {
return new Error(err);
}
const peopleForCsv = await Promise.all(people.map(async (person) => {
const mostRecentThing = await this.getMostRecentThingByPerson(person.id)
return {
'First Name': person.first_name,
'Last Name': person.last_name,
'Most Recent Thing': mostRecentThing
}
});
const json2csvParser = new Parser({ delimiter: ',' });
const csv = json2csvParser.parse(peopleForCsv);
return res
.status(200)
.attachment('people_' + id + '.csv')
.set('Content-Type', 'text/csv')
.send(csv);
});
};
Note that your current code for exportPeople does not return anything so it relies on a side-effect from calling PlaceStore.findPerson. I don't know if that works but I think a better practice would be to promisify PlaceStore.findPerson instead, and return its promised value (csv). Then you await it and return res.status(200)/* rest of the code */
I currently have the following code
router.get('/uri', (request,response) => {
let final = [];
TP.find({userID: request.userID})
.then(tests =>{
tests.forEach(test => {
A.findById(test.assignmentID)
.then(assignment => {
final.push({
testID: test._id,
name: assignment.title,
successRate: `${test.passedTests}/${test.totalTests}`
})
})
.catch(error => {
console.log(error)
})
})
return response.send(final);
})
.catch(err => {
console.log(err);
return response.sendStatus(500);
})
})
The code is supposed to query 2 MongoDB databases and construct an array of objects with specific information which will be sent to the client.
However, I always get an empty array when I call that endpoint.
I have tried making the functions async and make them wait for results of the nested functions but without success - still an empty array.
Any suggestions are appreciated!
forEach doesn't care about promises inside it. Either use for..of loop or change it to promise.all. The above code can be simplified as
router.get('/uri', async (request,response) => {
const tests = await TP.find({userID: request.userID});
const final = await Promise.all(tests.map(async test => {
const assignment = await A.findById(test.assignmentID);
return {
testID: test._id,
name: assignment.title,
successRate: `${test.passedTests}/${test.totalTests}`
};
}));
return response.send(final);
});
Hope this helps.
I'm currently trying to send 2 objects to the front .hbs front end. However I cant seem to work out how to do this because I'm using promises.
Currently, my thinking is i perform the sql query, the country and organisation name is extracted, and then each sent to a geocoding api, returned and then squashed together in the same promises. But i'm not sure how to extract these for the render function.
Node
//route for homepage
app.get('/', (req, res) => {
let sql = "SELECT org_name, country_name from places;
let query = conn.query(sql, (err, results) => {
if (err) throw err;
const geoPromise = param => new Promise((resolve, reject) => {
geo.geocode('mapbox.places', param, function(err, geoData) {
if (err) return reject(err);
if (geoData) {
resolve(geoData.features[0])
} else {
reject('No result found');
}
});
});
const promises = results.map(result =>
Promise.all([
geoPromise(result.country_name),
geoPromise(result.org_name)
]));
Promise.all(promises).then((geoLoc, geoBus) => {
res.render('layouts/layout', {
results: JSON.stringify(geoLoc),
businesses: JSON.stringify(geoBus)
});
});
});
});
Front end call
results1 = {{{results}}}
console.log(results1.length)
business1 = {{{businesses}}}
console.log(business1.length)
Wrap your geo.geocode into a Promise
const geoPromise = param => new Promise((resolve, reject) => {
geo.geocode('mapbox.places', param, function(err, geoData) {
if (err) return reject(err);
if (geoData) {
resolve(geoData.features[0])
} else {
reject('No result found');
}
});
});
Combine both calls to geo.geocode
const promises = results.map(result =>
Promise.all([
geoPromise(result.country_name),
geoPromise(result.org_name)
]));
Call them
Promise.all(promises).then(([geoLoc, geoBus]) => {
res.render('layouts/layout', {
results: JSON.stringify(geoLoc),
businesses: JSON.stringify(geoBus)
});
});
As MadWard's answer mentions, deconstructing the argument of the callback of Promise.all is necessary since everything will be in the first argument. Make sure you check out his post for more details
Something important to recall: you will never have more than one argument in a then() callback.
Now you may ask: in the case of Promise.all(), what is this value?
Well, it is an array with all the values from the promises it awaits, in the order in which they are called.
If you do:
Promise.all([
resolveVariable1, resolveVariable2, resolveVariable3
]).then((values) => {
})
values will be [variable1, variable2, variable3], the three variables that the promises resolve to.
Your case is, however, a bit more complicated. What is gonna be returned at the end is a 2-D array containing every entry. It is an array of length results.length, and each of its element has a length of 2. The first element is the result, and the second one is the business.
Here is your snippet:
Promise.all(promises)
.then((values) => {
let results = values.map(elmt => elmt[0]);
let businesses = values.map(elmt => elmt[1]);
res.render('layouts/layout', {
results: JSON.stringify(results),
businesses: JSON.stringify(businesses)
});
})
I may be missing something really obvious here, but how do I use util.promisify with a function which looks like this?
function awkwardFunction (options, data, callback) {
// do stuff ...
let item = "stuff message"
return callback(null, response, item)
}
Which I can call like this:
awkwardFunction ({doIt: true},'some data', (err, result, item) => {
console.log('result')
console.log(result)
console.log('item')
console.log(item)
done()
})
And get back
result
{ data: 'some data' }
item
stuff message
When using the promisified version:
let kptest = require('util').promisify(awkwardFunction)
kptest({doIt: true},'some data')
.then((response, item) => {
console.log('response')
console.log(response)
console.log('item')
console.log(item)
})
.catch(err => {
console.log(err)
})
and trying to access both "response" and "item", it seems the 2nd param is ignored...
result
{ data: 'some data' }
item
undefined
Is there a way to do this WITHOUT changing the function (in reality, it is a library function, so I can't).
util.promisify is intended to be used with Node-style callbacks with function (err, result): void signature.
Multiple arguments can be treated manually:
let kptest = require('util').promisify(
(options, data, cb) => awkwardFunction(
options,
data,
(err, ...results) => cb(err, results)
)
)
kptest({doIt: true},'some data')
.then(([response, item]) => {...});
In case more sophisticated functionality is wanted, some third-party solution like pify can be used instead of util.promisify, it has multiArgs option to cover this case.
You could make your own promisify, where you return a promise that resolves with the arguments of the callback and on the then block you destructure them. Hope this helps.
function awkwardFunction (options, data, callback) {
// do stuff ...
let item = "stuff message";
return callback(null, data, item);
}
const mypromisify = (fn) =>
(...args) =>
new Promise(resolve =>
fn(...args,
(...a) => resolve(a)
)
);
const kptest = mypromisify(awkwardFunction);
kptest({ doIt: true }, 'some data')
.then(([error, response, item]) => {
console.log(response);
console.log(item);
})
.catch(err => {
console.log(err);
});
It is not possible to have .then((response, item) => { because a promise represents single value. But you could have it like this .then(({response, item}) => { an object w/ two fields.
You'll need to provide a custom promisify implementation for the function.
const { promisify } = require('util')
awkwardFunction[promisify.custom] = (options, data) => new Promise((resolve, reject) => {
awkwardFunction(options, data, (err, response, item) => {
if(err) { reject(err) }
else { resolve({ response, item }) }
})
})
const kptest = promisify(awkwardFunction)
Or if this is the only place where the function is promisified you could use the promisified version directly const kptest = (options, data) => new Promise(... w/o additional promisification step.
I was just rollling up my sleeves for an open heart surgery to achieve this, but I am glad I found someone has already done this.
If you use Bluebird's Promisify (it's getting so popular) then there actually is a flag of { multiArgs: true } that you can pass and will do exactly what you need here! (Source)
It turns multiple arguments of callback into an array. So, in my case for MySQL's query that the default callback has 3 arguments of (error, result, fields), getting fields is not possible with typical promisify. But with that {multiArgs: true} flag being passed, the resolved value will become an array of [result, fields].
I can't decide which approach I like the best - all 3 answers are great. Yury Tarabanko's is probably the most "standard", Alex G's is nicely generic, and estus's super simple.
I don't want to leave this question "Unanswered" because that is not true, and not useful for others looking for the same information.
If there is a better way t handle this, please can the moderators let me know!
I have a fairly straightforward CRUD app which renders the results of two queries onto one page. The problem that arose once I got this to "work" was that the page required a refresh in order to display the results. On first load, no results were displayed.
I came to figure out that this is a problem/symptom of Node's asynchronous nature. I've been trying to approach this problem by using async/await, and from hours of messing with things, I feel like I'm quite close to the solution, but it's just not working out - I still need a manual refresh to display/render the results on the .ejs page.
The code:
var entries = [];
var frontPageGoals = [];
app.get('/entries', async (req,res) => {
if (req.session.password) {
const entriesColl = await
db.collection('entries')
.find()
.sort({date: -1})
.toArray((err, result) => {
if (err) { console.log(err) }
else {
for (i=0; i<result.length; i++) {
entries[i] = result[i];
}
}
});
const goalsColl = await
db.collection('goals')
.find()
.toArray((err, result) => {
if (err) {console.log(err)}
else {
for (i=0; i<result.length; i++) {
frontPageGoals[i] = result[i];
}
}
});
res.render('index.ejs', {entries: entries, frontPageGoals: frontPageGoals});
}
else {
res.redirect('/');
}
});
Now, I can conceive of a few problems here, but honestly I'm just at my wits end trying to figure this out. For example, I'm sure it's problematic that the empty lists which will contain the results to be passed when the page renders are outside the actual async function. But after trying to move them a dozen different places within the async area... still no dice.
Any help would be hugely appreciated! This is basically the last big "thing" I need done for this app.
I'm not 100% sure about your database driver, but assuming that the toArray() returns a promise (which it does in the default mongodb driver), the await will actually return the value you expect in your callback, result in your case, or in case there was an error, which you expected it as err in your callback, it will be thrown, thus forcing you to use try-catch blocks, in your case, you would just use console.log(err) in the catch block, since you aren't doing any handling
Here's your code after updating :
app.get("/entries", async (req, res) => {
if (req.session.password) {
try {
const entries = await db
.collection("entries")
.find()
.sort({ date: -1 })
.toArray();
const frontPageGoals = await db
.collection("goals")
.find()
.toArray();
res.render("index.ejs", {
entries: entries,
frontPageGoals: frontPageGoals
});
} catch (err) {
console.log(err);
}
} else {
res.redirect("/");
}
});
EDIT
However, if you don't know about promises -which async/await are basically promises-, and wanna just do it using callbacks -not advised-, you would have to just send your response in the callback, and nest the 2nd query in the first query's callback, here is the code,, with some comments to hopefully help you out:
app.get("/entries", (req, res) => {
if (req.session.password) {
// First query
db.collection("entries")
.find()
.sort({ date: -1 })
.toArray((err, entryResult) => {
if (err) {
console.log(err);
} else {
// In the callback of the first query, so it will
// execute 2nd query, only when the first one is done
db.collection("goals")
.find()
.toArray((err, frontPageResult) => {
if (err) {
console.log(err);
} else {
// In the callback of the 2nd query, send the response
// here since both data are at hand
res.render("index.ejs", {
entries: entryResult,
frontPageGoals: frontPageResult
});
}
});
}
});
} else {
res.redirect("/");
}
});
I have removed the async keyword since you no longer need it
I renamed the callback arguments, instead of just result, because both callbacks would have the same argument name, and you would have had to store it in a temp variable