How to create multiple queries in a single GET request - node.js

I am using node-pg and would like to make multiple queries within a single GET request.
For example consider I make two queries like so:
const getSomeInfo = (request, response) => {
pool.query('SELECT * FROM jobs', (error, results) => {
if (error) {
throw error
}
var jobObj = results.rows;
response.render('pages/jobinfo', {
jobObj: jobObj
});
})
pool.query('SELECT * FROM description INNER JOIN views', (error, results) => {
if (error) {
throw error
}
var descObj = results.rows;
response.render('pages/jobinfo', {
descObj: descObj
});
})
}
This code results in the following error Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client.
Is there a way to make both these queries within the same GET request so that the objects containing the results can be used on the same page?

You can render only one document, but this can contain two local variables, one from each query. To have the queries executed in parallel, use Promise.all and make use of the fact that pool.query without a callback function returns a promise.
const getSomeInfo = (request, response) => {
Promise.all([
pool.query('SELECT * FROM jobs'),
pool.query('SELECT * FROM description INNER JOIN views')
]).then(function([jobResults, descResults]) {
response.render('pages/jobinfo', {
jobObj: jobResults.rows,
descObj: descResults.rows
});
}, function(error) {
throw error;
});
}

Related

Find document from mongoose collections with specific condition

Recently I start using MongoDB with Mongoose on Nodejs.
This code works as it should, and returns me all data i need :
const getAllPosts = async () => {
try {
return (await PostModel.find().populate('user')).reverse();
} catch (error) {
console.log(error);
throw Error('Error while getting all posts');
}
};
But now I only need individual posts, which in the tags (represented as an array in the PostModel) contain the data that I will pass in the request.
For example, I will make a GET request to /posts/tag111 and should get all posts that have "tag111" in the tags array.
Any ways to do this?
If you are using expressjs, your route should be something like:
whatever.get('/:tag', postController.getAllPostTagged)
The function you use in your route (called getAllPostTagged above) should be similar to this one, in which you get the path param tag from req:
const postController = {
getAllPostTagged = async(req, res) => {
try {
return (await PostModel.find({tags: req.params.tag}));
} catch (error) {
console.log(error);
throw Error('Error while getting all posts');
}
}
}
The key here is to know where the params are obtained (from req.params) .

Returning body of a request function inside a variable

I have an endpoint in my node backend in which will need to retrieve for each item in my Adhoc collection from my local database the _id along with a number value which I need to calculate from the body of a request() function in an array of objects. The objects will be like this
{id: "id", sum: 3}
To do this I need to iterate through the Adhocs with a for loop and make a request for each to get the sum value and I need to be able to store these values before I have all of them and res.send() the array to the front end. I am having trouble storing the sum value in a variable. I have provided below the code of the request.
let theSum = request(options, function (error, response, body) {
if (error) throw new Error(error);
console.log(
'Response: ' + response.statusCode + ' ' + response.statusMessage
);
let bodyy = JSON.parse(body);
let sum = bodyy.fields.timetracking.originalEstimateSeconds / 3600 * theRate;
return sum;
});
I know this is wrong as the return statement is for the function inside the request function so it won't return the sum to my variable. And adding another callback function will basically be the same scenario. Anyone has any suggestions of how I can store the value from the request function so I can make further calls?
I found an answer that works for me pretty well. I didn't try Terry's answer above but I suppose that works as well since it's using Promise as well as me. What I've done is in a function I wrapped the request call in a Promise with a callback which is returned. Code below:
function asyncRequest (url) {
return new Promise (function (resolve, reject) {
var options = {
url: 'http://localhost:8080/rest/' + url,
auth: { username: 'username', password: 'password' },
headers: {
'Accept': 'application/json'
}
}
request(options, function (err, response, body) {
if (err) reject(err);
resolve(JSON.parse(body))
});
})
}
When I want to retrieve something I just have something like this:
let json = await asyncRequest('agile/1.0/issue/'+ adhoc[u].jIssue);
And that variable has the body of the request function inside and I can use it.
You can use async and await along with request-promise-native to loop over your objects and get the list of results you wish to have.
You can call the readEstimates function in a express.get( ).. handler as long as the handler is asynchronous (or you can use readEstimates().then(..)).
Now, we will wrap an error handler around the readEstimates call since, this could potentially throw an error.
For example:
const rp = require('request-promise-native');
async function readEstimates() {
const sumList = [];
for(const adhoc of adhocList) {
// Set your options here, e.g. url for each request.. by setting json to true we don't need to JSON.parse the body.
let options = { url: SOME_URL, json: true, resolveWithFullResponse: true };
let response = await rp(options);
console.log('Response: ' + response.statusCode + ' ' + response.statusMessage);
const sum = response.body.fields.timetracking.originalEstimateSeconds / 3600 * theRate;
sumList.push(sum);
}
return sumList;
}
async function testReadEstimates() {
try {
const sumList = await readEstimates();
console.log("Sumlist:", sumList);
} catch (error) {
console.error("testReadEstimates: An error has occurred:", error);
}
}
testReadEstimates();
You can also use readEstimates in an Express route:
app.get('/', async (req, res) => {
try {
const sumList = await readEstimates();
res.json({sumList}); // Send the list to the client.
} catch (error) {
console.error("/: An error has occurred:", error);
res.status(500).send("an error has occurred");
}
})

How to send multiple objects from node backend to .hbs

I'm currently trying to send 2 objects to the front .hbs front end. However I cant seem to work out how to do this because I'm using promises.
Currently, my thinking is i perform the sql query, the country and organisation name is extracted, and then each sent to a geocoding api, returned and then squashed together in the same promises. But i'm not sure how to extract these for the render function.
Node
//route for homepage
app.get('/', (req, res) => {
let sql = "SELECT org_name, country_name from places;
let query = conn.query(sql, (err, results) => {
if (err) throw err;
const geoPromise = param => new Promise((resolve, reject) => {
geo.geocode('mapbox.places', param, function(err, geoData) {
if (err) return reject(err);
if (geoData) {
resolve(geoData.features[0])
} else {
reject('No result found');
}
});
});
const promises = results.map(result =>
Promise.all([
geoPromise(result.country_name),
geoPromise(result.org_name)
]));
Promise.all(promises).then((geoLoc, geoBus) => {
res.render('layouts/layout', {
results: JSON.stringify(geoLoc),
businesses: JSON.stringify(geoBus)
});
});
});
});
Front end call
results1 = {{{results}}}
console.log(results1.length)
business1 = {{{businesses}}}
console.log(business1.length)
Wrap your geo.geocode into a Promise
const geoPromise = param => new Promise((resolve, reject) => {
geo.geocode('mapbox.places', param, function(err, geoData) {
if (err) return reject(err);
if (geoData) {
resolve(geoData.features[0])
} else {
reject('No result found');
}
});
});
Combine both calls to geo.geocode
const promises = results.map(result =>
Promise.all([
geoPromise(result.country_name),
geoPromise(result.org_name)
]));
Call them
Promise.all(promises).then(([geoLoc, geoBus]) => {
res.render('layouts/layout', {
results: JSON.stringify(geoLoc),
businesses: JSON.stringify(geoBus)
});
});
As MadWard's answer mentions, deconstructing the argument of the callback of Promise.all is necessary since everything will be in the first argument. Make sure you check out his post for more details
Something important to recall: you will never have more than one argument in a then() callback.
Now you may ask: in the case of Promise.all(), what is this value?
Well, it is an array with all the values from the promises it awaits, in the order in which they are called.
If you do:
Promise.all([
resolveVariable1, resolveVariable2, resolveVariable3
]).then((values) => {
})
values will be [variable1, variable2, variable3], the three variables that the promises resolve to.
Your case is, however, a bit more complicated. What is gonna be returned at the end is a 2-D array containing every entry. It is an array of length results.length, and each of its element has a length of 2. The first element is the result, and the second one is the business.
Here is your snippet:
Promise.all(promises)
.then((values) => {
let results = values.map(elmt => elmt[0]);
let businesses = values.map(elmt => elmt[1]);
res.render('layouts/layout', {
results: JSON.stringify(results),
businesses: JSON.stringify(businesses)
});
})

Async/await in Express with multiple MongoDB queries

I have a fairly straightforward CRUD app which renders the results of two queries onto one page. The problem that arose once I got this to "work" was that the page required a refresh in order to display the results. On first load, no results were displayed.
I came to figure out that this is a problem/symptom of Node's asynchronous nature. I've been trying to approach this problem by using async/await, and from hours of messing with things, I feel like I'm quite close to the solution, but it's just not working out - I still need a manual refresh to display/render the results on the .ejs page.
The code:
var entries = [];
var frontPageGoals = [];
app.get('/entries', async (req,res) => {
if (req.session.password) {
const entriesColl = await
db.collection('entries')
.find()
.sort({date: -1})
.toArray((err, result) => {
if (err) { console.log(err) }
else {
for (i=0; i<result.length; i++) {
entries[i] = result[i];
}
}
});
const goalsColl = await
db.collection('goals')
.find()
.toArray((err, result) => {
if (err) {console.log(err)}
else {
for (i=0; i<result.length; i++) {
frontPageGoals[i] = result[i];
}
}
});
res.render('index.ejs', {entries: entries, frontPageGoals: frontPageGoals});
}
else {
res.redirect('/');
}
});
Now, I can conceive of a few problems here, but honestly I'm just at my wits end trying to figure this out. For example, I'm sure it's problematic that the empty lists which will contain the results to be passed when the page renders are outside the actual async function. But after trying to move them a dozen different places within the async area... still no dice.
Any help would be hugely appreciated! This is basically the last big "thing" I need done for this app.
I'm not 100% sure about your database driver, but assuming that the toArray() returns a promise (which it does in the default mongodb driver), the await will actually return the value you expect in your callback, result in your case, or in case there was an error, which you expected it as err in your callback, it will be thrown, thus forcing you to use try-catch blocks, in your case, you would just use console.log(err) in the catch block, since you aren't doing any handling
Here's your code after updating :
app.get("/entries", async (req, res) => {
if (req.session.password) {
try {
const entries = await db
.collection("entries")
.find()
.sort({ date: -1 })
.toArray();
const frontPageGoals = await db
.collection("goals")
.find()
.toArray();
res.render("index.ejs", {
entries: entries,
frontPageGoals: frontPageGoals
});
} catch (err) {
console.log(err);
}
} else {
res.redirect("/");
}
});
EDIT
However, if you don't know about promises -which async/await are basically promises-, and wanna just do it using callbacks -not advised-, you would have to just send your response in the callback, and nest the 2nd query in the first query's callback, here is the code,, with some comments to hopefully help you out:
app.get("/entries", (req, res) => {
if (req.session.password) {
// First query
db.collection("entries")
.find()
.sort({ date: -1 })
.toArray((err, entryResult) => {
if (err) {
console.log(err);
} else {
// In the callback of the first query, so it will
// execute 2nd query, only when the first one is done
db.collection("goals")
.find()
.toArray((err, frontPageResult) => {
if (err) {
console.log(err);
} else {
// In the callback of the 2nd query, send the response
// here since both data are at hand
res.render("index.ejs", {
entries: entryResult,
frontPageGoals: frontPageResult
});
}
});
}
});
} else {
res.redirect("/");
}
});
I have removed the async keyword since you no longer need it
I renamed the callback arguments, instead of just result, because both callbacks would have the same argument name, and you would have had to store it in a temp variable

How to use Promises correctly with multiple requests

I am using twitter's API to receive recent tweets by querying specific hash tags. The first GET request is to search/tweets which takes the hashtag and others queries in it's parameters. The response for this returns an array of tweets (objects). I push each of these id's into an external array. Now I want to loop through this array and make another call to statuses/show for each of these IDs so that I can get the location data of the user posting the tweet. The statuses/show end-point only takes a single tweet id but I have an array. How would I go about using the same getRequest function for an array of IDs?
I tried to implement it by reading online about Promises but it's not working.
Here's my code:
function getRequest(url, params) {
return new Promise(function (success, failure) {
twitterSearch.get(url, params, function (error, body, response) {
if (!error) {
success(body);
} else {
failure(error);
}
});
});
}
app.post('/twitter', (req, res) => {
console.log("body", req.body);
var tweetIDs = [];
var bounds = [];
getRequest('search/tweets', {q: req.body.tag, result_type:'recent', count:'100'})
.then((tweets) => {
tweets.statuses.map((status) => {
if(status.user.geo_enabled) {
console.log("ID: ", status.id);
tweetIDs.push(status.id);
}
});
return getRequest('statuses/show', tweetIDs); //I know tweetIDs is wrong, dont know what to do from here.
}).then((data) => {
console.log("User data for the ID")
console.log(data);
}).catch((error) => {
console.log(error)
});
res.send(bounds);
bounds.length = 0;
});
You nearly got it right! Map all tweets to Promises returned by your getRequest() function. Then, the Promise.all() method will return you a single Promise that resolves when all of the promises in your array resolves.
By the way, you can use Array.reduce instead of map to both filter and map your array at the same time, this factor your code a little better.
getRequest('search/tweets', {q: req.body.tag, result_type:'recent', count:'100'})
.then( tweets => {
let requests = tweets.statuses.reduce( (ret,status) => {
if(status.user.geo_enabled)
// The following is still wrong though, I guess. Format the params of getRequest accordingly.
ret.push( getRequest('statuses/show', status.id) );
return ret;
}, []);
return Promise.all(requests);
})
.then( results => {
results.forEach( res => {
console.log("User data for the ID");
console.log(res);
})
})
EDIT : Related jsfiddle
Replace the line
return getRequest('statuses/show', tweetIDs); //I know tweetIDs is wrong, dont know what to do from here.
with
return Promisel.all( tweetIDs.map( (tId) => getRequest('statuses/show', tId) );

Resources