I'm currently writing a small API for a cooking app. I have a Recipe model and would like to implement sorting by columns based on the req Parameter given.
I'd like to sort by whatever is passed in the api call. the select parameter works perfectly fine, I can select the columns to be displayed but when I try to sort anything (let's say by rating) the return does sort but I'm not sure what it does sort by.
The code i'm using:
query = Recipe.find(JSON.parse(queryStr));
if(req.query.select){
const fields = req.query.select.split(',').join(' ');
query = query.select(fields);
}
if(req.query.sort){
const sortBy = req.query.sort.split(',').join(' ');
query = query.sort({ sortBy: 1 });
} else {
query = query.sort({ _id: -1 });
}
The result, when no sorting is set: https://pastebin.com/rPLv8n5s
vs. the result when I pass &sort=rating: https://pastebin.com/7eYwAvQf
also, when sorting my name the result is also mixed up.
You are not using the value of sortBy but the string "sortBy". You will need to create an object that has the rating as an object key.
You need the sorting object to look like this.
{
rating: 1
}
You can use something like this so it will be dynamic.
if(req.query.sort){
const sortByKey = req.query.sort.split(',').join(' ');
const sortByObj = {};
sortByObj[sortByKey] = 1; // <-- using sortBy as the key
query = query.sort(sortByObj);
} else {
query = query.sort({ _id: -1 });
}
I've a node.js api in which user sends the required fields as an array to be fetched from the mongodb database. I need to find the data of that fields using Find query. I've written forEach statement to loop through that array and got the array elements. But when I try to get the results by inserting the array elements in the query, it doesn't giving the required results. Could any one please help me in resolving the issue by seeing the code below?
templateLevelGraphData: async function(tid,payload){
let err, templateData, respData = [], test, currentValue;
[err,templateData] = await to(Template.findById(tid));
var templateId = templateData.templateId;
payload.variables.forEach(async data=>{
console.log(data); //data has the array elements like variables=["humidity"]
[err, currentValue] = await to(mongoose.connection.db.collection(templateId).find({},{data:1}).sort({"entryDayTime":-1}).limit(1).toArray());
console.log(currentValue);
});
return "success";
}
The expected output is,
[ { humidity: 36 } ]
But I'm getting only _id like,
[ { _id: 5dce3a2df89ab63ee4d95495 } ]
I think data is not applying in the query. But I'm printing the data in the console where it's giving the correct results by displaying the array elements like, humidity. What I need to do to make it work?
When you are passing {data: 1} you are passing an array where is expecting name of column.
You have to create an object where the keys are going to be the elements of the array and set them to 1.
const projection = data.reduce((a,b) => (a[b]=1, a), {});
[...] .find({}, projection) [...]
Actually I got the solution.
for(let i=0;i<payload.variables.length;i++){
var test = '{"'+ payload.variables[i] +'":1,"_id":0}';
var query = JSON.parse(test);
[err, currentValue] = await to(mongoose.connection.db.collection(templateId).find({"deviceId":deviceId},query).sort({"entryDayTime":-1}).limit(1).toArray());
console.log(currentValue); //It's giving the solution
}
I am using NodeJS, PostgreSQL and the amazing pg-promise library. In my case, I want to execute three main queries:
Insert one tweet in the table 'tweets'.
In case there is hashtags in the tweet, insert them into another table 'hashtags'
Them link both tweet and hashtag in a third table 'hashtagmap' (many to many relational table)
Here is a sample of the request's body (JSON):
{
"id":"12344444",
"created_at":"1999-01-08 04:05:06 -8:00",
"userid":"#postman",
"tweet":"This is the first test from postman!",
"coordinates":"",
"favorite_count":"0",
"retweet_count":"2",
"hashtags":{
"0":{
"name":"test",
"relevancetraffic":"f",
"relevancedisaster":"f"
},
"1":{
"name":"postman",
"relevancetraffic":"f",
"relevancedisaster":"f"
},
"2":{
"name":"bestApp",
"relevancetraffic":"f",
"relevancedisaster":"f"
}
}
All the fields above should be included in the table "tweets" besides hashtags, that in turn should be included in the table "hashtags".
Here is the code I am using based on Nested transactions from pg-promise docs inside a NodeJS module. I guess I need nested transactions because I need to know both tweet_id and hashtag_id in order to link them in the hashtagmap table.
// Columns
var tweetCols = ['id','created_at','userid','tweet','coordinates','favorite_count','retweet_count'];
var hashtagCols = ['name','relevancetraffic','relevancedisaster'];
//pgp Column Sets
var cs_tweets = new pgp.helpers.ColumnSet(tweetCols, {table: 'tweets'});
var cs_hashtags = new pgp.helpers.ColumnSet(hashtagCols, {table:'hashtags'});
return{
// Transactions
add: body =>
rep.tx(t => {
return t.one(pgp.helpers.insert(body,cs_tweets)+" ON CONFLICT(id) DO UPDATE SET coordinates = "+body.coordinates+" RETURNING id")
.then(tweet => {
var queries = [];
for(var i = 0; i < body.hashtags.length; i++){
queries.push(
t.tx(t1 => {
return t1.one(pgp.helpers.insert(body.hashtags[i],cs_hashtags) + "ON CONFLICT(name) DO UPDATE SET fool ='f' RETURNING id")
.then(hash =>{
t1.tx(t2 =>{
return t2.none("INSERT INTO hashtagmap(tweetid,hashtagid) VALUES("+tweet.id+","+hash.id+") ON CONFLICT DO NOTHING");
});
});
}));
}
return t.batch(queries);
});
})
}
The problem is with this code I am being able to successfully insert the tweet but nothing happens then. I cannot insert the hashtags nor link the hashtag to the tweets.
Sorry but I am new to coding so I guess I didn't understood how to properly return from the transaction and how to perform this simple task. Hope you can help me.
Thank you in advance.
Jean
Improving on Jean Phelippe's own answer:
// Columns
var tweetCols = ['id', 'created_at', 'userid', 'tweet', 'coordinates', 'favorite_count', 'retweet_count'];
var hashtagCols = ['name', 'relevancetraffic', 'relevancedisaster'];
//pgp Column Sets
var cs_tweets = new pgp.helpers.ColumnSet(tweetCols, {table: 'tweets'});
var cs_hashtags = new pgp.helpers.ColumnSet(hashtagCols, {table: 'hashtags'});
return {
/* Tweets */
// Add a new tweet and update the corresponding hash tags
add: body =>
db.tx(t => {
return t.one(pgp.helpers.insert(body, cs_tweets) + ' ON CONFLICT(id) DO UPDATE SET coordinates = ' + body.coordinates + ' RETURNING id')
.then(tweet => {
var queries = Object.keys(body.hashtags).map((_, idx) => {
return t.one(pgp.helpers.insert(body.hashtags[i], cs_hashtags) + 'ON CONFLICT(name) DO UPDATE SET fool = $1 RETURNING id', 'f')
.then(hash => {
return t.none('INSERT INTO hashtagmap(tweetid, hashtagid) VALUES($1, $2) ON CONFLICT DO NOTHING', [+tweet.id, +hash.id]);
});
});
return t.batch(queries);
});
})
.then(data => {
// transaction was committed;
// data = [null, null,...] as per t.none('INSERT INTO hashtagmap...
})
.catch(error => {
// transaction rolled back
})
},
NOTES:
As per my notes earlier, you must chain all queries, or else you will end up with loose promises
Stay away from nested transactions, unless you understand exactly how they work in PostgreSQL (read this, and specifically the Limitations section).
Avoid manual query formatting, it is not safe, always rely on the library's query formatting.
Unless you are passing the result of transaction somewhere else, you should at least provide the .catch handler.
P.S. For the syntax like +tweet.id, it is the same as parseInt(tweet.id), just shorter, in case those are strings ;)
For those who will face similar problem, I will post the answer.
Firstly, my mistakes:
In the for loop : body.hashtag.length doesn't exist because I am dealing with an object (very basic mistake here). Changed to Object.keys(body.hashtags).length
Why using so many transactions? Following the answer by vitaly-t in: Interdependent Transactions with pg-promise I removed the extra transactions. It's not yet clear for me how you can open one transaction and use the result of one query into another in the same transaction.
Here is the final code:
// Columns
var tweetCols = ['id','created_at','userid','tweet','coordinates','favorite_count','retweet_count'];
var hashtagCols = ['name','relevancetraffic','relevancedisaster'];
//pgp Column Sets
var cs_tweets = new pgp.helpers.ColumnSet(tweetCols, {table: 'tweets'});
var cs_hashtags = new pgp.helpers.ColumnSet(hashtagCols, {table:'hashtags'});
return {
/* Tweets */
// Add a new tweet and update the corresponding hashtags
add: body =>
rep.tx(t => {
return t.one(pgp.helpers.insert(body,cs_tweets)+" ON CONFLICT(id) DO UPDATE SET coordinates = "+body.coordinates+" RETURNING id")
.then(tweet => {
var queries = [];
for(var i = 0; i < Object.keys(body.hashtags).length; i++){
queries.push(
t.one(pgp.helpers.insert(body.hashtags[i],cs_hashtags) + "ON CONFLICT(name) DO UPDATE SET fool ='f' RETURNING id")
.then(hash =>{
t.none("INSERT INTO hashtagmap(tweetid,hashtagid) VALUES("+tweet.id+","+hash.id+") ON CONFLICT DO NOTHING");
})
);
}
return t.batch(queries);
});
}),
I have the following Object Structure:
[
{
name: "someThing"
,securities: [ {"id": "2241926"} ]
}
]
I want to be able to return all objects in the outer array, that has at least one child secuirty with an id that starts with a value. I have tried a few things and keep running up short. On the mongoo console, this query works:
db.caseSummary.find({"securities.id": /^224.*/i})
We are using ES6, so please apologies for the generator syntax:
const q = require("q");
const startsWith = function(term){
return new RegExp('^' + term + '.*', 'i')
}
const find = function*(collection, criteria){
const command = q.nbind(collection.find, collection),
myCriteria = criteria || {},
results = yield command(myCriteria),
toArray = q.nbind(results.toArray, results) ;
return yield toArray()
}
const searchBySecurity = function*(mongo, term) {
const collection = mongo.collection("caseSummary"),
criteria = {"securities.id": startsWith(term) };
return yield find(collection, criteria);
}
so searchBySecurity(this.mongo, '224') It returns an empty array, it should return the same results as the mongo console. I guess I am missing a pretty basic concept in translating my search criteria or invoking the find writing this in node from raw mongo console query.
Edit 1: to be clear I want to return all parent objects, which have subarrays that contain a value that starts with the term passed in...
Edit 2:
I changed the criteria to be:
criteria = { "securities": {
"$elemMatch": {
"id": "901774109" //common.startsWith(term)
}
}
};
Still the same results.
Edit 3:
Using nodejs - mongodb "version": "1.4.38"
Edit 4:
this ended up not being an issue
Query parsed from URL, example :
?year=2014&cat=sonny
Or it can be
?year=2014&id=223&something=high&cat=sonny
I could do
Model.find({year: 2014}).where('cat').equals('sonny')
But what if there a second example? How can I make it dynamic?
You can set the query to a variable and add multiple conditions:
var query = Model.find();
query.where('year').equals('2014');
query.where('cat').equals('sonny');
query.where('id').equals('223');
query.where('something').equals('high');
query.exec(callback);
For dynamic, just pass the query to a for loop and iterate through an array of your filter objects:
var query = Model.find();
var filters = [
{fieldName: "year", value: "2014"},
{fieldName: "cat", value: "sonny"}
...
];
for (var i = 0; i < filters.length; i++) {
query.where(filters[i].fieldName).equals(filters[i].value)
}
query.exec(callback);
Building on the cdbajorin's answer - I suspect many coders are trying to take input from a form and dynamically build a Mongoose filter from the end users input. (or at least that was my scenario).
If you 'name' the html input fields the same as your Mongoose Schema name
<input type='text' name='person.address'>
Then in your code you can use the req.body object
var query = Model.find();
for (var fieldName in req.body)
{
if(req.body.hasOwnProperty(fieldName)) //no inherited properties
{
if(req.body[fieldName]) //get rid of empty fields
{
query.where(fieldName).equals(req.body[fieldName]);
}
}
}
query.exec(function(err,data){console.log('QUERY EXECUTE : ' + err, data, data.length);});