I have this string:
var filter = '{stamps:{$gte: 2020-11-06 06:42:25.000+01:00, $lte: 2020-11-06 09:52:25.000+01:00}'
Somehow I have to query mongoDB collection with this string.
Message.find(filter, function(err, messages){
if(err){
console.log(err);
}else{
//do something here
}
})
Of course I am getting this error:
ObjectParameterError: Parameter "filter" to find() must be an object, got {stamps:{$gte: 2020-11-06 06:42:25.000+01:00, $lte: 2020-11-06 09:52:25.000+01:00}
is there some way to cast this string to object that will be acceptable to mongoDB?
You could either change the way you construct your query, to pass it as JSON an then use JSON.parse() / JSON.stringify()
// Note the necessary double quotes around the keys
const jsonFilter = '{ "stamps": { "$gte": "2020-11-06 06:42:25.000+01:00", "$lte": "2020-11-06 09:52:25.000+01:00" } }'
const query = JSON.parse(jsonFilter)
Message.find(query, callback)
Or you could introduce another dependency like mongodb-query-parser that can parse the string for you.
const parser = require('mongodb-query-parser')
// Note it still needs double quotes around the dates to run properly
const filter = '{stamps:{$gte: "2020-11-06 06:42:25.000+01:00", $lte: "2020-11-06 09:52:25.000+01:00" } }'
const query = parser(filter)
Message.find(query, callback)
https://runkit.com/5fbd473b95d0a9001a2359b3/5fbd473b98492c001a8bba06
If you can't include the quotes around the values in the string, you can use a regular expression to add them. But keep in mind that this regex is tailored to match this exact date format. If the format can change or you want to also match other data types, you should try to include them in your filter string from the start.
const parser = require("mongodb-query-parser")
const filter = '{stamps:{$gte: 2020-11-06 06:42:25.000+01:00, $lte: 2020-11-06 09:52:25.000+01:00} }'
const filterWithQuotedDates = filter.replace(/(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{3}(\+\d{2}:\d{2})?)/g, '"$1"')
parser(filterWithQuotedDates)
Message.find(query, callback)
https://runkit.com/5fbd494bcd812c0019b491fb/5fbd4951ebe43f001a5b590a
It should be noted that a common use-case is to pass a MongoDB query via URL params and that there are special packages for that:
https://github.com/Turistforeningen/node-mongo-querystring
https://github.com/fox1t/qs-to-mongo
Related
In nodejs I'm trying to run an $or statement to retrieve objects from mongo that could have certain IDs. I use a for loop to push the ids into a string. My code looks like this after the loop.
var filter = "{ $or: [{'id':1290383},{'id':1290381},{'id':1290382}]}"
const results = await collection.find(filter).toArray();
When I run this it returns everything in the collection. I'm assuming I need to JSON parse it beforehand but I get an error about the $ when I try to parse it. I can't find any information about the correct way to use $or in nodejs. I feel like there should be another way to query different IDs. If someone knows, please let me know.
You need to convert the filter variable from string to json object as follow:
var filter = '{ "$or": [{"id":1290383},{"id":1290381},{"id":1290382}]}'
var filterobj = JSON.parse(filter);
const results = await collection.find(filterobj).toArray();
Also JSON.parse() does not allow single quotes so you will need to enquote the full string with single quotes and the keys with the double quotes.
You could also replace the $or operation with $in operation for the same task (since it is generally easier to read and pretend to be more effective) as follow:
var filter = '{ "id":{"$in": [1290383,1290381,1290382] } }'
example from mongo shell:
mongos> var filter = '{ "id":{"$in": [1290383,1290381,1290382] } }'
mongos> var filterobj = JSON.parse(filter);
mongos> filterobj
{ "id" : { "$in" : [ 1290383, 1290381, 1290382 ] } }
mongos>
Let's say I have a collection of books that contains multiple documents that looks like this:
{
"author":"Gud Author",
"title":"Gud Book"
}
{
"author":"Lazy Writer",
"title":"Lesson about lazyness"
}
{
"author":"Mysterious Enigma",
"title":"Ways of the unknown"
}
In the request I'm getting one query parameter (string) called "filter". I want to use this parameter to get only the books whose author or title matches at least a part of the filter string.
This is my code so far:
bookRouter.get("/books", (req,res) => {
const {filter} = req.query
let query = {}
if (filter) query.$or = [{author: {$regex: filter}}, {title: {$regex: filter}}]
Book.find(query)
.then((books) => res.json(books))
.catch((err) => res.json(err))
})
If the filter parameter is shorter than the values in my db everything works ok.
"" as filter would return everything,
"Gud" would only return the first object
"Mysterious" would only return the third object"
However when my query is " Mysterious Enigma Ways" I'm getting back an empty array. Also a combination of both title and author that is only a partial match with both db values like "Mysterious Ways" won't work.
How do i modify my query to also return documents that only match with a part of the query string?
I've tried turning the string into an array (separating words) using .join(" ") to compare matches to the single elements of the array but I couldn't figure out how to combine the operators for this query.
PS: Yes I will convert both query string and values to lowercase for case-insensitive comparison. Just wanted to keep this short.
Instead of $regex, try using plain Javascript regex, by splitting the input words and checking if any of the words are present using $in,
let splitWords = filter.split(" ");
let regexWords = splitWords.map(e =>{return new RegExp(e,i)}
if (filter) query.$or = [{author: {$in: regexWords}}, {title: {{$in: regexWords}}]
I'm currently writing a small API for a cooking app. I have a Recipe model and would like to implement sorting by columns based on the req Parameter given.
I'd like to sort by whatever is passed in the api call. the select parameter works perfectly fine, I can select the columns to be displayed but when I try to sort anything (let's say by rating) the return does sort but I'm not sure what it does sort by.
The code i'm using:
query = Recipe.find(JSON.parse(queryStr));
if(req.query.select){
const fields = req.query.select.split(',').join(' ');
query = query.select(fields);
}
if(req.query.sort){
const sortBy = req.query.sort.split(',').join(' ');
query = query.sort({ sortBy: 1 });
} else {
query = query.sort({ _id: -1 });
}
The result, when no sorting is set: https://pastebin.com/rPLv8n5s
vs. the result when I pass &sort=rating: https://pastebin.com/7eYwAvQf
also, when sorting my name the result is also mixed up.
You are not using the value of sortBy but the string "sortBy". You will need to create an object that has the rating as an object key.
You need the sorting object to look like this.
{
rating: 1
}
You can use something like this so it will be dynamic.
if(req.query.sort){
const sortByKey = req.query.sort.split(',').join(' ');
const sortByObj = {};
sortByObj[sortByKey] = 1; // <-- using sortBy as the key
query = query.sort(sortByObj);
} else {
query = query.sort({ _id: -1 });
}
I am trying to convert a string for use in mongodb but fails.
let pipeline = JSON.parse('[{"$match": {"_id": ObjectId("5b5637acbd3e9c2068ef80c3")}]');
// results in "SyntaxError: Unexpected token O in JSON at position 20"s
let pipeline = JSON.parse('[{"$match": {"_id": "5b5637acbd3e9c2068ef80c3"}]');
let response = await db.collect('<collection_name>').aggregate(pipeline).toArray();
// returns [] parse works but mongodb doesn't return any rows!
// This works but its not the solution I am looking for.
let pipeline = [{"$match": {"_id": ObjectId("5b5637acbd3e9c2068ef80c3")}];
let response = await db.collect('<collection_name>').aggregate(pipeline).toArray();
I tried using the BSON type but had no luck.
My current work around is to remove the ObjectId() from the string and use a Reviver function with JSON.parse
const ObjectId = require('mongodb').ObjectID;
let convertObjectId = function (key,value){
if (typeof value === 'string' && value.match(/^[0-9a-fA-F]{24}$/)){
return ObjectId(value);
} else {
return value;
};
}
let pipeline = JSON.parse('[{"$match": {"_id": "5b5637acbd3e9c2068ef80c3"}]',convertObjectId);
let response = await db.collect('<collection_name>').aggregate(pipeline).toArray();
// returns one record.
Unfortunately, [{"$match": {"_id": ObjectId("5b5637acbd3e9c2068ef80c3")}] is not valid JSON.
The value of a property in JSON can only be an object (ex.: {}), an array (ex.: []), a string (ex.: "abc"), a number (ex.: 1), a boolean (ex.: true), or null. See an example of these values here: https://en.wikipedia.org/wiki/JSON#Example.
What you could do is add ObjectId() manually after parsing the JSON. This would mean that the value of _id would be a string first, which is valid JSON.
Then, you can loop through your parsed JSON to add ObjectId (see reference here: https://mongodb.github.io/node-mongodb-native/api-bson-generated/objectid.html):
const ObjectId = require('mongodb').ObjectID;
const pipeline = JSON.parse('[{"$match": {"_id": "5b5637acbd3e9c2068ef80c3"}]');
const pipelineWithObjectId = pipeline.map(query => ({
$match: {
...query.$match,
_id: ObjectId(query.$match._id)
}
});
const response = await db.collect('<collection_name>').aggregate(pipelineWithObjectId).toArray();
This should work with the example you provided but there are multiple caveats:
Parsing a query like that could be a vulnerability if the string contains user input that has not been sanitized: https://blog.websecurify.com/2014/08/hacking-nodejs-and-mongodb.html.
This particular code snippet would only work for queries with $match, which means that this code is not very scalable.
This code is not elegant.
All these reasons, for what they are worth, make me think that you would be better off using an object rather than a string for your queries.
I am creating a rest API for my company for reporting
Routs to pass dates to find data from 2 given dates
router.get('/smDcr/:currentDate?&:lastDate:', authorize, getDcrBetweenDates);
Controller action to get the data between dates
exports.getDcrBetweenDates = async(req, res, next) => {
try{
const lastDate = req.params.lastDate;
const currentDate = req.params.currentDate;
console.log(lastDate);
const dcr = await Dcr.find({
where: {
createdAt: {
$between:[currentDate, lastDate]
}
}
});
if(!dcr) return res.status(400).json({message: "No DCR found between mentioned Dates!!"});
return res.status(200).json({ dcr});
}catch(ex){
next(ex)
}
}
while passing parameter in postman I am getting all reports not specific to the dates given in params
http://localhost:3000/smDcr/?currentDate=2019/09/11&?lastDate=2019/09/11
You don't need to use $where condition, you just can set your request object inside find function
I think you should use $gte and $lte operators, instead of $between (cf #ponzao answer)
Here is an example :
const dcr = await Dcr.find({
"createdAt": {
"$gte": new Date(currentDate),
"$lt": new Date(lastDate)
}
});
EDIT:
If it still doesn't works, it is probably because your dates are not formatted, maybe you should use a date library, like moment, and try like this :
const moment = require('moment');
const dcr = await Dcr.find({
"createdAt": {
"$gte": new Date(moment(currentDate, 'YYYY/MM/DD').format()),
"$lt": new Date(moment(lastDate, 'YYYY/MM/DD').format())
}
});
Hope it helps.
It will be better if you use query strings instead of query params for atleast dates. In the route you provided that is :
router.get('/smDcr/:currentDate?&:lastDate:', authorize, getDcrBetweenDates);
using ?&: in url to pass 2nd date value will get interpreted as querystring because of (?).
And you are accessing this end point by calling this :
http://localhost:3000/smDcr/?currentDate=2019/09/11&?lastDate=2019/09/11
In this, currentDate will be considered as query string and same is the case with lastDate
So I would suggest you to use this instead :
router.get('/smDcr', authorize, getDcrBetweenDates);
And in your controller, access the values like this :
const lastDate = req.query.lastDate;
const currentDate = req.query.currentDate;
And to access it you should be calling :
http://localhost:3000/smDcr?currentDate=2019/09/11&lastDate=2019/09/11
Here is a official doc for routing.
Express Routing