I'm trying to return all documents which satisfy document.field.value > document.array.length. I'm trying to do this using MongoClient in ExpressJS and was not able to find answers on SO.
Here is what I have tried but this gives an empty list.
const db = ref()
const c = db.collection(t)
const docs = await c.find({
"stop.time":{$gt: new Date().toISOString()},
"stop.expect": {$gt: { $size: "stop.riders"}}
})
console.log(docs)
Also tried replacing
"stop.expect": {$gt: { $size: "stop.riders"}}
with the below code which does not compile
$expr: {$lt: [{$size: "stop.riders"}, "stop.expect"]}
Sample data:
{ "stop": { "expect":3, "riders": ["asd", "erw", "wer"] } },
{ "stop": { "expect":4, "riders": ["asq", "frw", "wbr"] } }
To filter the query with a complex query involving the calculation, you need to use the $expr operator, which allows the aggregation operators.
Next, within the $expr operator, to refer to the field, you need to add the prefix $.
db.collection.find({
$expr: {
$lt: [
{
$size: "$stop.riders"
},
"$stop.expect"
]
}
})
Demo # Mongo Playground
To compare fields within a document to each other, you must use $expr. It would look like this:
const docs = await c.find({
$expr: { $gt: ["$stop.expect", { $size: "$stop.riders" }] },
});
I omitted the stop.time condition because it's not in your sample data and it's unclear what type of field it is, if it actually is in your data (whether it's a Date or String would be very important).
Note that this sort of query is unable to use an index.
Related
Ive looked here: https://codelikethis.com/lessons/db/mongodb-array-queries and here: https://www.mongodb.com/community/forums/t/what-is-the-best-way-to-query-an-array-of-sub-documents-in-mongodb/115315 to try and figure out how to query only matching subdocuments from a parent document.
Here is an example of the data:
{'testname':'process',
'jobId':"job1",
"vt_cond":"cond1",
"testData":[{
'chip':'c1',
'name':'block1'},
{
'chip':'c1',
'name':'block1'},
{
'chip':'c1',
'name':'block2'},
{
'chip':'c1',
'name':'block3'}]}
It contains subdocuments within the "testData" field. What I want to do is something like the following:
db.collection.find({'jobId':'job1', 'testData.name':'block3'})
and have it return the following:
{'testname':'process',
'jobId':"job1",
"vt_cond":"cond1",
"testData":[{'chip':'c1',
'name':'block3'}]}
I understand that the mongodb documentation states it will query and return a document which matches the conditions in the subdocument query conditions. It does. I get the entire example document as above. Is there anyway where I can make a similar query with similar conditions as above, and only return the parent node with the desired subdocument, instead of ALL the subdocuments?
Option 1: You can use $elemMatch in the project part as follow:
db.collection.find({
"jobId": "job1",
"testData.name": "block3"
},
{
"testData": {
"$elemMatch": {
name: "block3"
}
}
})
Explained:
Find the object and project only the array sub-object that match the criteria
Attention: This is only working if you need only 1x matching element from array , if there is more then one element in the array matching the criteria better use Option 2.
Playground
Option 2: aggregate/$filter ( cover the case when you have more then one objects in the array matching by the filter )
db.collection.aggregate([
{
$match: {
"jobId": "job1",
"testData.name": "block3"
}
},
{
"$addFields": {
"testData": {
"$filter": {
"input": "$testData",
"as": "t",
"cond": {
$eq: [
"$$t.name",
"block3"
]
}
}
}
}
}
])
Explained:
Match the document
Filter only the matching objects inside the array of objects
Playground 2
So I'm using NodeJS to query MongoDB (4.4). I'm trying to figure how to search for a field inside an object inside a document and retrieve the object (or at least the _id). The field I'm querying by is the created field within the transactions document. How the table looks like is bellow.
I tried:
const result = await Model.findOne({}, { created: createdDate });
Didn't work. Never worked with these kinds of DB and am a bit lost. Any help is appreciated.
Maybe something like this:
Option 1: ( Find )
db.collection.find({
"transactions.created": "2022-12-21"
},
{
transactions: {
$elemMatch: {
created: "2022-12-21"
}
}
})
Explained:
Find the document based on "transaction.created".
Project only the matching "transaction.created" object.
playground1
Option 2: (Aggregation)
db.collection.aggregate([
{
$match: {
"transactions.created": "2022-12-21"
}
},
{
$addFields: {
transactions: {
"$filter": {
"input": "$transactions",
"as": "t",
"cond": {
$eq: [
"$$t.created",
"2022-12-21"
]
}
}
}
}
}
])
Explained:
Find the document based on transaction.created
Filter only the matched transaction created object.
playground2
For best performance index need to be created on the "transaction.created" field.
If I have this schema...
person = {
name : String,
favoriteFoods : Array
}
... where the favoriteFoods array is populated with strings. How can I find all persons that have "sushi" as their favorite food using mongoose?
I was hoping for something along the lines of:
PersonModel.find({ favoriteFoods : { $contains : "sushi" }, function(...) {...});
(I know that there is no $contains in mongodb, just explaining what I was expecting to find before knowing the solution)
As favouriteFoods is a simple array of strings, you can just query that field directly:
PersonModel.find({ favouriteFoods: "sushi" }, ...); // favouriteFoods contains "sushi"
But I'd also recommend making the string array explicit in your schema:
person = {
name : String,
favouriteFoods : [String]
}
The relevant documentation can be found here: https://docs.mongodb.com/manual/tutorial/query-arrays/
There is no $contains operator in mongodb.
You can use the answer from JohnnyHK as that works. The closest analogy to contains that mongo has is $in, using this your query would look like:
PersonModel.find({ favouriteFoods: { "$in" : ["sushi"]} }, ...);
I feel like $all would be more appropriate in this situation. If you are looking for person that is into sushi you do :
PersonModel.find({ favoriteFood : { $all : ["sushi"] }, ...})
As you might want to filter more your search, like so :
PersonModel.find({ favoriteFood : { $all : ["sushi", "bananas"] }, ...})
$in is like OR and $all like AND. Check this : https://docs.mongodb.com/manual/reference/operator/query/all/
In case that the array contains objects for example if favouriteFoods is an array of objects of the following:
{
name: 'Sushi',
type: 'Japanese'
}
you can use the following query:
PersonModel.find({"favouriteFoods.name": "Sushi"});
In case you need to find documents which contain NULL elements inside an array of sub-documents, I've found this query which works pretty well:
db.collection.find({"keyWithArray":{$elemMatch:{"$in":[null], "$exists":true}}})
This query is taken from this post: MongoDb query array with null values
It was a great find and it works much better than my own initial and wrong version (which turned out to work fine only for arrays with one element):
.find({
'MyArrayOfSubDocuments': { $not: { $size: 0 } },
'MyArrayOfSubDocuments._id': { $exists: false }
})
Incase of lookup_food_array is array.
match_stage["favoriteFoods"] = {'$elemMatch': {'$in': lookup_food_array}}
Incase of lookup_food_array is string.
match_stage["favoriteFoods"] = {'$elemMatch': lookup_food_string}
Though agree with find() is most effective in your usecase. Still there is $match of aggregation framework, to ease the query of a big number of entries and generate a low number of results that hold value to you especially for grouping and creating new files.
PersonModel.aggregate([
{
"$match": {
$and : [{ 'favouriteFoods' : { $exists: true, $in: [ 'sushi']}}, ........ ] }
},
{ $project : {"_id": 0, "name" : 1} }
]);
There are some ways to achieve this. First one is by $elemMatch operator:
const docs = await Documents.find({category: { $elemMatch: {$eq: 'yourCategory'} }});
// you may need to convert 'yourCategory' to ObjectId
Second one is by $in or $all operators:
const docs = await Documents.find({category: { $in: [yourCategory] }});
or
const docs = await Documents.find({category: { $all: [yourCategory] }});
// you can give more categories with these two approaches
//and again you may need to convert yourCategory to ObjectId
$in is like OR and $all like AND. For further details check this link : https://docs.mongodb.com/manual/reference/operator/query/all/
Third one is by aggregate() function:
const docs = await Documents.aggregate([
{ $unwind: '$category' },
{ $match: { 'category': mongoose.Types.ObjectId(yourCategory) } }
]};
with aggregate() you get only one category id in your category array.
I get this code snippets from my projects where I had to find docs with specific category/categories, so you can easily customize it according to your needs.
For Loopback3 all the examples given did not work for me, or as fast as using REST API anyway. But it helped me to figure out the exact answer I needed.
{"where":{"arrayAttribute":{ "all" :[String]}}}
In case You are searching in an Array of objects, you can use $elemMatch. For example:
PersonModel.find({ favoriteFoods : { $elemMatch: { name: "sushiOrAnytthing" }}});
With populate & $in this code will be useful.
ServiceCategory.find().populate({
path: "services",
match: { zipCodes: {$in: "10400"}},
populate: [
{
path: "offers",
},
],
});
If you'd want to use something like a "contains" operator through javascript, you can always use a Regular expression for that...
eg.
Say you want to retrieve a customer having "Bartolomew" as name
async function getBartolomew() {
const custStartWith_Bart = await Customers.find({name: /^Bart/ }); // Starts with Bart
const custEndWith_lomew = await Customers.find({name: /lomew$/ }); // Ends with lomew
const custContains_rtol = await Customers.find({name: /.*rtol.*/ }); // Contains rtol
console.log(custStartWith_Bart);
console.log(custEndWith_lomew);
console.log(custContains_rtol);
}
I know this topic is old, but for future people who could wonder the same question, another incredibly inefficient solution could be to do:
PersonModel.find({$where : 'this.favouriteFoods.indexOf("sushi") != -1'});
This avoids all optimisations by MongoDB so do not use in production code.
{
"_id" : ObjectId("5852725660632d916c8b9a38"),
"response_log" : [
{
"campaignId" : "AA",
"created_at" : ISODate("2016-12-20T11:53:55.727Z")
},
{
"campaignId" : "AB",
"created_at" : ISODate("2016-12-20T11:55:55.727Z")
}]
}
I have a document which contains an array. I want to select all those documents that do not have response_log.created_at in last 2 hours from current time and count of response_log.created_at in last 24 is less than 3.
I am unable to figure out how to go about it. Please help
You can use the aggregation framework to filter the documents. A pipeline with $match and $redact steps will do the filtering.
Consider running the following aggregate operation where $redact allows you to proccess the logical condition with the $cond operator and uses the system variables $$KEEP to "keep" the document where the logical condition is true or $$PRUNE to "remove" the document where the condition was false.
This operation is similar to having a $project pipeline that selects the fields in the collection and creates a new field that holds the result from the logical condition query and then a subsequent $match, except that $redact uses a single pipeline stage which is more efficient:
var moment = require('moment'),
last2hours = moment().subtract(2, 'hours').toDate(),
last24hours = moment().subtract(24, 'hours').toDate();
MongoClient.connect(config.database)
.then(function(db) {
return db.collection('MyCollection')
})
.then(function (collection) {
return collection.aggregate([
{ '$match': { 'response_log.created_at': { '$gt': last2hours } } },
{
'$redact': {
'$cond': [
{
'$lt': [
{
'$size': {
'$filter': {
'input': '$response_log',
'as': 'res',
'cond': {
'$lt': [
'$$res.created_at',
last24hours
]
}
}
}
},
3
]
},
'$$KEEP',
'$$PRUNE'
]
}
}
]).toArray();
})
.then(function(docs) {
console.log(docs)
})
.catch(function(err) {
throw err;
});
Explanations
In the above aggregate operation, if you execute the first $match pipeline step
collection.aggregate([
{ '$match': { 'response_log.created_at': { '$gt': last2hours } } }
])
The documents returned will be the ones that do not have "response_log.created_at" in last 2 hours from current time where the variable last2hours is created with the momentjs library using the subtract API.
The preceding pipeline with $redact will then further filter the documents from the above by using the $cond ternary operator that evaluates this logical expression that uses $size to get the count and $filter to return a filtered array with elements that match other logical condition
{
'$lt': [
{
'$size': {
'$filter': {
'input': '$response_log',
'as': 'res',
'cond': { '$lt': ['$$res.created_at', last24hours] }
}
}
},
3
]
}
to $$KEEP the document if this condition is true or $$PRUNE to "remove" the document where the evaluated condition is false.
I know that this is probably not the answer that you're looking for but this may not be the best use case for Mongo. It's easy to do that in a relational database, it's easy to do that in a database that supports map/reduce but it will not be straightforward in Mongo.
If your data looked different and you kept each log entry as a separate document that references the object (with id 5852725660632d916c8b9a38 in this case) instead of being a part of it, then you could make a simple query for the latest log entry that has that id. This is what I would do in your case if I ware to use Mongo for that (which I wouldn't).
What you can also do is keep a separate collection in Mongo, or add a new property to the object that you have here which would store the latest date of campaign added. Then it would be very easy to search for what you need.
When you are working with a database like Mongo then how your data looks like must reflect what you need to do with it, like in this case. Adding a last campaign date and updating it on every campaign added would let you search for those campaign that you need very easily.
If you want to be able to make any searches and aggregates possible then you may be better off using a relational database.
I have this collection :
[
{
_id: ObjectId('myId1'),
probes: ['id_probe_1', 'id_probe_2']
},
{
_id: ObjectId('myId2'),
probes: ['id_probe_1', 'id_probe_3']
}
]
I want to get an array like this :
['id_probe_1', 'id_probe_2', 'id_probe_3']
So I try this request (from nodeJS driver) :
let find = [
{
$match: {
_id: {
$in: [new ObjectId('miId1'), new ObjectId('myId2')]
}
}
},
{
$group: {
_id: null,
probes: {
$addToSet: {
$each: '$probes'
}
}
}
}
];
This doesn't work, give me this error :
invalid operator '$each'
From the doc, they mention that it will appends the whole array as a single element.
If the value of the expression is an array, $addToSet appends the whole array as a single element.
But they don't say how to have an unique array. So I use the $each operator like this page indicates (I don't really know what's the difference...)
Is there a way to make this work ?
Thanks !
insert $unwind before $group
{$unwind:"$probes"},
then remove $each
Why don't you try distinct operation? In mongo shell, db.col.distinct('probs'); you can try the distinct function in nodejs mongo driver.