I have some dynamic query for a collection
var condition = dynmic_query
db.collection.find(condition)
is giving me an output
now what I need whatever I am getting from previous query need to lookup with an another collection.
I am scared if this is not possible yet in mongodb
[https://jira.mongodb.org/browse/SERVER-22497]
I did some google but not getting idea how to achieve this one
I have to execute this query in node.js
Please help
Thanks
It sounds like you need the Aggregation Framework with the $lookup pipeline stage with a $match pipeline. It's a bit like doing a JOIN from the SQL world. Here's an example:
db.orders.aggregate([
{
$lookup:
{
from: "warehouses",
let: { order_item: "$item", order_qty: "$ordered" },
pipeline: [
{ $match:
{ $expr:
{ $and:
[
{ $eq: [ "$stock_item", "$$order_item" ] },
{ $gte: [ "$instock", "$$order_qty" ] }
]
}
}
},
{ $project: { stock_item: 0, _id: 0 } }
],
as: "stockdata"
}
}
])
You can not use one query output in second query. But you need to handle this at your code level.
hit one query and kept result in one variable then use that result to search in another query.
Mongodb is not meant for complex fetch. That's the whole reason it gives you high performance.
Related
Imaging I have an array of objects, available before the aggregate query:
const groupBy = [
{
realm: 1,
latest_timestamp: 1318874398, //Date.now() values, usually different to each other
item_id: 1234, //always the same
},
{
realm: 2,
latest_timestamp: 1312467986, //actually it's $max timestamp field from the collection
item_id: 1234,
},
{
realm: ..., //there are many of them
latest_timestamp: ...,
item_id: 1234,
},
{
realm: 10,
latest_timestamp: 1318874398, //but sometimes then can be the same
item_id: 1234,
},
]
And collection (example set available on MongoPlayground) with the following schema:
{
realm: Number,
timestamp: Number,
item_id: Number,
field: Number, //any other useless fields in this case
}
My problem is, how to $group the values from the collection via the aggregation framework by using the already available set of data (from groupBy) ?
What have been tried already.
Okay, let skip crap ideas, like:
for (const element of groupBy) {
//array of `find` queries
}
My current working aggregation query is something like that:
//first stage
{
$match: {
"item": 1234
"realm" [1,2,3,4...,10]
}
},
{
$group: {
_id: {
realm: '$realm',
},
latest_timestamp: {
$max: '$timestamp',
},
data: {
$push: '$$ROOT',
},
},
},
{
$unwind: '$data',
},
{
$addFields: {
'data.latest_timestamp': {
$cond: {
if: {
$eq: ['$data.timestamp', '$latest_timestamp'],
},
then: '$latest_timestamp',
else: '$$REMOVE',
},
},
},
},
{
$replaceRoot: {
newRoot: '$data',
},
},
//At last, after this stages I can do useful job
but I found it a bit obsolete, and I already heard that using [.mapReduce][1] could solve my problem a bit faster, than this query. (But official docs doesn't sound promising about it) Does it true?
As for now, I am using 4 or 5 stages, before start working with useful (for me) documents.
Recent update:
I have checked the $facet stage and I found it curious for this certain case. Probably it will help me out.
For what it's worth:
After receiving documents after the necessary stages I am building a representative cluster chart, that you may also know as a heatmap
After that I was iterating each document (or array of objects) one-by-one to find their correct x and y coordinated in place which should be:
[
{
x: x (number, actual $price),
y: y (number, actual $realm),
value: price * quantity,
quantity: sum_of_quantity_on_price_level
}
]
As for now, it's old awful code with for...loop inside each other, but in the future, I will be using $facet => $bucket operators for that kind of job.
So, I have found an answer to my question in another, but relevant way.
I was thinking about using $facet operator and to be honest, it's still an option, but using it, as below is a bad practice.
//building $facet query before aggregation
const ObjectQuery = {}
for (const realm of realms) {
Object.assign(ObjectQuery, { `${realm.name}` : [ ... ] }
}
//mongoose query here
aggregation([{
$facet: ObjectQuery
},
...
])
So, I have chosen a $project stage and $switch operator to filter results, such as $groups do.
Also, using MapReduce could also solve this problem, but for some reason, the official Mongo docs recommends to avoid using it, and choose aggregation: $group and $merge operators instead.
Trying to wrap my head around aggregation features
and having trouble figuring out how can I filter out results
with no aggregated children?
lets say I have things:
{ _id: abc1, thingColor: "green" }
{ _id: abc2, thingColor: "red" }
{ _id: abc3, thingColor: "amazing" }
and I have birds:
{ _id: 1, thing_id: "abc1", type: "singing", isBiting: false }
{ _id: 2, thing_id: "abc1", type: "notFlying", isBiting: true }
{ _id: 3, thing_id: "abc3", type: "manEating", isBiting: false }
now I want to get a list of things, but only those that have at least one bird associated with them by id and only birds that bite.
So basically from this example I would like to get from things only:
{ _id: abc1, birds_id: "abc1" }
My query is like this - querying things:
{
$lookup:
{
from: 'birds',
let: { thingIdVar: '$_id'},
pipeline: [
{$match:
{$expr:
{$and: [
{$eq: ['$thing_id', '$$thingIdVar']},
{$eq: ['$isBiting', true]}
]}
}
}
],
as: 'birds'
}
},
this will return the things aggregated with the birds but it will get back all the things even if they don't have birds aggregated.
If I had 1 to 1 things to birds i could use {$unwind: '$birds'}
but each thing can have a lot of birds
At this point, Im doing the filtering programmatically but this messes up some other stuff (this example is a simplified version).
So I would prefer to get the results from mongo already filtered..
Is there a way to do so??
Thanks
Use a $match stage after the $lookup
{ "$match": { "birds": { "$ne": [] }}}
Since there is no barrier for the parent collection it returns all the documents. So to filter out the documents (things) which do not contain atleast one birds you have to use $match stage in the parent collection
I've looked high and low for this answer and nothing has worked. I have a pipeline query with a match term like this:
$match: {
$expr: {
$and: [
....
]
}
}
Inside the $and I have all sorts of conditions using $eq, $in, $ne, $gt, $lt, etc.
However try as I may I can't get it to recognize $nin or $exists. I'm trying to add a term where I search for a key not existing, eg:
{ $exists: [ '$key', 0 ] }
I keep getting
MongoError: Unrecognized expression '$exists'
and
MongoError: Unrecognized expression '$nin'
Can anyone help??
You can only use aggregation operators inside the $expr and the $nin and $exists are query operators not aggregation ones. Use the above conditions outside the $expr expression.
Something like
{ "$match": {
"key": { "$exists": true },
"$expr": {
"$and": [
{...}, {...}
]
}
}}
I'm trying to create a mongodb query using the filtered value in the filter. For example:
var myIdVariable = '1jig23h34r34r30h';
var myVisibleVariable = false;
var myDistanceVariable = 100;
db.getCollection.find({
'_id': myIdVariable,
'isVisible': myVisibleVariable,
'distanceRange': {$lte: {myDistanceVariable - distanceRange}}
})
So, I want filter the distanceRange from database based on the calculation of (myDistanceVariable - distanceRange), with the distanceRange given in the same query.
I don't know if I give you a clear explanation of my problem. It's possible?
Thanks you.
Use the $expr operator to build a query expression that allows you to compare fields from the same document as well as compare the distanceRange field with the calculation of the field itself and your variables.
You would need to use the logical $and query operator to include the other query expressions thus your final query would look like the following:
db.getCollection('collectionName').find({
'$expr': {
'$and': [
{ 'isVisible': myVisibleVariable },
{ '$lte': [
'$distanceRange', {
'$subtract': [
myDistanceVariable, '$distanceRange'
]
}
] }
]
}
})
If your MongoDB server doesn't support the $expr operator then go for the aggregation framework route with $redact
db.getCollection('collectionName').aggregate([
{ "$redact": {
"$cond": [
{
'$and': [
{ 'isVisible': myVisibleVariable },
{ '$lte': [
'$distanceRange', {
'$subtract': [
x, '$distanceRange'
]
}
] }
]
},
"$$KEEP",
"$$PRUNE"
]
} }
])
Note
Including the _id in the query expressions means you are narrowing down your selection to just a single document and the query may not return any results since it's looking for a specific document with that _id AND the same document should satisfy the other query expressions.
so in my MongoDB database I have a collection holding user posts.
Within that collection I have a set called "likes", which holds an array of the ids of the users that have liked that post. When querying I would like to pass a user id to my query and have a boolean in the result telling me whether the id exists in the array to see whether the user has already liked the post. I understand this would be easy to do with two queries, one to get the post and one to check if the user has liked it, but I would like to find the most efficient way to do this.
For example, one of my documents looks like this
{
_id: 24jef247jos991,
post: "Test Post",
likes: ["userid1", "userid2"]
}
When I query from "userid1" I would like the return
{
_id: 24jef247jos991,
post: "Test Post",
likes: ["userid1", "userid2"],
userLiked: true
}
But when I query from let's say "userid3" I would like
{
_id: 24jef247jos991,
post: "Test Post",
likes: ["userid1", "userid2"],
userLiked: false
}
You can add the $addFields stage checking each of the document likes arrays against the input user.
db.collection.aggregate( [
{
$addFields: {
"userLiked":{ $in: [ "userid1", "$likes" ] }
}
}
] )
Starting from MongoDB 3.4 you can use the $in aggregation operator to check if an array contains a given element. You can use the $addFields operator aggregation operator to add the newly computed value to your document without explicitly including other fields.
db.collection.aggregate( [
{ "$addFields": { "userLiked": { "$in": [ "userid1", "$likes" ] } } }
])
In MongoDB 3.2, you can use the $setIsSubset operator and the square bracket [] operator to do this. The downside of this approach is that you need to manually $project all the field in your document. Also the $setIsSubset operator with de-duplicate your array which may not be what you want.
db.collection.aggregate([
{ "$project": {
"post": 1, "likes": 1,
"userLiked": { "$setIsSubset": [ [ "userid3" ], "$likes" ] }
}}
])
Finally if your mongod version is 3.0 or older you need to use the $literal operator instead of the [] operator.