I am having trouble with aggregation in Mongodb. I have a model User and UserExtra with extra details about the user. I need to do a $lookup to connect UserExtra to User output. But want to be able to filter results, based on age, gender and city, which are part of UserExtra. My current query is as follows, and it should return results but returs empty array.
const match = {
'userExtra.age': {
$gt: dateTo.toISOString(),
$lt: dateFrom.toISOString()
},
'userExtra.gender': gender
}
if (city) match['userExtra.city'] = city;
const users = await User.aggregate([
{
$lookup: {
from: 'user_extra',
localField: 'userExtra',
foreignField: '_id',
as: 'userExtra'
}
},
{
$match: match
},
{
$unwind: "$userExtra"
}
]);
res.send(users);
You have to unwind the lookup result first, then match pipeline.
let pipelineQuery = [
{
$lookup: {
from: 'user_extra',
localField: 'userExtra',
foreignField: '_id',
as: 'userExtra'
}
},
{
$unwind: "$userExtra"
},
{
$match: match
},
]
Hey guys I figured it out. I was using toIsoString() method, which turns the date into a string. All I did was remove the method and just passed the dateTo and dateFrom dates and now it works.
I would suggest testing each part of query one by one..First just run the lookup part in aggregate to see what the result looks like.After that run match.and subsequently add unwind.
Related
I am actually struggling with this from a long time.
I have a sql query
select * from posts where post_uploader='$user_id' or post_uploader in(select follow_user_id from follow where follow_follower_id='$user_id') order by updated_at
Can anyone tell me how to do the same thing with mongo db?
I tried to do this but I am getting error in LocalField. It requires string where I am passing object.
postModel.aggregate([{
$lookup: {
from: "follow",
localField: req.user.id,
foreignField: "follow_follower_id",
as: "follow"
}
}])
I guess foreignField's correct Field is "_id"
postModel.aggregate([
{
$lookup:
{
from: "follow",
localField: req.user.id,
foreignField: "_id",
as: "follow"
}
}])
but your question don't have enough information
I'm using a aggregate query to retrieve data from multiple collections, however there is a strange behavior that I dont seem to understand.
I need to lookup throw two collections, thus the lookup inside the pipeline. And also use the _id from the collection I'm making the aggregation(campaignadgroups) to match on the second nested collection (broadcastplans)
This is my query:
db.getCollection('campaignadgroups').aggregate([
{
$match: { "campaign_id": ObjectId("5fc8f7125148d7d0a19dcbcb")} // hardcoded just for tests
},
{
$lookup: {
from: "broadcastreports",
let: {campaignadgroupid: "$_id"},
pipeline: [
{
$match: {"reported_at": { $gte:ISODate("2020-12-01T15:56:58.743Z"), $lte: ISODate("2020-12-03T15:56:58.743Z")} }
},
{
$lookup: {
from: "broadcastplans",
localField: "broadcast_plan_id",
foreignField: "_id",
as: "broadcastplan"
}
},
{$unwind: "$broadcastplan"},
{
$match: { "broadcastplan.campaign_ad_group_id": {$eq: "$$campaignadgroupid"} // The problem happens here
}
}
],
as: "report"
}
},
])
The issue is that by matching with $$campaignadgroupid the report documents is empty.
However, if I replace the variable with the hardcoded id like ObjectId("5fc8f7275148d7d0a19dcbcc") I get the documents that I pretend.
For reference I'm debugging this issue on Robot3T so I can then translate to mongoose later.
I tried already to use $toObjectId however the _ids are not strings but ObjectIds already.
Thank you very much
Ok this is why I love and hate to code. After 3h debugging after asking here I immediately discovered the issue... I just needed to change from
$match: { "broadcastplan.campaign_ad_group_id": {$eq: "$$campaignadgroupid"}
to
$match: { $expr: { $eq: ["$broadcastplan.campaign_ad_group_id", "$$campaignadgroupid"]}
I have three collections in MongoDB.
In Drug JSON, using MEDID get its respective CMIDs from Composition JSON. Using that CMIDs get its respective chemical Names from the Chemicals JSON, then stores in one array with their medid, cmid and its name.
I'm using NodeJS. Please help me, thanks in advance.
DRUG collection
{
"MEDID":"AAA001-01",
"BRANDNAME":"TASULIN D",
"MEDTYPECODE":"CAP",
"DOSE":"",
"DOSEUNIT":"",
"CHEMICAL1":"TAMSULSIN HYDROCHLORIDE",
"CHEMICAL2":"DUTASTERIDE",
"CHEMICAL3":"",
"CHEMICAL4":"",
"CHEMICAL5":"",
"CHEMICAL6":"",
"CHEMICAL7":"",
"CHEMICAL8":"",
"CHEMICAL9":"",
"CHEMICAL10":"",
"CHEMICAL11":"",
"SPECIALITY":"GEN",
"MANUFACTURER":"IPCA",
"MFTID":"xyz123"
}
Compostion Collection
{"MEDID":"AAA001-01","CMID":"ABC001"},
{"MEDID":"AAA001-01","CMID":"ABC002"}
Chemical Collection
{"CMID":"ABC001","Name":"TAMSULSIN HYDROCHLORIDE"},
{"CMID":"ABC002","Name":"DUTASTERIDE"},
db.drug.aggregate([
{
$lookup:
{
from: "composition",
localField: "medid",
foreignField: "medid",
as: "composition"
}
},
$lookup:
{
from: "chemical",
localField: "composition.cmid",
foreignField: "cmid",
as: "chemical"
}
},
$group: {
_id: "$medid",
cmid: { $first: "$composition.cmid" },
name: { $first: "$chemical.name" }
]);
I have two collections. A 'users' collection and an 'events' collection. There is a primary key on the events collection which indicates which user the event belongs to.
I would like to count how many events a user has matching a certain condition.
Currently, I am performing this like:
db.users.find({ usersMatchingACondition }).forEach(user => {
const eventCount = db.events.find({
title: 'An event title that I want to find',
userId: user._id
}).count();
print(`This user has ${eventCount} events`);
});
Ideally what I would like returned is an array or object with the UserID and how many events that user has.
With 10,000 users - this is obviously producing 10,000 queries and I think it could be made a lot more efficient!
I presume this is easy with some kind of aggregate query - but I'm not familiar with the syntax and am struggling to wrap my head around it.
Any help would be greatly appreciated!
You need $lookup to get the data from events matched by user_id. Then you can use $filter to apply your event-level condition and to get a count you can use $size operator
db.users.aggregate([
{
$match: { //users matching condition }
},
{
$lookup:
{
from: 'events',
localField: '_id', //your "primary key"
foreignField: 'user_id',
as: 'user_events'
}
},
{
$addFields: {
user_events: {
$filter: {
input: "$user_events",
cond: {
$eq: [
'$$this.title', 'An event title that I want to find'
]
}
}
}
}
},
{
$project: {
_id: 1,
// other fields you want to retrieve: 1,
totalEvents: { $size: "$user_events" }
}
}
])
There isn't much optimization that can be done without aggregate but since you specifically said that
First, instead of
const eventCount = db.events.find({
title: 'An event title that I want to find',
userId: user._id
}).count();
Do
const eventCount = db.events.count({
title: 'An event title that I want to find',
userId: user._id
});
This will greatly speed up your queries because the find query actually fetches the documents first and then does the counting.
For returning an array you can just initialize an array at the start and push {userid: id, count: eventCount} objects to it.
In order to expand the JSON-API capabilities of my node.js application, I'm trying to sort a query based on relationships (AKA other documents), although I don't want to return them.
According to the JSON-API documentation:
a sort field of author.name could be used to request that the primary data be sorted based upon the name attribute of the author relationship.
E.g. db.collection('books').find({}) returns:
[
{
type: "book",
id: "2349",
attributes: {
title: "My Sweet Book"
},
relationships: {
author: {
data: {
type: "authors",
id: "9"
}
}
}
},
{} // etc ...
]
db.collection('authors').find({id: "9"}) returns:
[
{
type: "author",
id: "9",
attributes: {
name: "Hank Moody"
}
}
]
Now I need some way to do something similar to e.g.:
db.collection('books').find({}).sort({"author.name": -1})
I think I need to convert the query to an aggregation so I can use the $lookup operator, but I'm not sure how to use localField and foreignField.
db.collection('books').aggregate([
{$match: {}},
{$lookup: {from: "authors", localField: "attributes.author.data.id", foreignField: "id", as: "temp.author"}},
{$sort: {"$books.temp.author.name": -1}},
{$project: {temp: false}},
])
Notes
This will be a global function for fetching JSON-API data.
This means we don't know wether a sort key is an attribute or a relationship.
Most servers run LTS versions and have MongoDB 3.2
You can try below aggregation.
$lookup to join to authors collection followed by $unwind to flatten the book_author array for applying $sort on name field and $project with exclusion to remove book_author field ( only works starting Mongo 3.4 version ). For lower versions you have to include all the other fields you want to keep and excluding book_author field in the $project stage.
db.collection('books').aggregate([{
$lookup: {
from: "authors",
localField: "relationships.author.data.id",
foreignField: "id",
as: "book_author"
}
}, {
$unwind: "$book_author"
}, {
$sort: {
"book_author.attributes.name": -1
}
}, {
$project: {
"book_author": 0
}
}])