Populating all items from ref model [duplicate] - node.js

This question already has answers here:
$lookup multiple levels without $unwind?
(1 answer)
Mongoose: deep population (populate a populated field)
(10 answers)
Closed 3 years ago.
I have a database structured like this:
I want to get all listItems, of all lists from a user with a x ID. What's the correct way of doing that?
I'm using node with mongoose, and I tried the following:
await User.findById(user._id).populate('list');
But realized that I can't populate all ListItem's from that. Meaning, I can't do this:
await User.findById(user._id).populate('list').populate('listItem');
How can I get all listItems, of all lists from a user with a x ID?

Assuming User, List, and ListItem are Collections, you should be able to accomplish this using $lookup..
Here is a live demo of the following query..
Query:
db.users.aggregate([
{
$match: {
uniqueId: 1
}
},
{
$lookup: {
from: "lists",
localField: "uniqueId",
foreignField: "userId",
as: "lists"
}
},
{
$lookup: {
from: "listItems",
localField: "uniqueId",
foreignField: "userId",
as: "listItems"
}
}
])
Dataset:
db={ // Simulates a DB ********
"users": [ // Simulates a Collection ********
{
"firstname": "John",
"lastname": "Smith",
"email": "jsmith#gmail.com",
"password": "password123",
"uniqueId": 1
},
{
"firstname": "Jane",
"lastname": "Doe",
"email": "doe#yahoo.com",
"password": "123password",
"uniqueId": 2
}
],
"lists": [ // Simulates a Collection ********
{
"userId": 1,
"name": "Johns List 1",
"items": [
11,
12,
13
]
},
{
"userId": 2,
"name": "Groceries",
"items": [
21,
22,
23
]
}
],
"listItems": [ // Simulates a Collection ********
{
"userId": 2,
"itemId": 21,
"title": "Apple",
"notes": []
},
{
"userId": 2,
"itemId": 22,
"title": "Banana",
"notes": []
},
{
"userId": 2,
"itemId": 23,
"title": "Strawberry",
"notes": []
},
{
"userId": 1,
"itemId": 11,
"title": "Oil Change",
"notes": []
},
{
"userId": 1,
"itemId": 12,
"title": "Pick Up Grandma",
"notes": []
},
{
"userId": 1,
"itemId": 13,
"title": "Go For Run",
"notes": []
}
]
}
Result:
[
{
"_id": ObjectId("5a934e000102030405000008"),
"email": "jsmith#gmail.com",
"firstname": "John",
"lastname": "Smith",
"listItems": [
{
"_id": ObjectId("5a934e000102030405000003"),
"itemId": 11,
"notes": [],
"title": "Oil Change",
"userId": 1
},
{
"_id": ObjectId("5a934e000102030405000004"),
"itemId": 12,
"notes": [],
"title": "Pick Up Grandma",
"userId": 1
},
{
"_id": ObjectId("5a934e000102030405000005"),
"itemId": 13,
"notes": [],
"title": "Go For Run",
"userId": 1
}
],
"lists": [
{
"_id": ObjectId("5a934e000102030405000006"),
"items": [
11,
12,
13
],
"name": "Johns List 1",
"userId": 1
}
],
"password": "password123",
"uniqueId": 1
}
]

Related

Nodejs mongoose : how to retreive data from db collection related to other collection

I want to implement a function that would process students, courses and instructors data (loaded from MongoDB collections) in order to produce information about students enrolled on courses (and store it in another MongoDB collection). the function reports each student’s:
primary key
their name
the number of courses they are enrolled in
I want to Store the output of this routine in a MongoDB collection called coursereport like this :
[
{
"_id": "jeff",
"value": {
"name": "Jeff Holland",
"numbercourses": 2
}
},
{
"_id": "john.shore",
"value": {
"name": "John Shore",
"numbercourses": 2
}
},
{
"_id": "lee2331",
"value": {
"name": "Lee Aldwell",
"numbercourses": 1
}
},
{
"_id": "rcotter",
"value": {
"name": "Ray Cotter",
"numbercourses": 2
}
},
{
"_id": "scott",
"value": {
"name": "Scott Mills",
"numbercourses": 3
}
}
]
my data :
{
"students": [
{
"_id": "john.shore",
"name": {"first": "John", "last": "Shore"},
"email": "john.shore#gmail.com",
"major": "Electrical Engineering"
},
{
"_id": "jeff",
"name": {"first": "Jeff", "last": "Holland"},
"email": "jeff#yahoo.com",
"major": "Business"
},
{
"_id": "scott",
"name": {"first": "Scott", "last": "Mills"},
"email": "scott#hotmail.com",
"major": "Humanities/Art"
},
{
"_id": "rcotter",
"name": {"first": "Ray", "last": "Cotter"},
"email": "rcotter#msn.com",
"major": "Computer Science"
},
{
"_id": "lee2331",
"name": {"first": "Lee", "last": "Aldwell"},
"email": "lee2331#aol.com",
"major": "Graphic Design"
}
],
"courses": [
{
"_id": "HIST-1010",
"name": "History of the World 1010",
"description": "A bunch of really interesting things that actually happened",
"students": ["scott", "john.shore"],
"ratings": [3, 5, 4, 5, 4, 4, 2, 4]
},
{
"_id": "ENGCOMP-1010",
"name": "English Composition 1010",
"description": "If you can't write well, you've got nothing!",
"students": ["scott", "lee2331", "rcotter", "john.shore", "jeff"],
"ratings": [4, 4, 5, 4, 5, 1, 5]
},
{
"_id": "ART-1050",
"name": "Artistic Interpretation 1050",
"description": "Discover your inner beholder",
"students": ["rcotter", "scott", "jeff"],
"ratings": [3, 4, 3, 4, 4, 3, 4, 4]
}
],
"instructors": [
{
"_id": "wally.r.binns",
"name": {"first": "Wally", "middle": "r", "last": "Binns"},
"email": "wally.r.binns#ssu.edu",
"bio": "I was born in the middle of my mother's doctoral dissertation on Faraday Cage isolation. I've been an academic ever since...",
"publications": [{
"title": "Inverted Celestial Poetry",
"source": "http://www.pubcentral.com/poetry/inverted-celestial-poetry"
}],
"courses": ["ENGLIT-2500"]
},
{
"_id": "gerald.waterford.iii",
"name": {"first": "Gerald", "last": "Waterford", "suffix": "III"},
"email": "gerald.waterford.iii#ssu.edu",
"bio": "My father's father was a great man. My father, not so much. I am restoring the family honor.",
"publications": [{
"title": "Grow, grow, little Dandelion",
"source": "http://www.hopefulstories.com/my-dandelion"
}, {"title": "The teapot and the spoon", "source": "http://www.dishsoap.com/teapot-spoon"}],
"courses": ["ENGCOMP-1010", "HIST-1010"]
},
{
"_id": "kim.b",
"name": {"prefix": "Mrs.", "first": "Kim", "last": "Binnley"},
"email": "kim.b#ssu.edu",
"bio": "My mother told me 'Don't let those dopes push you around'. My life has been a constant struggle against dopeness ever since. Sigh...",
"publications": [],
"courses": ["ART-1050"]
}
]
}
I started with that :
async function produceReport(db, callback) {
const students = db.collection('students');
const courses = db.collection('courses');
const instructors = db.collection('instructors');
// implement missing code that will create 'coursereport' mongo collection with student courses data
callback();
}
module.exports = produceReport;
can any one give a help please
You have to use $lookup
Defined as:
To perform an equality match between a field from the input documents with a field from the documents of the “joined” collection,
So you have to join students and courses and later the output value with instructors..
Take a look at this query:
db.students.aggregate([
{
$lookup: {
from: "courses",
localField: "item",
foreignField: "email",
as: "students_courses"
}
},
{
"$lookup": {
"from": "instructors",
"localField": "students_courses._id",
"foreignField": "courses",
"as": "students_courses"
}
},
{
"$set": {
"_id": "$_id",
"value": {
"name": {
"$concat": [
"$name.first",
" ",
"$name.last"
]
},
"numbercourses": {
"$size": "$students_courses"
}
}
}
},
{
"$project": {
"_id": 1,
"value": 1
}
}
])
First stage is a joing with two collections with the fields indicated and soterd into a field called students_courses.
Then, another join with the _ids from the students_courses.
With the data then you can create a field named value with the name and count the number of courses in other field.
And $project to show only fields you want.
Check an example here
db.students.aggregate([
{
$lookup: {
from: "courses",
localField: "_id",
foreignField: "students",
as: "students_courses"
}
},
{
$project: {
_id: 1,
value: {
name: {
$concat: [
"$name.first",
" ",
"$name.last"
]
},
numbercourses: {
"$size": "$students_courses"
}
}
}
}
])

How to filter results from collection the $lookup in mongoose

i want to filter the result as the following in mongodb. I use $lookup to populate the result from another collection. Please check my following code
This code below is what i get
{
"_id": "5f3d563122de0730d0f6a754",
"barcode": "1234",
"productname": "Lays Packet",
"brandName": "Lays",
"productSize": "12",
"price": "12",
"quant": "12",
"imageurl": "http://localhost:3000/images/imageurl-1597855281050.jpg",
"remaining": "12",
"creator": "3d943b957fb5db510d824c5cbd6e8f7d",
"__v": 0,
"source": [
{
"_id": "5f3a9bbc325a074240a1a815",
"firstname": "test",
"lastname": "test",
"storename": "test",
"gst": "test",
"phoneNumber": 1,
"email": "1#demo.com",
"address1": "test",
"address2": "test",
"city": "test",
"state": "test",
"country": "test",
"zip": "1",
"password": "1",
"usertype": 3,
"unique_SHOP": "3d943b957fb5db510d824c5cbd6e8f7d",
"__v": 0
}
]
},
How to retrieve only unique_SHOP and zip from source listing.I want result like the one below with one or more fields
{
"_id": "5f3d563122de0730d0f6a754",
"barcode": "1234",
"productname": "Lays Packet",
"brandName": "Lays",
"productSize": "12",
"price": "12",
"quant": "12",
"imageurl": "http://localhost:3000/images/imageurl-1597855281050.jpg",
"remaining": "12",
"creator": "3d943b957fb5db510d824c5cbd6e8f7d",
"__v": 0,
"source": [
{
"zip": "1",
"unique_SHOP": "3d943b957fb5db510d824c5cbd6e8f7d",
}
]
},
The query i use
List.aggregate([
{$match:
{ productname: { $regex: req.params.query,$options: "i" }}
},
{ $lookup:{
from: "suppliers",
localField: "creator",
foreignField: "unique_SHOP",
as: "source"
}
},
])
You can try $lookup with pipeline,
$match condition of creator id
$project to display required fields
{
$lookup: {
from: "suppliers",
as: "source",
let: { creator: "$creator" },
pipeline: [
{
$match: {
$expr: { $eq: ["$$creator", "$_id"] }
}
},
{
$project: {
_id: 0,
zip: 1,
unique_SHOP: 1
}
}
]
}
}
Playground

Flter mongodb database using mongoose nodejs

I need to filter some users according to some fixed criteria. I have a user collection and a talent collection. The talent collection holds the reference to a master category collection.
What I need is to filter these users according to the category in the talent collection and some keys from the user collection.
For example I need to search for a user whose gender is 'male' and education 'BTech' and will have talents as a programmer and tester
my user collection is like,
{
"_id": "5f1939239bd35429ac9cd78f",
"isOtpVerified": "false",
"role": "user",
"adminApproved": 1,
"status": 0,
"languages": "Malayalam, Tamil, Telugu, Kannada",
"name": "Test user",
"email": "test#email.com",
"phone": "1234567890",
"otp": "480623",
"uid": 100015,
"bio": "Short description from user",
"dob": "1951-09-07T00:00:00.000Z",
"gender": "Male",
"education": "Btech",
"bodyType": "",
"complexion": "",
"height": "",
"weight": "",
"requests": [],
"location": {
"place": "place",
"state": "state",
"country": "country"
},
"image": {
"avatar": "5f1939239bd35429ac9cd78f_avatar.jpeg",
"fullsize": "5f1939239bd35429ac9cd78f_fullsize.png",
"head_shot": "5f1939239bd35429ac9cd78f_head_shot.jpeg",
"left_profile": "5f1939239bd35429ac9cd78f_left_profile.png",
"right_profile": "5f1939239bd35429ac9cd78f_right_profile.png"
},
"__v": 42,
"createdAt": "2020-07-23T07:15:47.387Z",
"updatedAt": "2020-08-18T18:54:22.272Z",
}
Talent collection
[
{
"_id": "5f38efef179aca47a0089667",
"userId": "5f1939239bd35429ac9cd78f",
"level": "5",
"chars": {
"type": "Fresher",
},
"category": "5f19357b50bcf9158c6be572",
"media": [],
"createdAt": "2020-08-16T08:35:59.692Z",
"updatedAt": "2020-08-16T08:35:59.692Z",
"__v": 0
},
{
"_id": "5f3b7e6f7e322948ace30a2c",
"userId": "5f1939239bd35429ac9cd78f",
"level": "3",
"chars": {
"type": "Fresher",
},
"category": "5f19359250bcf9158c6be573",
"media": [
{
"adminApproved": 0,
"status": 0,
"_id": "5f3c22573065f84a48e04a14",
"file": "id=5f1939239bd35429ac9cd78f&dir=test&img=5f1939239bd35429ac9cd78f_image_undefined.jpeg",
"description": "test",
"fileType": "image",
"caption": "test file"
},
{
"adminApproved": 0,
"status": 0,
"_id": "5f3c2d7a8c7f8336b0bfced2",
"file": "id=5f1939239bd35429ac9cd78f&dir=test&img=5f1939239bd35429ac9cd78f_image_1.jpeg",
"description": "this is a demo poster for testing",
"fileType": "image",
"caption": "A Test Poster"
}
],
"createdAt": "2020-08-18T07:08:31.532Z",
"updatedAt": "2020-08-18T19:35:22.899Z",
"__v": 2
}
]
And the category in the above document is a separate one populated to this. the category collection as,
[
{
"_id": "5f19359250bcf9158c6be573",
"status": true,
"title": "Testing",
"description": "Application tester",
"code": "test",
"characteristics": [],
"createdAt": "2020-07-23T07:00:34.221Z",
"updatedAt": "2020-07-23T07:00:34.221Z",
"__v": 0
},
{
"status": true,
"_id": "5f29829a705b4e648c28bc88",
"title": "Designer",
"description": "UI UX Designer",
"code": "uiux",
"createdAt": "2020-08-04T15:45:30.125Z",
"updatedAt": "2020-08-04T15:45:30.125Z",
"__v": 0
},
{
"_id": "5f19357b50bcf9158c6be572",
"status": true,
"title": "programming",
"description": "Java programmer",
"code": "program",
"createdAt": "2020-07-23T07:00:11.137Z",
"updatedAt": "2020-07-23T07:00:11.137Z",
"__v": 0
}
]
So my filter terms will be;
{
categories: ["5f19359250bcf9158c6be573", "5f19357b50bcf9158c6be572"],
minAge: 18,
maxAge: 25,
minHeight: 5,
maxHeight: 6,
minWeight: 50,
maxWeight: 80,
complexion: "white",
gender: "male",
}
And the expected result will be a user have both the above talents and followed conditions,
{
users: { ..User details.. },
medias: { ...medias from the matching talents.. }
}
If there are two collections you need to join them either by primary key or _id with foriegn fields and you can use $lookup with $match to filter down.
Documentation
You need to use $lookup with pipeline,
$match you condition for category match
$lookup to join users collection
$match conditions for users collections fields
$match exclude documents that don't found matching users of criteria passed in conditions
db.talents.aggregate([
{
$match: {
category: { $in: ["5f19359250bcf9158c6be573", "5f19357b50bcf9158c6be572"] }
}
},
{
$lookup: {
from: "users",
as: "users",
let: { userId: "$userId" },
pipeline: [
{
$match: {
$expr: {
$and: [
{ $eq: ["$$userId", "$_id"] },
{ $eq: ["$gender", "Male"] },
{ $eq: ["$education", "Btech"] }
// ... add you other match criteria here
]
}
}
}
]
}
},
{ $match: { users: { $ne: [] } } }
])
Playground

Query to retrieve every subdocument alone without passing parent id using Mongoose

Here i like to explain my problem.
How can i write a mongoose query to retrieve every subdocument from JSON without passing parent_id.
[
{
"_id": "56a320003fe17cc7363dd0d7",
"name": "Leanna Jacobson",
"gender": "female",
"friends": [
{
"id": 0,
"name": "Riley Case"
},
{
"id": 1,
"name": "Herman Carter"
},
{
"id": 2,
"name": "Pacheco Woodard"
}
]
},
{
"_id": "56a3200001501cfa1ea2641d",
"name": "Juliana Bonner",
"gender": "female",
"friends": [
{
"id": 0,
"name": "Keller Woodward"
},
{
"id": 1,
"name": "Fern Knight"
},
{
"id": 2,
"name": "Cain Richards"
}
]
},
{
"_id": "56a3200006864c78ecb1aeed",
"name": "Gena Stark",
"gender": "female",
"friends": [
{
"id": 0,
"name": "Kate Franco"
},
{
"id": 1,
"name": "Araceli Mcclure"
},
{
"id": 2,
"name": "Molly Nelson"
}
]
},
{
"_id": "56a320006d868155161038b6",
"name": "Eve Gonzalez",
"gender": "female",
"friends": [
{
"id": 0,
"name": "Pam Lang"
},
{
"id": 1,
"name": "Christy Marks"
},
{
"id": 2,
"name": "Donovan Warren"
}
]
},
{
"_id": "56a3200066b94852f5680568",
"name": "Coleman Wooten",
"gender": "male",
"friends": [
{
"id": 0,
"name": "Roberta Olson"
},
{
"id": 1,
"name": "Roseann Reid"
},
{
"id": 2,
"name": "Kerri Russell"
}
]
}
]
Here i need to retrieve every friends details from the subdocument array friends for every parent.
so how can i write query for this?????
Suppose the name of your schema is Person, try this one.
//find all document, only select `friends` field from every document
Person.find({}, 'friends', function (err, friends) {
// the return friends is the [[friends], [friends], ...]
});

Mongoose, update of child will affect all parents?

What is the query to update below data with mongoose. So 3 fields are going to be updated. Top Parent Points, Categories Points and Tag Points.
{
"_id": "561fba5e7fac41a4055fad45",
"fullName": "Test",
"points": 45,
"level": 1,
"categories": [
{
"name": "Computer Science",
"points": 15,
"level": 1,
"_id": "561fba5e7fac41a4055fad46",
"tags": [
{
"name": "C#",
"points": 10,
"level": 1,
"_id": "561fba5e7fac41a4055fad47"
},
{
"name": "Java",
"points": 5,
"level": 1,
"_id": "561fba5e7fac41a4055ert12"
}
]
},
{
"name": "History",
"points": 30,
"level": 2,
"_id": "562407d4e3edf2113f61ac37",
"tags": [
{
"name": "WW2",
"points": 30,
"level": 2,
"_id": "56240797e3edf2113f61ac36"
}
]
}
]
}
to this one. When user gets a point from a specific tag, it will effect all parents. Let's say, user gets 10 points from C# then i have to update mongodb to this.
{
"_id": "561fba5e7fac41a4055fad45",
"fullName": "Test",
**"points": 55,**
"level": 1,
"categories": [
{
"name": "Computer Science",
**"points": 25,**
"level": 1,
"_id": "561fba5e7fac41a4055fad46",
"tags": [
{
"name": "c#",
**"points": 20,**
"level": 1,
"_id": "561fba5e7fac41a4055fad47"
},
{
"name": "Java",
"points": 5,
"level": 1,
"_id": "561fba5e7fac41a4055ert12"
}
]
},
{
"name": "History",
"points": 30,
"level": 2,
"_id": "562407d4e3edf2113f61ac37",
"tags": [
{
"name": "WW2",
"points": 30,
"level": 2,
"_id": "56240797e3edf2113f61ac36"
}
]
}
]
}
you should use $elemMatch for querying your object
db.tests.update({_id: yourTestId, categories: {$elemMatch: {_id: categoryId}}}, {$set: {$inc: {"categories.$.points": 10, points: 10}}})
So you querying only needed array element and update it values with $ reference

Resources