MongoDB - Populate document field recursively - node.js

I've got a Page:
const PageSchema = new Schema({
children: [{type: mongoose.Schema.Types.ObjectId, ref: 'Page'}]
});
As you can see each Page has an array of children which is also a Page.
Now what I need to do is fetch all "main" pages and populate their children array, easy enough except for the fact that I need to do this recursively since a Page contains an array of Pages.
MongoDB doesn't have any out of the box support for this, it only supports a 2 level deep population.
Here's my current query (removed all extra stuff for readability) without using the current .populate method (since it's not gonna work anyway):
Page.find(query)
.exec((err, pages) => {
if (err) {
return next(err);
}
res.json(pages);
});
I looked at this question which is similar but not exactly what I need:
mongoose recursive populate
That seems to use a parent to populate recursively and it also starts from just 1 document, rather than my scenario which uses an array of documents since I'm using .find and not .findOne for example.
How can I create my own deep recursive populate function for this?
Sidenote:
I am aware that the solution I need isn't recommended due to performance but I've come to the conclusion that it is the only solution that is going to work for me. I need to do recursive fetching regardless if it's in the frontend or backend, and doing it right in the backend will simplify things massively. Also the number of pages won't be big enough to cause performance issues.

You can recursively populate a field also like:
User.findOne({ name: 'Joe' })
.populate({
path: 'blogPosts',
populate: {
path: 'comments',
model: 'comment',
populate: {
path: 'user',
model: 'user'
}
}
})
.then((user) => {});
Please note that for first population, you don't need to specify model attribute, as it is already defined in your model's schema, but for next nested populations, you need to do that.

The answer actually lied in one of the answers from the previous questions, although a bit vague. Here's what I ended up with and it works really well:
Page.find(query)
.or({label: new RegExp(config.query, 'i')})
.sort(config.sortBy)
.limit(config.limit)
.skip(config.offset)
.exec((err, pages) => {
if (err) {
return next(err);
}
// takes a collection and a document id and returns this document fully nested with its children
const populateChildren = (coll, id) => {
return coll.findOne({_id: id})
.then((page) => {
if (!page.children || !page.children.length) {
return page;
}
return Promise.all(page.children.map(childId => populateChildren(coll, childId)))
.then(children => Object.assign(page, {children}))
});
}
Promise.all(pages.map((page) => {
return populateChildren(Page, page._id);
})).then((pages) => {
res.json({
error: null,
data: pages,
total: total,
results: pages.length
});
});
});
The function itself should be refactored into a utils function that can be used anywhere and also it should be a bit more general so it can be used for other deep populations as well.
I hope this helps someone else in the future :)

Related

Mongoose display comments and stars(likes) for each post [duplicate]

In Mongoose, I can use a query populate to populate additional fields after a query. I can also populate multiple paths, such as
Person.find({})
.populate('books movie', 'title pages director')
.exec()
However, this would generate a lookup on book gathering the fields for title, pages and director - and also a lookup on movie gathering the fields for title, pages and director as well. What I want is to get title and pages from books only, and director from movie. I could do something like this:
Person.find({})
.populate('books', 'title pages')
.populate('movie', 'director')
.exec()
which gives me the expected result and queries.
But is there any way to have the behavior of the second snippet using a similar "single line" syntax like the first snippet? The reason for that, is that I want to programmatically determine the arguments for the populate function and feed it in. I cannot do that for multiple populate calls.
After looking into the sourcecode of mongoose, I solved this with:
var populateQuery = [{path:'books', select:'title pages'}, {path:'movie', select:'director'}];
Person.find({})
.populate(populateQuery)
.execPopulate()
you can also do something like below:
{path:'user',select:['key1','key2']}
You achieve that by simply passing object or array of objects to populate() method.
const query = [
{
path:'books',
select:'title pages'
},
{
path:'movie',
select:'director'
}
];
const result = await Person.find().populate(query).lean();
Consider that lean() method is optional, it just returns raw json rather than mongoose object and makes code execution a little bit faster! Don't forget to make your function (callback) async!
This is how it's done based on the Mongoose JS documentation http://mongoosejs.com/docs/populate.html
Let's say you have a BookCollection schema which contains users and books
In order to perform a query and get all the BookCollections with its related users and books you would do this
models.BookCollection
.find({})
.populate('user')
.populate('books')
.lean()
.exec(function (err, bookcollection) {
if (err) return console.error(err);
try {
mongoose.connection.close();
res.render('viewbookcollection', { content: bookcollection});
} catch (e) {
console.log("errror getting bookcollection"+e);
}
//Your Schema must include path
let createdData =Person.create(dataYouWant)
await createdData.populate([{path:'books', select:'title pages'},{path:'movie', select:'director'}])

Passing Route Parameter to Mongoose Query Express

Basically I'm trying to find a way in which to find a way to plug req.params.name into a find() query.
I have tried:
Trying to pass through my req.params variable in my find() object parameter Card.find({cardName: req.params.name}, callback) and any other possible variance of that.
I've tried a static method for findByName in which I just did Card.findByName(req.params.name, callback);
When I do console.lo(req.params.name) it returns the name of the card; however, when I got to show.ejs I do console.log(cardstatsok.cardName) it comes back undefined.
I've searched here on Stack Overflow, I've checked my Udemy message board, and I've tried finding any tutorial on passing route parameters to a find query, and alas, nothing.
Here's my code:
My schema and model:
var cardSchema = new mongoose.Schema({
cardName: String,
id: Number,
cardSet: String,
image: String,
ability: String
});
var Card = mongoose.model("Card", cardSchema);
My route for my single card page:
app.get("/cards/:name", function(req, res) {
Card.find({"cardName":req.params.name}, function(err, cardInfo){
if(err){
res.redirect("/cards");
console.log(err);
} else {
res.render("show", {cardstatsok: cardInfo});
}
});
});
When I do console.log(cardInfo) it returns many objects since I used "Forest" and the copy of the Magic: The Gathering card Forest has been printed many times. Here's one, though:
{ _id: 5a85bb554c8fff0c04f15b8e,
cardName: 'Forest',
id: 243461,
cardSet: 'Duel Decks: Knights vs. Dragons',
image: 'http://gatherer.wizards.com/Handlers/Image.ashx?multiverseid=243461&type=card',
__v: 0 }
find() returns an array which means cardstatsok is an array.
So console.log(cardstatsok.cardName) won't work. Use console.log(cardstatsok[0].cardName) instead for the first card or console.log(cardstatsok) for everything. If you want to print all the card names you have to loop over the array.
To find only one card you can use findOne() instead.

How Do I Query A Child Model Restricting Results Based on Parent Model in Mongoose

I'm not sure how to even phrase this question... but here is a try. I'm calling the Book the "Parent" model and the Author the "Child" model.
I have two mongoose models--- Author and Books:
var Author = mongoose.model("Author", {
name: String
});
var Book = mongoose.model("Book", {
title: String,
inPrint: Boolean,
authors: [ { type: mongoose.Schema.ObjectId, ref: "Author"} ]
});
I am trying to run a query which would return all of the authors (child model) who have books (parent model) which are inPrint.
I could think of ways to do it with multiple queries, but I'm wondering if there is a way to do it with one query.
You could use populate as stated in the docs
There are no joins in MongoDB but sometimes we still want references to documents in other collections. This is where population comes in. Read more about how to include documents from other collections in your query results here.
In your case, it would look something like this:
Book.find().populate('authors')
.where('inPrint').equals(true)
.select('authors')
.exec(function(books) {
// Now you should have an array of books containing authors, which can be
// mapped to a single array.
});
I just stumbled upon this problem today and solved it:
Author.find()
.populate({ path: 'books', match: { inPrint: true } })
.exec(function (err, results) {
console.log(results); // Should do the trick
});
The magic occurs in the match option of populate, which refers to a property of the nested document to populate.
Also check my original post
EDIT: I was confusing books for authors, now it's corrected

Get related items in keystone

Working on a project in KeystoneJS and I'm having trouble figuring out the mongoose relationship bit.
According to the keystone docs, let's say we have the following models: User and Post. Now a post has a relationship to a user, so I'll write:
Post.add({
author: { type: Types.Relationship, ref: 'User' }
});
and then:
User.relationship({ path: 'posts', ref: 'Post', refPath: 'author' });
Now, I want to be able to see all posts related to that User without having to query for both a User and Posts. For example, if I queried for a user object I would like to be able to do user.posts and have access to those related posts. Can you do this with mongoose/keystone?
As far as I understand, keystone's List Relationship has nothing to do with mongoose and querying. Instead, it is used by keystone's admin UI to build out the relationship queries before rendering them in the view. This said I would forget User.relationship(...); solving your problem, although you want it for what I just mentioned.
The following should work fine based on your schema, but only populates the relationship on the one side:
var keystone = require('keystone');
keystone.list('Post').model.findOne().populate('author', function (err, doc) {
console.log(doc.author.name); // Joe
console.log(doc.populated('author')); // '1234af9876b024c680d111a1' -> _id
});
You could also try this the other way, however...
keystone.list('User').model.findOne().populate('posts', function (err, doc) {
doc.posts.forEach(function (post) {
console.log(post);
});
});
...mongoose expects that this definition is added to the Schema. This relationship is added by including this line in your User list file:
User.schema.add({ posts: { type: Types.Relationship, ref: 'Post', many: true } })
After reading the keystone docs, this seems to be logically equivalent the mongoose pure way, User.schema.add({ posts: [{ type: Schema.Types.ObjectId, ref: 'Post' }] });. And now you are now maintaining the relationship on both lists. Instead, you may want to add a method to your keystone list.
User.schema.methods.posts = function(done){
return keystone.list('Post').model.find()
.where('author', this.id )
.exec(done);
};
By adding a method to your User list, it saves you from persisting the array of ObjectIds relating the MongoDB document back to the Post documents. I know this requires a second query, but one of these two options look to be your best bet.
I found this on their github https://github.com/Automattic/mongoose/issues/1888, check it for context, but basically says to use the keystone populateRelated() method. I tested it and does work
// if you've got a single document you want to populate a relationship on, it's neater
Category.model.findOne().exec(function(err, category) {
category.populateRelated('posts', function(err) {
// posts is populated
});
});
I'm aware the question is old but this has to be out there for further reference

How to populate Comment count to an Item list?

I have two models in my app: Item and Comment. An Item can have many Comments, and a Comment instance contains a reference to an Item instance with key 'comment', to keep track of the relationship.
Now I have to send a JSON list of all Items with their Comment count when user requests on a particular URL.
function(req, res){
return Item.find()
.exec(function(err, items) {
return res.send(items);
});
};
I am not sure how can I "populate" comment count to the items. This seems to be a common problem and I tend to think there should be some nicer way of doing this job than brute force.
So please share your thoughts. How would you "populate" the Comment count to the Items?
check the MongoDB documentation and look for the method findAndModify() -- with it you can atomically update a document, e.g. add a comment and increment the document counter at the same time.
findAndModify
The findAndModify command atomically modifies and returns a single document. By default, the returned document does not include the modifications made on the update. To return the document with the modifications made on the update, use the new option.
Example
Use the update option, with update operators $inc for the counter, and $addToSet for adding the actual comment to an embedded array of comments.
db.runCommand(
{
findAndModify: "item",
query: { name: "MyItem", state: "active", rating: { $gt: 10 } },
sort: { rating: 1 },
update: { $inc: { commentCount: 1 },
$addToSet: {comments: new_comment} }
}
)
See:
MongoDB: findAndModify
MongoDB: Update Operators
I did some research on this issue and came up with following results. First, MongoDB docs suggest:
In general, use embedded data models when:
you have “contains” relationships between entities.
you have one-to-many relationships where the “many” objects always appear with or are viewed in the context of their parent documents.
So in my situation, it makes much more sense if Comments are embedded into Items, instead of having independent existence.
Nevertheless, I was curious to know the solution without changing my data model. As mentioned in MongoDB docs:
Referencing provides more flexibility than embedding; however, to
resolve the references, client-side applications must issue follow-up
queries. In other words, using references requires more roundtrips to
the server.
As multiple roundtrips are kosher now, I came up with following solution:
var showList = function(req, res){
// first DB roundtrip: fetch all items
return Item.find()
.exec(function(err, items) {
// second DB roundtrip: fetch comment counts grouped by item ids
Comment.aggregate({
$group: {
_id: '$item',
count: {
$sum: 1
}
}
}, function(err, agg){
// iterate over comment count groups (yes, that little dash is underscore.js)
_.each(agg, function( itr ){
// for each aggregated group, search for corresponding item and put commentCount in it
var item = _.find(items, function( item ){
return item._id.toString() == itr._id.toString();
});
if ( item ) {
item.set('commentCount', itr.count);
}
});
// send items to the client in JSON format
return res.send(items);
})
});
};
Agree? Disagree? Please enlighten me with your comments!
If you have a better answer, please post here, I'll accept it if I find it worthy.

Resources