Consider the blog/comment schemas where nesting is appropriate (even if you disagree):
var CommentSchema = new Schema({ name: String, body: String });
var BlogPostSchema = new Schema({ title: String, comments: [CommentSchema] });
I understand how to add, update, delete comments for a blog post, but all of these methods require the save() method to be called on the parent blog post document:
blog_post.comments.push( new Comment({...}) );
blog_post.save();
I would like to be able to make the Comment schema aware that it is nested inside of another schema so that I can call save() on a comment document and it's smart enough to update the parent blog post. In my app logic, I already know the blog post id, so I would like to do something like this:
CommentSchema.virtual('blog_post_id');
CommentSchema.pre('save', function (next) {
var comment = this;
if( !comment.blog_post_id ) throw new Error('Need a blog post id');
BlogModel.findById( comment.blog_post_id, function(err, post) {
post.comments.push( comment );
post.save(next);
});
});
var comment = new Comment({ blog_post_id: 123, name: 'Joe', body: 'foo' });
comment.save();
The above works, but I still end up with a top-level Comments collection separate from the blog posts (this is just how mongoose works, I accept that).
Question: How do I prevent Mongoose from creating a separate "Comments" collection. In the pre-save method I would like to call next() without any write operations taking place afterwards. Any thoughts?
This has earned me the Tumbleweed badge... hooray!?!?
So I have written a lot of code which accomplished the above. I don't want to release it until I have done more testing. But if anybody is interested in this, please let me know by posting here. I will gladly hand over what I have (which is going into production soon).
Right now my code doesn't support deep nesting... meaning you can only work with "simple" nesting similar to the blog/comments example above. I have the architecture in place to handle more complex nesting in the future, but I don't have the time to test right now (darn deadlines). Here are some of the big points about my solution so far:
All operations require the parent document's id (this makes sense once you start using it)
find, findOne, save, and remove directly on a nested model
findById doesn't (can't) work - well it maybe could work but would require searching the entire collection, which is slow. Must use findOne + parent id instead (see examples).
Super fast - uses projection for finding, and saves using Model.update() on the parent model (which is really fast).
All middleware still executes (pre/post and validation)
None of the findAndUpdate/Remove methods work [yet?]
Setup
// setup the "nestedSchema" plugin
var nestedSchema = require("./plugins/nestedSchema");
CommentSchema.plugin(nestedSchema, {
path: 'comments',
ownerModelPath: './BlogPostModel',
ownerIdFieldName: 'blogpost_id'
});
Examples - take note that the parent's blogpost_id is ALWAYS used - this is a requirement which makes it stay fast (callbacks and error handling removed for brevity):
// create a new comment
var comment = new CommentModel({
blogpost_id: [id],
name: 'Joe Schmoe',
body: 'The content of the comment'
});
comment.save();
// use findOne in leu of findById
CommentModel.findOne({blogpost_id: [id], _id: [id]}, function( err, comment ) {
comment.set('body', 'This comment has been updated directly!');
comment.save();
});
// find all hateful comments and remove
CommentModel.find({blogpost_id: [id], body: /sucks|stupid|dumb/gi}).remove();
Using the mongoose-relationship plugin from https://www.npmjs.org/package/mongoose-relationship
it is possible to make your documents aware of their relations.
Corresponding references are updated by the plugin when adding/removing documents.
There is a good example on the github page: https://github.com/sabymike/mongoose-relationship
Related
The two types of objects seem to be so close to one another that having both feels redundant. What is the point of having both schemas and models?
EDIT: Although this has been useful for many people, as mentioned in the comments it answers the "how" rather than the why. Thankfully, the why of the question has been answered elsewhere also, with this answer to another question. This has been linked in the comments for some time but I realise that many may not get that far when reading.
Often the easiest way to answer this type of question is with an example. In this case, someone has already done it for me :)
Take a look here:
http://rawberg.com/blog/nodejs/mongoose-orm-nested-models/
EDIT: The original post (as mentioned in the comments) seems to no longer exist, so I am reproducing it below. Should it ever return, or if it has just moved, please let me know.
It gives a decent description of using schemas within models in mongoose and why you would want to do it, and also shows you how to push tasks via the model while the schema is all about the structure etc.
Original Post:
Let’s start with a simple example of embedding a schema inside a model.
var TaskSchema = new Schema({
name: String,
priority: Number
});
TaskSchema.virtual('nameandpriority')
.get( function () {
return this.name + '(' + this.priority + ')';
});
TaskSchema.method('isHighPriority', function() {
if(this.priority === 1) {
return true;
} else {
return false;
}
});
var ListSchema = new Schema({
name: String,
tasks: [TaskSchema]
});
mongoose.model('List', ListSchema);
var List = mongoose.model('List');
var sampleList = new List({name:'Sample List'});
I created a new TaskSchema object with basic info a task might have. A Mongoose virtual attribute is setup to conveniently combine the name and priority of the Task. I only specified a getter here but virtual setters are supported as well.
I also defined a simple task method called isHighPriority to demonstrate how methods work with this setup.
In the ListSchema definition you’ll notice how the tasks key is configured to hold an array of TaskSchema objects. The task key will become an instance of DocumentArray which provides special methods for dealing with embedded Mongo documents.
For now I only passed the ListSchema object into mongoose.model and left the TaskSchema out. Technically it's not necessary to turn the TaskSchema into a formal model since we won’t be saving it in it’s own collection. Later on I’ll show you how it doesn’t harm anything if you do and it can help to organize all your models in the same way especially when they start spanning multiple files.
With the List model setup let’s add a couple tasks to it and save them to Mongo.
var List = mongoose.model('List');
var sampleList = new List({name:'Sample List'});
sampleList.tasks.push(
{name:'task one', priority:1},
{name:'task two', priority:5}
);
sampleList.save(function(err) {
if (err) {
console.log('error adding new list');
console.log(err);
} else {
console.log('new list successfully saved');
}
});
The tasks attribute on the instance of our List model (sampleList) works like a regular JavaScript array and we can add new tasks to it using push. The important thing to notice is the tasks are added as regular JavaScript objects. It’s a subtle distinction that may not be immediately intuitive.
You can verify from the Mongo shell that the new list and tasks were saved to mongo.
db.lists.find()
{ "tasks" : [
{
"_id" : ObjectId("4dd1cbeed77909f507000002"),
"priority" : 1,
"name" : "task one"
},
{
"_id" : ObjectId("4dd1cbeed77909f507000003"),
"priority" : 5,
"name" : "task two"
}
], "_id" : ObjectId("4dd1cbeed77909f507000001"), "name" : "Sample List" }
Now we can use the ObjectId to pull up the Sample List and iterate through its tasks.
List.findById('4dd1cbeed77909f507000001', function(err, list) {
console.log(list.name + ' retrieved');
list.tasks.forEach(function(task, index, array) {
console.log(task.name);
console.log(task.nameandpriority);
console.log(task.isHighPriority());
});
});
If you run that last bit of code you’ll get an error saying the embedded document doesn’t have a method isHighPriority. In the current version of Mongoose you can’t access methods on embedded schemas directly. There’s an open ticket to fix it and after posing the question to the Mongoose Google Group, manimal45 posted a helpful work-around to use for now.
List.findById('4dd1cbeed77909f507000001', function(err, list) {
console.log(list.name + ' retrieved');
list.tasks.forEach(function(task, index, array) {
console.log(task.name);
console.log(task.nameandpriority);
console.log(task._schema.methods.isHighPriority.apply(task));
});
});
If you run that code you should see the following output on the command line.
Sample List retrieved
task one
task one (1)
true
task two
task two (5)
false
With that work-around in mind let’s turn the TaskSchema into a Mongoose model.
mongoose.model('Task', TaskSchema);
var Task = mongoose.model('Task');
var ListSchema = new Schema({
name: String,
tasks: [Task.schema]
});
mongoose.model('List', ListSchema);
var List = mongoose.model('List');
The TaskSchema definition is the same as before so I left it out. Once its turned into a model we can still access it’s underlying Schema object using dot notation.
Let’s create a new list and embed two Task model instances within it.
var demoList = new List({name:'Demo List'});
var taskThree = new Task({name:'task three', priority:10});
var taskFour = new Task({name:'task four', priority:11});
demoList.tasks.push(taskThree.toObject(), taskFour.toObject());
demoList.save(function(err) {
if (err) {
console.log('error adding new list');
console.log(err);
} else {
console.log('new list successfully saved');
}
});
As we’re embedding the Task model instances into the List we’re calling toObject on them to convert their data into plain JavaScript objects that the List.tasks DocumentArray is expecting. When you save model instances this way your embedded documents will contain ObjectIds.
The complete code example is available as a gist. Hopefully these work-arounds help smooth things over as Mongoose continues to develop. I’m still pretty new to Mongoose and MongoDB so please feel free to share better solutions and tips in the comments. Happy data modeling!
Schema is an object that defines the structure of any documents that will be stored in your MongoDB collection; it enables you to define types and validators for all of your data items.
Model is an object that gives you easy access to a named collection, allowing you to query the collection and use the Schema to validate any documents you save to that collection. It is created by combining a Schema, a Connection, and a collection name.
Originally phrased by Valeri Karpov, MongoDB Blog
I don't think the accepted answer actually answers the question that was posed. The answer doesn't explain why Mongoose has decided to require a developer to provide both a Schema and a Model variable. An example of a framework where they have eliminated the need for the developer to define the data schema is django--a developer writes up their models in the models.py file, and leaves it to the framework to manage the schema. The first reason that comes to mind for why they do this, given my experience with django, is ease-of-use. Perhaps more importantly is the DRY (don't repeat yourself) principle--you don't have to remember to update the schema when you change the model--django will do it for you! Rails also manages the schema of the data for you--a developer doesn't edit the schema directly, but changes it by defining migrations that manipulate the schema.
One reason I could understand that Mongoose would separate the schema and the model is instances where you would want to build a model from two schemas. Such a scenario might introduce more complexity than is worth managing--if you have two schemas that are managed by one model, why aren't they one schema?
Perhaps the original question is more a relic of the traditional relational database system. In world NoSQL/Mongo world, perhaps the schema is a little more flexible than MySQL/PostgreSQL, and thus changing the schema is more common practice.
To understand why? you have to understand what actually is Mongoose?
Well, the mongoose is an object data modeling library for MongoDB and Node JS, providing a higher level of abstraction. So it's a bit like the relationship between Express and Node, so Express is a layer of abstraction over regular Node, while Mongoose is a layer of abstraction over the regular MongoDB driver.
An object data modeling library is just a way for us to write Javascript code that will then interact with a database. So we could just use a regular MongoDB driver to access our database, it would work just fine.
But instead we use Mongoose because it gives us a lot more functionality out of the box, allowing for faster and simpler development of our applications.
So, some of the features Mongoose gives us schemas to model our data and relationship, easy data validation, a simple query API, middleware, and much more.
In Mongoose, a schema is where we model our data, where we describe the structure of the data, default values, and validation, then we take that schema and create a model out of it, a model is basically a wrapper around the schema, which allows us to actually interface with the database in order to create, delete, update, and read documents.
Let's create a model from a schema.
const tourSchema = new mongoose.Schema({
name: {
type: String,
required: [true, 'A tour must have a name'],
unique: true,
},
rating: {
type: Number,
default: 4.5,
},
price: {
type: Number,
required: [true, 'A tour must have a price'],
},
});
//tour model
const Tour = mongoose.model('Tour', tourSchema);
According to convetion first letter of a model name must be capitalized.
Let's create instance of our model that we created using mongoose and schema. also, interact with our database.
const testTour = new Tour({ // instance of our model
name: 'The Forest Hiker',
rating: 4.7,
price: 497,
});
// saving testTour document into database
testTour
.save()
.then((doc) => {
console.log(doc);
})
.catch((err) => {
console.log(err);
});
So having both schama and modle mongoose makes our life easier.
Think of Model as a wrapper to schemas. Schemas define the structure of your document , what kind of properties can you expect and what will be their data type (String,Number etc.). Models provide a kind of interface to perform CRUD on schema. See this post on FCC.
Schema basically models your data (where you provide datatypes for your fields) and can do some validations on your data. It mainly deals with the structure of your collection.
Whereas the model is a wrapper around your schema to provide you with CRUD methods on collections. It mainly deals with adding/querying the database.
Having both schema and model could appear redundant when compared to other frameworks like Django (which provides only a Model) or SQL (where we create only Schemas and write SQL queries and there is no concept of model). But, this is just the way Mongoose implements it.
First of all, I'm fairy new to MEAN Stack and Node.js, so go easy on me :)
I know all these kind of functions in Node.js are callback functions and are asynchronous, but since Node.js is widely used, what I think as a downside is probably not a downside at all. Let me define my problem;
Let's say you have a Schema of a user whose one field is a reference to another schema. e.g.
PersonSchema = {
name: String,
...
...
...
projects : [{
type: Schema.ObjectId,
ref: 'Project',
required: true
}],
...
...
}
How do I exactly make sure that I get a particular project from database and add to the user's database with mongoose?
What I'm doing is something like:
var proj = Project.findOne({"name":"The Manhattan Project"})
Since these all happen in app.post(...) method, they eventually happen, but not necessarily when they are supposed to.
var newUser = new User();
newUser.name = req.body.name;
newUser... = ...
newUser.projects = [proj.id];
...
newUser.save(callback function{})
When I save the new User object with this project, it gives me an error because proj is undefined (which is understandable because the query function is a callback function) buy I literally have no idea to fix this right now.
Can someone give me some idea to how to fix this, or fix my way of understanding if it's wrong, or at least push me to the right direction so I can find more resources by researching.
Thanks a lot!
FindOne takes a callback as its second param. You should not assign project to its return, you should just call it, then carry on your use logic inside the callback...
Model.findOne({ age: 5}, function (err, doc){
// doc is a Document
// do things that happen after a sync retrieve in here...
});
So, I am building a small blog-like project in Node, and I am running into an issue with orphaned database references. I have two models in separate files that reference each other.
Here are my models:
// ./models/user
Var UserSchema = mongoose.Schema({
name: String,
posts: [{type: mongoose.SchemaTypes.ObjectId, ref:'Post'}]
});
// ./models/post
var PostSchema = mongoose.Schema({
title:String,
post_body: String,
posted_by: mongoose.SchemaTypes.ObjectId
});
My question is when you delete say, a Post, how would you delete the reference in the User's post array? My thinking was I could create a middleware to run before the delete route and delete the reference in the User's post array before I actually delete the post. Would that be considered the best way to go about it? I found a post on Stack that uses a 'pre' function in the schema like this:
// ./models/post
PostSchema.pre('remove', function(next){
this.model('User').remove({posts: this._id}, next);
});
Here is the actual stack post: Automatically remove referencing objects on deletion in MongoDB . I could not get this work though. I did ,however, implement a custom middleware to delete the references, but feel it might not be best practice. Any tips/advice would be greatly appreciated. Thanks!
You don't want .remove() here but you want .update() with $pull instead:
PostSchema.pre('update',function(next) {
this.model('User').update(
{ },
{ "$pull": { "posts": this._id } },
{ "multi": true },
next
);
})
That's the correct operation to remove something from an array. The "multi" makes sure that the "post" would be removed for all User objects that reference it, thought it probably really is only one document anyway.
The .remove() method is for "removing" whole documents. The .update() method makes "changes" to documents.
Let's assume I have mongoose models for books and pages like this:
mongoose.model("Book", new Schema({
title: String
});
and this
mongoose.model("Page", new Schema({
pageNumber: Number,
_bookId: {type: ObjectId, ref: "Book"}
});
Every page keeps track which book it belongs to. Now I want to have an array of books that have a page with pageNumber 500.
I could do the following:
Page.find({pageNumber: 500})
.populate("_bookId")
.then(function (pages) {
var books = [];
pages.forEach(function (page) {
books.push(page._bookId); // page._bookId now contains a Book document
});
return q(books);
}).then(function (books) {
// Do something with the books
});
Yet, the part where I loop over the pages seems cumbersome and that kind of extraction could probably be done by mongo. My question is how that would work.
Is using populate even the best way to go here?
I would like to keep the schemas the way they are though.
I think your schema design is the issue here. Why is pages a separate schema? You should use Mongo's embedding capabilities to make Page an array in Book:
mongoose.model("Book", new Schema({
title: String,
pages: [...]
});
Then you can search for books that have a page #N.
Additionally, if your pages is nothing more than a page number and an associated book you can just make pages a number representing the total number of pages.
Edit: If such a schema is just a simplification of your use case and really you cant do embedding, then you're out of luck. The abstraction your looking for is called a join, and Mongo doesn't support that because its not what Mongo is going for. If thats really a primary use case of yours you should look to using a relational database (or change your schema).
Here is my Mongoose Schema:
var SchemaA = new Schema({
field1: String,
.......
fieldB : { type: Schema.Types.ObjectId, ref: 'SchemaB' }
});
var SchemaB = new Schema({
field1: String,
.......
fieldC : { type: Schema.Types.ObjectId, ref: 'SchemaC' }
});
var SchemaC = new Schema({
field1: String,
.......
.......
.......
});
While i access schemaA using find query, i want to have fields/property
of SchemaA along with SchemaB and SchemaC in the same way as we apply join operation in SQL database.
This is my approach:
SchemaA.find({})
.populate('fieldB')
.exec(function (err, result){
SchemaB.populate(result.fieldC,{path:'fieldB'},function(err, result){
.............................
});
});
The above code is working perfectly, but the problem is:
I want to have information/properties/fields of SchemaC through SchemaA, and i don't want to populate fields/properties of SchemaB.
The reason for not wanting to get the properties of SchemaB is, extra population will slows the query unnecessary.
Long story short:
I want to populate SchemaC through SchemaA without populating SchemaB.
Can you please suggest any way/approach?
As an avid mongodb fan, I suggest you use a relational database for highly relational data - that's what it's built for. You are losing all the benefits of mongodb when you have to perform 3+ queries to get a single object.
Buuuuuut, I know that comment will fall on deaf ears. Your best bet is to be as conscious as you can about performance. Your first step is to limit the fields to the minimum required. This is just good practice even with basic queries and any database engine - only get the fields you need (eg. SELECT * FROM === bad... just stop doing it!). You can also try doing lean queries to help save a lot of post-processing work mongoose does with the data. I didn't test this, but it should work...
SchemaA.find({}, 'field1 fieldB', { lean: true })
.populate({
name: 'fieldB',
select: 'fieldC',
options: { lean: true }
}).exec(function (err, result) {
// not sure how you are populating "result" in your example, as it should be an array,
// but you said your code works... so I'll let you figure out what goes here.
});
Also, a very "mongo" way of doing what you want is to save a reference in SchemaC back to SchemaA. When I say "mongo" way of doing it, you have to break away from your years of thinking about relational data queries. Do whatever it takes to perform fewer queries on the database, even if it requires two-way references and/or data duplication.
For example, if I had a Book schema and Author schema, I would likely save the authors first and last name in the Books collection, along with an _id reference to the full profile in the Authors collection. That way I can load my Books in a single query, still display the author's name, and then generate a hyperlink to the author's profile: /author/{_id}. This is known as "data denormalization", and it has been known to give people heartburn. I try and use it on data that doesn't change very often - like people's names. In the occasion that a name does change, it's trivial to write a function to update all the names in multiple places.
SchemaA.find({})
.populate({
path: "fieldB",
populate:{path:"fieldC"}
}).exec(function (err, result) {
//this is how you can get all key value pair of SchemaA, SchemaB and SchemaC
//example: result.fieldB.fieldC._id(key of SchemaC)
});
why not add a ref to SchemaC on SchemaA? there will be no way to bridge to SchemaC from SchemaA if there is no SchemaB the way you currently have it unless you populate SchemaB with no other data than a ref to SchemaC
As explained in the docs under Field Selection, you can restrict what fields are returned.
.populate('fieldB') becomes populate('fieldB', 'fieldC -_id'). The -_id is required to omit the _id field just like when using select().
I think this is not possible.Because,when a document in A referring a document in B and that document is referring another document in C, how can document in A know which document to refer from C without any help from B.