Learning in progress - Stuck working out how have to have 2 separate documents in different collections: events & event reviews
I'm able to reference objectId's between the two but what's the best way to reference data from within those documents if I wanted to populate information from the event into the event review as it's created? I've been trying to trying to google the answer for a few hours now but have so far been unsuccessful with each method I've found. Not looking for a specific answer just somebody with a bit more knowledge to point me in the right direction.
You need to explicitly state that a field in your event reviews schema references the ObjectID of a document in your event collection. Like so...
var mongoose = require('mongoose'), Schema = mongoose.Schema
var EventReviewSchema = new Schema({
someField:{
type: Schema.Types.ObjectId,
ref: 'Event' // Note this is the name of the event model, not schema
}
})
You mentioned that you want to do this "as it's created."
Remember that in MongoDB, write operations are only atomic at the single document level. So there is no "built in" way to guarantee that every event document you create will have a corresponding event review document (I assume that this is what you're trying to do.)
You will need to save an event document, grab that document's ObjectId and use it to populate the appropriate field in your event review document. But remember, there is chance that the second write operation will fail if some error occurs between the time of the first and second write operation.
My advice is to try to write your application in a way that can use a mongoose FindOrCreate method (with a plugin) when querying for event review documents. It's a quick and dirty way to solve this problem.
EDIT:
var Review = mongoose.model('Review', EventReviewSchema)
module.exports.reviewCreate = function(req, res, next) {
var eventid = req.params.eventid
if (eventid) {
var review = new Review() // Review model
review.someField = eventId
review.save(function(err){
if(err) return next(err)
// save successful
})
}
}
Related
I want to validate some inner elements of an object against schemas that are members of a larger schema before posting that object on the model correspondent to that larger schema.
Someone already asked this question and this answer has something similar on the second option proposed:
var p = new People({
name: 'you',
age: 3
});
p.validate(function(err) {
if (err)
console.log(err);
else
console.log('pass validate');
});
The problem is that before doing the validation one has to create a new model with var People = mongoose.model('People', peopleSchema);.
Does this mean in this case that a new collection 'People' will be stored in the MongoDB database?
Is the act of calling mongoose.model('People', peopleSchema); altering the database in any way?
If so, is there a way to do this validation without creating a new collection, in this case 'People'?
The schema I want to validate will be stored as a component of another document already in the database and I don't want to pollute the database with unused collections.
The two types of objects seem to be so close to one another that having both feels redundant. What is the point of having both schemas and models?
EDIT: Although this has been useful for many people, as mentioned in the comments it answers the "how" rather than the why. Thankfully, the why of the question has been answered elsewhere also, with this answer to another question. This has been linked in the comments for some time but I realise that many may not get that far when reading.
Often the easiest way to answer this type of question is with an example. In this case, someone has already done it for me :)
Take a look here:
http://rawberg.com/blog/nodejs/mongoose-orm-nested-models/
EDIT: The original post (as mentioned in the comments) seems to no longer exist, so I am reproducing it below. Should it ever return, or if it has just moved, please let me know.
It gives a decent description of using schemas within models in mongoose and why you would want to do it, and also shows you how to push tasks via the model while the schema is all about the structure etc.
Original Post:
Let’s start with a simple example of embedding a schema inside a model.
var TaskSchema = new Schema({
name: String,
priority: Number
});
TaskSchema.virtual('nameandpriority')
.get( function () {
return this.name + '(' + this.priority + ')';
});
TaskSchema.method('isHighPriority', function() {
if(this.priority === 1) {
return true;
} else {
return false;
}
});
var ListSchema = new Schema({
name: String,
tasks: [TaskSchema]
});
mongoose.model('List', ListSchema);
var List = mongoose.model('List');
var sampleList = new List({name:'Sample List'});
I created a new TaskSchema object with basic info a task might have. A Mongoose virtual attribute is setup to conveniently combine the name and priority of the Task. I only specified a getter here but virtual setters are supported as well.
I also defined a simple task method called isHighPriority to demonstrate how methods work with this setup.
In the ListSchema definition you’ll notice how the tasks key is configured to hold an array of TaskSchema objects. The task key will become an instance of DocumentArray which provides special methods for dealing with embedded Mongo documents.
For now I only passed the ListSchema object into mongoose.model and left the TaskSchema out. Technically it's not necessary to turn the TaskSchema into a formal model since we won’t be saving it in it’s own collection. Later on I’ll show you how it doesn’t harm anything if you do and it can help to organize all your models in the same way especially when they start spanning multiple files.
With the List model setup let’s add a couple tasks to it and save them to Mongo.
var List = mongoose.model('List');
var sampleList = new List({name:'Sample List'});
sampleList.tasks.push(
{name:'task one', priority:1},
{name:'task two', priority:5}
);
sampleList.save(function(err) {
if (err) {
console.log('error adding new list');
console.log(err);
} else {
console.log('new list successfully saved');
}
});
The tasks attribute on the instance of our List model (sampleList) works like a regular JavaScript array and we can add new tasks to it using push. The important thing to notice is the tasks are added as regular JavaScript objects. It’s a subtle distinction that may not be immediately intuitive.
You can verify from the Mongo shell that the new list and tasks were saved to mongo.
db.lists.find()
{ "tasks" : [
{
"_id" : ObjectId("4dd1cbeed77909f507000002"),
"priority" : 1,
"name" : "task one"
},
{
"_id" : ObjectId("4dd1cbeed77909f507000003"),
"priority" : 5,
"name" : "task two"
}
], "_id" : ObjectId("4dd1cbeed77909f507000001"), "name" : "Sample List" }
Now we can use the ObjectId to pull up the Sample List and iterate through its tasks.
List.findById('4dd1cbeed77909f507000001', function(err, list) {
console.log(list.name + ' retrieved');
list.tasks.forEach(function(task, index, array) {
console.log(task.name);
console.log(task.nameandpriority);
console.log(task.isHighPriority());
});
});
If you run that last bit of code you’ll get an error saying the embedded document doesn’t have a method isHighPriority. In the current version of Mongoose you can’t access methods on embedded schemas directly. There’s an open ticket to fix it and after posing the question to the Mongoose Google Group, manimal45 posted a helpful work-around to use for now.
List.findById('4dd1cbeed77909f507000001', function(err, list) {
console.log(list.name + ' retrieved');
list.tasks.forEach(function(task, index, array) {
console.log(task.name);
console.log(task.nameandpriority);
console.log(task._schema.methods.isHighPriority.apply(task));
});
});
If you run that code you should see the following output on the command line.
Sample List retrieved
task one
task one (1)
true
task two
task two (5)
false
With that work-around in mind let’s turn the TaskSchema into a Mongoose model.
mongoose.model('Task', TaskSchema);
var Task = mongoose.model('Task');
var ListSchema = new Schema({
name: String,
tasks: [Task.schema]
});
mongoose.model('List', ListSchema);
var List = mongoose.model('List');
The TaskSchema definition is the same as before so I left it out. Once its turned into a model we can still access it’s underlying Schema object using dot notation.
Let’s create a new list and embed two Task model instances within it.
var demoList = new List({name:'Demo List'});
var taskThree = new Task({name:'task three', priority:10});
var taskFour = new Task({name:'task four', priority:11});
demoList.tasks.push(taskThree.toObject(), taskFour.toObject());
demoList.save(function(err) {
if (err) {
console.log('error adding new list');
console.log(err);
} else {
console.log('new list successfully saved');
}
});
As we’re embedding the Task model instances into the List we’re calling toObject on them to convert their data into plain JavaScript objects that the List.tasks DocumentArray is expecting. When you save model instances this way your embedded documents will contain ObjectIds.
The complete code example is available as a gist. Hopefully these work-arounds help smooth things over as Mongoose continues to develop. I’m still pretty new to Mongoose and MongoDB so please feel free to share better solutions and tips in the comments. Happy data modeling!
Schema is an object that defines the structure of any documents that will be stored in your MongoDB collection; it enables you to define types and validators for all of your data items.
Model is an object that gives you easy access to a named collection, allowing you to query the collection and use the Schema to validate any documents you save to that collection. It is created by combining a Schema, a Connection, and a collection name.
Originally phrased by Valeri Karpov, MongoDB Blog
I don't think the accepted answer actually answers the question that was posed. The answer doesn't explain why Mongoose has decided to require a developer to provide both a Schema and a Model variable. An example of a framework where they have eliminated the need for the developer to define the data schema is django--a developer writes up their models in the models.py file, and leaves it to the framework to manage the schema. The first reason that comes to mind for why they do this, given my experience with django, is ease-of-use. Perhaps more importantly is the DRY (don't repeat yourself) principle--you don't have to remember to update the schema when you change the model--django will do it for you! Rails also manages the schema of the data for you--a developer doesn't edit the schema directly, but changes it by defining migrations that manipulate the schema.
One reason I could understand that Mongoose would separate the schema and the model is instances where you would want to build a model from two schemas. Such a scenario might introduce more complexity than is worth managing--if you have two schemas that are managed by one model, why aren't they one schema?
Perhaps the original question is more a relic of the traditional relational database system. In world NoSQL/Mongo world, perhaps the schema is a little more flexible than MySQL/PostgreSQL, and thus changing the schema is more common practice.
To understand why? you have to understand what actually is Mongoose?
Well, the mongoose is an object data modeling library for MongoDB and Node JS, providing a higher level of abstraction. So it's a bit like the relationship between Express and Node, so Express is a layer of abstraction over regular Node, while Mongoose is a layer of abstraction over the regular MongoDB driver.
An object data modeling library is just a way for us to write Javascript code that will then interact with a database. So we could just use a regular MongoDB driver to access our database, it would work just fine.
But instead we use Mongoose because it gives us a lot more functionality out of the box, allowing for faster and simpler development of our applications.
So, some of the features Mongoose gives us schemas to model our data and relationship, easy data validation, a simple query API, middleware, and much more.
In Mongoose, a schema is where we model our data, where we describe the structure of the data, default values, and validation, then we take that schema and create a model out of it, a model is basically a wrapper around the schema, which allows us to actually interface with the database in order to create, delete, update, and read documents.
Let's create a model from a schema.
const tourSchema = new mongoose.Schema({
name: {
type: String,
required: [true, 'A tour must have a name'],
unique: true,
},
rating: {
type: Number,
default: 4.5,
},
price: {
type: Number,
required: [true, 'A tour must have a price'],
},
});
//tour model
const Tour = mongoose.model('Tour', tourSchema);
According to convetion first letter of a model name must be capitalized.
Let's create instance of our model that we created using mongoose and schema. also, interact with our database.
const testTour = new Tour({ // instance of our model
name: 'The Forest Hiker',
rating: 4.7,
price: 497,
});
// saving testTour document into database
testTour
.save()
.then((doc) => {
console.log(doc);
})
.catch((err) => {
console.log(err);
});
So having both schama and modle mongoose makes our life easier.
Think of Model as a wrapper to schemas. Schemas define the structure of your document , what kind of properties can you expect and what will be their data type (String,Number etc.). Models provide a kind of interface to perform CRUD on schema. See this post on FCC.
Schema basically models your data (where you provide datatypes for your fields) and can do some validations on your data. It mainly deals with the structure of your collection.
Whereas the model is a wrapper around your schema to provide you with CRUD methods on collections. It mainly deals with adding/querying the database.
Having both schema and model could appear redundant when compared to other frameworks like Django (which provides only a Model) or SQL (where we create only Schemas and write SQL queries and there is no concept of model). But, this is just the way Mongoose implements it.
Im developing an app that uses node-->express-->mongodb-->mongoose.
The app has a concept similar to an auction, where there are bids made and it is very likely that most users will try to make their bid at the last seconds of the auction.
Before users bid is accepted, there is a validation process ( dates, times, bid sizes, users available account balance etc) , and once that is done, the bid is accepted and auction document is saved.
The question is, is it possible that users will update the document between
findOne() call, and a save() call made on the resulting document.
In other words,
auction.findOne(_id: "something").exec(function(err, doc) {
if ( !err && doc ) {
//use the found document to do some validation that may or may not
//take some time.
//then document validates,
if ( document is valid ) {
doc.last_bid = "xxx";
//is it possible that someone makes an update to the document
//here that invalidates the document, and i will end up saving
//an invalid document
doc.save();
}
// else -> dont save anything and reject deferred promise.
//Is it possible that after ive done my validation and document validated successfully, that someone updates it, before i get to call save().
}
});
Is it possible to "lock" a document when making findOne/find query ,and reject any updates to it until im done with using the document ?
I think that you can't lock a mongoose document. I propose you a solution, I think that you have a classical problem of "exclussion", so, I will use a mutex to try to access critical section, in your case, "critical section" is udpate mongoose document "doc.last_bid = 'xxx'". Try "locks":
https://github.com/Wizcorp/locks
var rwlock = locks.createReadWriteLock();
rwlock.readLock(function () {
if(document is valid) {
doc.last_bid = 'xxx'
doc.set_is_not_valid_or_used = true; // invalidate document => if condition
}
rwlock.unlock();
});
Consider the blog/comment schemas where nesting is appropriate (even if you disagree):
var CommentSchema = new Schema({ name: String, body: String });
var BlogPostSchema = new Schema({ title: String, comments: [CommentSchema] });
I understand how to add, update, delete comments for a blog post, but all of these methods require the save() method to be called on the parent blog post document:
blog_post.comments.push( new Comment({...}) );
blog_post.save();
I would like to be able to make the Comment schema aware that it is nested inside of another schema so that I can call save() on a comment document and it's smart enough to update the parent blog post. In my app logic, I already know the blog post id, so I would like to do something like this:
CommentSchema.virtual('blog_post_id');
CommentSchema.pre('save', function (next) {
var comment = this;
if( !comment.blog_post_id ) throw new Error('Need a blog post id');
BlogModel.findById( comment.blog_post_id, function(err, post) {
post.comments.push( comment );
post.save(next);
});
});
var comment = new Comment({ blog_post_id: 123, name: 'Joe', body: 'foo' });
comment.save();
The above works, but I still end up with a top-level Comments collection separate from the blog posts (this is just how mongoose works, I accept that).
Question: How do I prevent Mongoose from creating a separate "Comments" collection. In the pre-save method I would like to call next() without any write operations taking place afterwards. Any thoughts?
This has earned me the Tumbleweed badge... hooray!?!?
So I have written a lot of code which accomplished the above. I don't want to release it until I have done more testing. But if anybody is interested in this, please let me know by posting here. I will gladly hand over what I have (which is going into production soon).
Right now my code doesn't support deep nesting... meaning you can only work with "simple" nesting similar to the blog/comments example above. I have the architecture in place to handle more complex nesting in the future, but I don't have the time to test right now (darn deadlines). Here are some of the big points about my solution so far:
All operations require the parent document's id (this makes sense once you start using it)
find, findOne, save, and remove directly on a nested model
findById doesn't (can't) work - well it maybe could work but would require searching the entire collection, which is slow. Must use findOne + parent id instead (see examples).
Super fast - uses projection for finding, and saves using Model.update() on the parent model (which is really fast).
All middleware still executes (pre/post and validation)
None of the findAndUpdate/Remove methods work [yet?]
Setup
// setup the "nestedSchema" plugin
var nestedSchema = require("./plugins/nestedSchema");
CommentSchema.plugin(nestedSchema, {
path: 'comments',
ownerModelPath: './BlogPostModel',
ownerIdFieldName: 'blogpost_id'
});
Examples - take note that the parent's blogpost_id is ALWAYS used - this is a requirement which makes it stay fast (callbacks and error handling removed for brevity):
// create a new comment
var comment = new CommentModel({
blogpost_id: [id],
name: 'Joe Schmoe',
body: 'The content of the comment'
});
comment.save();
// use findOne in leu of findById
CommentModel.findOne({blogpost_id: [id], _id: [id]}, function( err, comment ) {
comment.set('body', 'This comment has been updated directly!');
comment.save();
});
// find all hateful comments and remove
CommentModel.find({blogpost_id: [id], body: /sucks|stupid|dumb/gi}).remove();
Using the mongoose-relationship plugin from https://www.npmjs.org/package/mongoose-relationship
it is possible to make your documents aware of their relations.
Corresponding references are updated by the plugin when adding/removing documents.
There is a good example on the github page: https://github.com/sabymike/mongoose-relationship
NodeJS + Express, MongoDB + Mongoose
I have a JSON feed where each record has a set of "venue" attributes (things like "venue name" "venue location" "venue phone" etc). I want to create a collection of all venues in the feed -- one instance of each venue, no dupes.
I loop through the JSON and test whether the venue exists in my venue collection. If it doesn't, save it.
jsonObj.events.forEach(function(element, index, array){
Venue.findOne({'name': element.vname}, function(err,doc){
if(doc == null){
var instance = new Venue();
instance.name = element.vname;
instance.location = element.location;
instance.phone = element.vphone;
instance.save();
}
}
}
Desired: A list of all venues (no dupes).
Result: Plenty of dupes in the venue collection.
Basically, the loop created a new Venue record for every record in the JSON feed.
I'm learning Node and its async qualities, so I believe the for loop finishes before even the first save() function finishes -- so the if statement is always checking against an empty collection. Console.logging backs this claim up.
I'm not sure how to rework this so that it performs the desired task. I've tried caolan's async module but I can't get it to help. There's a good chance I'm using incorrectly.
Thanks so much for pointing me in the right direction -- I've searched to no avail. If the async module is the right answer, I'd love your help with how to implement it in this specific case.
Thanks again!
Why not go the other way with it? You didn't say what your persistence layer is, but it looks like mongoose or possibly FastLegS. In either case, you can create a Unique Index on your Name field. Then, you can just try to save anything, and handle the error if it's a unique index violation.
Whatever you do, you must do as #Paul suggests and make a unique index in the database. That's the only way to ensure uniqueness.
But the main problem with your code is that in the instance.save() call, you need a callback that triggers the next iteration, otherwise the database will not have had time to save the new record. It's a race condition. You can solve that problem with caolan's forEachSeries function.
Alternatively, you could get an array of records already in the Venue collection that match an item in your JSON object, then filter the matches out of the object, then iteratively add each item left in the filtered JSON object. This will minimize the number of database operations by not trying to create duplicates in the first place.
Venue.find({'name': { $in: jsonObj.events.map(function(event){ return event.vname; }) }}, function (err, docs){
var existingVnames = docs.map(function(doc){ return doc.name; });
var filteredEvents = jsonObj.events.filter(function(event){
return existingVnames.indexOf(event.vname) === -1;
});
filteredEvents.forEach(function(event){
var venue = new Venue();
venue.name = event.vname;
venue.location = event.location;
venue.phone = event.vphone;
venue.save(function (err){
// Optionally, do some logging here, perhaps.
if (err) return console.error('Something went wrong!');
else return console.log('Successfully created new venue %s', venue.name);
});
});
});