In my web app XY I'm showing a classic list/table of data (documents) to the user. While all the sort functions provided by MongoDB ( and Mongoose, i'm using Mongoose) are pretty clear to me, I'm not interested in sorting by date or alphabetical order. In my case it would be important to let the user maintain a custom sort as in manually drag/drop items around to set a specific order of the documents (e.g. putting favourites in top of the list ). The UI to do this is a no-brainer but how to actually save the order in the database, brain-freeze.
Problem : How would I go about saving such a custom order of documents ?
What I use : NodeJS / Express / Mongoose (MongoDB)
Ideas
So far I could think of 2 ideas on how to do this.
A : Having an additional key (e.g. orderKey) in the mongoose Schema. Big con : I would need to keep constantly updating all documents orderKeys. Also I would need some sort of auto-increment for new documents.
const mySch = new Schema({
orderKey : { type : Number }
});
B : Creating one Schema/Model only for sorting, with an Array including all documents _ids for example. The order of the elements within the array would be used as reference for the custom order of the documents. Whenever the order changes, this Array would be changed as well.
conts orderSch = new Schema({
orderArray : { type : Array }
});
mongoose.model('Order', orderSch);
/* const customOrder = new Order({
orderArray : [ _id1, _id2, _id3, _id10, _id7, .. ]
}); */
Any more ideas or best practises are highly appreciated !
I'm new to Node.js and Mongoose library. My problem is i have two collections schema
Restaurant
Reviews
I tried to add a virtual field (reviews_count) in Restaurent collection for reviews count.
How can I achieve this one? Is there any specific function available in mongoose?
Restaurant: Reviews:
_id: ObjectId restaurent_id: (Restaurant ref id)
name: String review: String
I expect the output while try to get the restaurant details.
{
"_id": "2344....",
"name": "Restaurant name".
"reviews_count": 220
}
yes, this is possible. There are many ways to do it: You could actually define an async virtual, that will query Reviews, but I don't like this approach too much, since you have to make sure to await it or to handle a callback, which looks and smells bad:
const count = await restaurant.reviews
Alternatively, you could just define an instance method something like this (async would be better, tough), which I like better, since it makes the code easier to read, when calling this function:
// assign a function to the "methods" object of your restaurant schema
restaurant.methods.getReviews = function(cb) {
return this.model('Reviews').find({ restaurant_id: this._id }, cb);
};
Finally, there is a third, and in my opinion the best option: Virtual Populate
Mongoose 4.5 introduced this feature and it perfectly fits your usecase. Define your field like this:
Restaurant.virtual('reviews', {
ref: 'Reviews',
localField: '_id',
foreignField: 'restaurant_id'
});
and then query it like this:
Restaurant.findOne().populate('reviews').exec(function(error, reviews) {
// `reviews.count` is the virtual you are looking for
});
This is imho the cleanest and most versatile solution since it keeps you from unnecessarily adding arrays with refs to you your main object and still get an easy access to all reviews of a restaurant. You can find a good article here:
http://thecodebarbarian.com/mongoose-virtual-populate
Note: This solution might give you performance problems, since we always query the whole review document and not just the count. If you feel like this might be a problem, you might want to first add the .lean() function to your query, and if that doesn't help much, you can resort to one of the first two approaches I outlined.
The two types of objects seem to be so close to one another that having both feels redundant. What is the point of having both schemas and models?
EDIT: Although this has been useful for many people, as mentioned in the comments it answers the "how" rather than the why. Thankfully, the why of the question has been answered elsewhere also, with this answer to another question. This has been linked in the comments for some time but I realise that many may not get that far when reading.
Often the easiest way to answer this type of question is with an example. In this case, someone has already done it for me :)
Take a look here:
http://rawberg.com/blog/nodejs/mongoose-orm-nested-models/
EDIT: The original post (as mentioned in the comments) seems to no longer exist, so I am reproducing it below. Should it ever return, or if it has just moved, please let me know.
It gives a decent description of using schemas within models in mongoose and why you would want to do it, and also shows you how to push tasks via the model while the schema is all about the structure etc.
Original Post:
Let’s start with a simple example of embedding a schema inside a model.
var TaskSchema = new Schema({
name: String,
priority: Number
});
TaskSchema.virtual('nameandpriority')
.get( function () {
return this.name + '(' + this.priority + ')';
});
TaskSchema.method('isHighPriority', function() {
if(this.priority === 1) {
return true;
} else {
return false;
}
});
var ListSchema = new Schema({
name: String,
tasks: [TaskSchema]
});
mongoose.model('List', ListSchema);
var List = mongoose.model('List');
var sampleList = new List({name:'Sample List'});
I created a new TaskSchema object with basic info a task might have. A Mongoose virtual attribute is setup to conveniently combine the name and priority of the Task. I only specified a getter here but virtual setters are supported as well.
I also defined a simple task method called isHighPriority to demonstrate how methods work with this setup.
In the ListSchema definition you’ll notice how the tasks key is configured to hold an array of TaskSchema objects. The task key will become an instance of DocumentArray which provides special methods for dealing with embedded Mongo documents.
For now I only passed the ListSchema object into mongoose.model and left the TaskSchema out. Technically it's not necessary to turn the TaskSchema into a formal model since we won’t be saving it in it’s own collection. Later on I’ll show you how it doesn’t harm anything if you do and it can help to organize all your models in the same way especially when they start spanning multiple files.
With the List model setup let’s add a couple tasks to it and save them to Mongo.
var List = mongoose.model('List');
var sampleList = new List({name:'Sample List'});
sampleList.tasks.push(
{name:'task one', priority:1},
{name:'task two', priority:5}
);
sampleList.save(function(err) {
if (err) {
console.log('error adding new list');
console.log(err);
} else {
console.log('new list successfully saved');
}
});
The tasks attribute on the instance of our List model (sampleList) works like a regular JavaScript array and we can add new tasks to it using push. The important thing to notice is the tasks are added as regular JavaScript objects. It’s a subtle distinction that may not be immediately intuitive.
You can verify from the Mongo shell that the new list and tasks were saved to mongo.
db.lists.find()
{ "tasks" : [
{
"_id" : ObjectId("4dd1cbeed77909f507000002"),
"priority" : 1,
"name" : "task one"
},
{
"_id" : ObjectId("4dd1cbeed77909f507000003"),
"priority" : 5,
"name" : "task two"
}
], "_id" : ObjectId("4dd1cbeed77909f507000001"), "name" : "Sample List" }
Now we can use the ObjectId to pull up the Sample List and iterate through its tasks.
List.findById('4dd1cbeed77909f507000001', function(err, list) {
console.log(list.name + ' retrieved');
list.tasks.forEach(function(task, index, array) {
console.log(task.name);
console.log(task.nameandpriority);
console.log(task.isHighPriority());
});
});
If you run that last bit of code you’ll get an error saying the embedded document doesn’t have a method isHighPriority. In the current version of Mongoose you can’t access methods on embedded schemas directly. There’s an open ticket to fix it and after posing the question to the Mongoose Google Group, manimal45 posted a helpful work-around to use for now.
List.findById('4dd1cbeed77909f507000001', function(err, list) {
console.log(list.name + ' retrieved');
list.tasks.forEach(function(task, index, array) {
console.log(task.name);
console.log(task.nameandpriority);
console.log(task._schema.methods.isHighPriority.apply(task));
});
});
If you run that code you should see the following output on the command line.
Sample List retrieved
task one
task one (1)
true
task two
task two (5)
false
With that work-around in mind let’s turn the TaskSchema into a Mongoose model.
mongoose.model('Task', TaskSchema);
var Task = mongoose.model('Task');
var ListSchema = new Schema({
name: String,
tasks: [Task.schema]
});
mongoose.model('List', ListSchema);
var List = mongoose.model('List');
The TaskSchema definition is the same as before so I left it out. Once its turned into a model we can still access it’s underlying Schema object using dot notation.
Let’s create a new list and embed two Task model instances within it.
var demoList = new List({name:'Demo List'});
var taskThree = new Task({name:'task three', priority:10});
var taskFour = new Task({name:'task four', priority:11});
demoList.tasks.push(taskThree.toObject(), taskFour.toObject());
demoList.save(function(err) {
if (err) {
console.log('error adding new list');
console.log(err);
} else {
console.log('new list successfully saved');
}
});
As we’re embedding the Task model instances into the List we’re calling toObject on them to convert their data into plain JavaScript objects that the List.tasks DocumentArray is expecting. When you save model instances this way your embedded documents will contain ObjectIds.
The complete code example is available as a gist. Hopefully these work-arounds help smooth things over as Mongoose continues to develop. I’m still pretty new to Mongoose and MongoDB so please feel free to share better solutions and tips in the comments. Happy data modeling!
Schema is an object that defines the structure of any documents that will be stored in your MongoDB collection; it enables you to define types and validators for all of your data items.
Model is an object that gives you easy access to a named collection, allowing you to query the collection and use the Schema to validate any documents you save to that collection. It is created by combining a Schema, a Connection, and a collection name.
Originally phrased by Valeri Karpov, MongoDB Blog
I don't think the accepted answer actually answers the question that was posed. The answer doesn't explain why Mongoose has decided to require a developer to provide both a Schema and a Model variable. An example of a framework where they have eliminated the need for the developer to define the data schema is django--a developer writes up their models in the models.py file, and leaves it to the framework to manage the schema. The first reason that comes to mind for why they do this, given my experience with django, is ease-of-use. Perhaps more importantly is the DRY (don't repeat yourself) principle--you don't have to remember to update the schema when you change the model--django will do it for you! Rails also manages the schema of the data for you--a developer doesn't edit the schema directly, but changes it by defining migrations that manipulate the schema.
One reason I could understand that Mongoose would separate the schema and the model is instances where you would want to build a model from two schemas. Such a scenario might introduce more complexity than is worth managing--if you have two schemas that are managed by one model, why aren't they one schema?
Perhaps the original question is more a relic of the traditional relational database system. In world NoSQL/Mongo world, perhaps the schema is a little more flexible than MySQL/PostgreSQL, and thus changing the schema is more common practice.
To understand why? you have to understand what actually is Mongoose?
Well, the mongoose is an object data modeling library for MongoDB and Node JS, providing a higher level of abstraction. So it's a bit like the relationship between Express and Node, so Express is a layer of abstraction over regular Node, while Mongoose is a layer of abstraction over the regular MongoDB driver.
An object data modeling library is just a way for us to write Javascript code that will then interact with a database. So we could just use a regular MongoDB driver to access our database, it would work just fine.
But instead we use Mongoose because it gives us a lot more functionality out of the box, allowing for faster and simpler development of our applications.
So, some of the features Mongoose gives us schemas to model our data and relationship, easy data validation, a simple query API, middleware, and much more.
In Mongoose, a schema is where we model our data, where we describe the structure of the data, default values, and validation, then we take that schema and create a model out of it, a model is basically a wrapper around the schema, which allows us to actually interface with the database in order to create, delete, update, and read documents.
Let's create a model from a schema.
const tourSchema = new mongoose.Schema({
name: {
type: String,
required: [true, 'A tour must have a name'],
unique: true,
},
rating: {
type: Number,
default: 4.5,
},
price: {
type: Number,
required: [true, 'A tour must have a price'],
},
});
//tour model
const Tour = mongoose.model('Tour', tourSchema);
According to convetion first letter of a model name must be capitalized.
Let's create instance of our model that we created using mongoose and schema. also, interact with our database.
const testTour = new Tour({ // instance of our model
name: 'The Forest Hiker',
rating: 4.7,
price: 497,
});
// saving testTour document into database
testTour
.save()
.then((doc) => {
console.log(doc);
})
.catch((err) => {
console.log(err);
});
So having both schama and modle mongoose makes our life easier.
Think of Model as a wrapper to schemas. Schemas define the structure of your document , what kind of properties can you expect and what will be their data type (String,Number etc.). Models provide a kind of interface to perform CRUD on schema. See this post on FCC.
Schema basically models your data (where you provide datatypes for your fields) and can do some validations on your data. It mainly deals with the structure of your collection.
Whereas the model is a wrapper around your schema to provide you with CRUD methods on collections. It mainly deals with adding/querying the database.
Having both schema and model could appear redundant when compared to other frameworks like Django (which provides only a Model) or SQL (where we create only Schemas and write SQL queries and there is no concept of model). But, this is just the way Mongoose implements it.
I have just started working with Mongo Db using Mongoose.js library.
I have two schemas, Orders and Payments.
I want to make a simple query to get all orders with property of sum of all payments for that order.
Schemas
Order
{ name : String }
Payment
{ date : Date, order : Schema.ObjectId, sum : Number }
In mysql I would do it this way:
select *,(select SUM(sum) from payments where payments.order = orders.id group by payments.order) as sum from orders
I was looking for aggregation methods to build the query. But all examples I have found is for the single object. I have no trouble to get the sum for single order (using findOne() and aggregation). But how I do it for array of orders?
There is no way to do this with just one query in MongoDB with the schema design you have. Your schema is fine for a relational DB, but it might not be ideal for NoSQL.
With your schema you would have to do two queries:
An aggregation job on payment to sum the payments grouped by order
A a find(),or findOne(), on order to get the name of the order
An alternative schema would be to have just one collection of order documents and storing the payments as sub-documents on the order document, eg. an order would look something like this:
{
name: "Foo"
payments: [ { date: xxxx, sum: 42.00}, { date: yyyy, sum: 12.00} ]
}
Take a look at the Mongo documentation on data models for more: http://docs.mongodb.org/manual/data-modeling/#data-models
Occasionally I get a document with an _id value which javascript could and does interpret as scientific notation.
ONLY illustrate the problem I ran the following query.
db.users.find({$where:'this._id > 1'}).count()
2
There are hundreds of docs in this collection, but those 2 that evaluate as numbers cause problems when they get used in {$in:[]} clauses.
db.users.findOne({$where:'this._id > 1'})._id
ObjectId("5225141850742e0000002331") - see it's looks like scientific notation right?
I think I run into trouble when I want to store that _id as a string in another collection say as
friendsCollection:
{
_uid:"5225141850742e0000002331"
//more properties here
}
When I retrieve that value, Node (or Mongoose) interprets it as a number like "Infinity". Then my code ends up trying to search for the user with {_id:{$in:[Infinity]}} which throws an error.
I'm guessing there's a more robust way to store _id values or to handle properties you know to be _ids, but I don't know how.
Converting from hex string to binary representation of ObjectID
If you want to convert from a 24-byte hex string representation of an _id value into a binary ObjectID as stored in MongoDB, you can use ObjectID.createFromHexString:
// Need to require ObjectID class if not already included
ObjectID = require('mongodb').ObjectID;
var uid = ObjectID.createFromHexString("5205c4bd7c21105d0d99648c")
Comparing ObjectIDs
You should be using the $gt operator for ObjectID comparison rather than $where. The $where operator evaluates a JavaScript string and cannot take advantage of indexes; it will be much less performant (particularly for this use case).
So the findOne() example to find an _id greater than a given ObjectID should instead be:
db.users.findOne(
{ _id: { $gt: ObjectID("5205c4bd7c21105d0d99648c") } }
)._id
For a predictable outcome on finding the next higher ObjectID you should specify an explicit sort order using find() with sort and limit:
// Find next _id greater than ObjectID("5205c4bd7c21105d0d99648c")
db.users.find(
{_id: { $gt: ObjectID("5205c4bd7c21105d0d99648c") } }
).sort({_id:1}).limit(1).toArray()[0]._id
You'll notice that these find examples doesn't explicitly call createFromHexString. The default ObjectID() constructor will try to create an appropriate ObjectID based on whether the given value is a 24 byte hex string, 12 byte binary string, or a Number. If you know what sort of value you are providing, it is better to call the expected constructor to limit unexpected conversions (for example if you accidentally provided a Number instead of a hex string).
Database References (DBRefs)
MongoDB explicitly does not support joins, however there is a convention for storing database references (DBRefs) when you want to store the _id of a related document as a reference in another document. Mongoose has a ref option that simplifies working with references; see 'Population' in the Mongoose docs.
At some point I had problems when querying _id using the native driver. I fixed it by using ObjectId in my queries. You might find this helpful:
var ObjectId = require("mongodb").ObjectID;
query._id = { $gt: ObjectId(idString) }
Ahhh, maybe I should use the Mongoose data type "ObjectId" in my schemas.
For example I was using mongoose schemas like this:
locations:{
_uid:{type:String},//<--probably not good
lat:{type:Number},
lng:{type:Number}
},
Started using the Schema type "ObjectId":
locations:{
_uid:Schema.Types.ObjectId,//<--better?
lat: {type:Number},
lng: {type:Number}
},
I have to rebuild a lot of data to support this change. Also, I won't be sure until another one of those scientific notation _ids pop up, (I deleted the offending docs). However, this seams promising.