I am developing an app where a user could store his model on a database using mongoDB and mongoose. Taken from mongoose tutorial the type of the field has to be defined. For example here we have to define that the name is a string.
const personSchema = new mongoose.Schema({
name: String
});
const Person = mongoose.model('Person', personSchema);
Is there any way to make it dynamic to user's input. I want to create a form where a user will enter a field name and select one of the field types that Mongoose offers [String,Number,Date etc], but I cannot figure any way to implement it. To be honest I don't know even if this is a good approach. An alternative would be to pass everything as a String and serialise the input in order to store it. I want to achieve something like that:
const {fieldName,fieldType} = userInput;
const customSchema = new mongoose.Schema({
fieldName: fieldType
});
const CustomModel = mongoose.model('CustomSchema', customSchema);
Is this possible or should I implement another approach? An alternative would be to pass everything as a String and serialise the input in order to store it.
Thank you in advance!
If I understand you correctly it should work like that:
User defines the model to store
Schema is created using the data provided by the user
User can pass the data to store using the previously created model which will validate the user's input later
In fact, I'm working on a project that has the same functionality. Here is how we did it.
A user sends the model and we store it as a string since we need to have the ability to create the model once again.
When the user passes new data to store using the created model we get the string from mongo and parse it to create the schema. This operation is relatively easy (but depends on what you want to achieve as it can get tricky if you want to have some advanced validation) as you have to just create an object with correct values from mongoose. Something like this for every field that the user has defined.
export const fieldConverter = ({name, type}) => {
switch (type) {
case 'String':
return { [name]: String };
case 'Number':
return { [name]: Number };
...
}
When you have your object ready then you can create a model out of it.
The line with accessing your model from mongoose.models is important as the mongoose will cache the model and throw an error if you try to create it once again.
const DatasetModel =
mongoose.models["your-model-name"] ??
mongoose.model("your-model-name", new mongoose.Schema(schema));
Now when you have the model the rest is just like with the normally created one.
This approach worked for us so I'm adding this as inspiration maybe it will help you. If you have any specific questions about the implementation feel free to ask I will be happy to help.
There is also a Mixed type in mongoose if you don't need the validation later. You can check it here: https://mongoosejs.com/docs/schematypes.html#mixed
You can use Schema.Types.Mixed, An "anything goes" SchemaType. Mongoose will not do any casting on mixed paths.
let customSchema = new Schema({custom: Schema.Types.Mixed})
Read more about it here
After some research I figure at that mongoose type can also be strings. For example
const personSchema = new mongoose.Schema({
name: "String"
});
const Person = mongoose.model('Person', personSchema);
Mongoose will handle it
I am not sure if this type of question has been asked before, but I was not able to find anything related to this. In my current project we use Joi schemas to perform the validations. I like the ability to define custom schemas and run validations on the incoming objects using that schema. I have a task where I need to filter out object properties. Something similar to _.pick but the properties are complex and deal with nested objets and arrays. We already have a joi schemas that we have designed to perform validations but I am thinking of using the same to get the specific properties of the object, like filtering object data using that schema. Something like this:
const Joi = require('joi');
const val = {
a: 'test-val1',
b: 'test-val2'
}
const schema = Joi.object({
a: Joi.string()
});
// now the below result have the object with both `a` and `b`
// properties but I want joi to strip the `b` property from the object
const result = schema.validate(value, { allowUnknown: true });
Joi's documentation doesn't mention anything like this. I have come across this(ajv) library which does do what I want but I wanted to know for sure if this can not be achieved using Joi. Thanks in advance.
Joi offers stripUnkown property that can be used to get only the fields defined in the schema.
I am new to MongoDB & working on a MEAN application.
In the mongo database(I am using mongoose), the collections are adding dynamically from third party API like schoolList1,schoolList2,schoolList3,schoolList4,....
I am facing problem to find a solution to get data from collections, Like If a user sends the argument from FrontEnd to find data from schoolList3.
The find function should apply on that collection only & return the data.
I am unable to solve it that how should I get data without passing schema and did not get any other way.
Set collection name option for your schema from user's input:
var collectionName = 'schoolList3'; // set value from the input
var dataSchema = new Schema({/** your schema here **/}, { collection: collectionName });
The two types of objects seem to be so close to one another that having both feels redundant. What is the point of having both schemas and models?
EDIT: Although this has been useful for many people, as mentioned in the comments it answers the "how" rather than the why. Thankfully, the why of the question has been answered elsewhere also, with this answer to another question. This has been linked in the comments for some time but I realise that many may not get that far when reading.
Often the easiest way to answer this type of question is with an example. In this case, someone has already done it for me :)
Take a look here:
http://rawberg.com/blog/nodejs/mongoose-orm-nested-models/
EDIT: The original post (as mentioned in the comments) seems to no longer exist, so I am reproducing it below. Should it ever return, or if it has just moved, please let me know.
It gives a decent description of using schemas within models in mongoose and why you would want to do it, and also shows you how to push tasks via the model while the schema is all about the structure etc.
Original Post:
Let’s start with a simple example of embedding a schema inside a model.
var TaskSchema = new Schema({
name: String,
priority: Number
});
TaskSchema.virtual('nameandpriority')
.get( function () {
return this.name + '(' + this.priority + ')';
});
TaskSchema.method('isHighPriority', function() {
if(this.priority === 1) {
return true;
} else {
return false;
}
});
var ListSchema = new Schema({
name: String,
tasks: [TaskSchema]
});
mongoose.model('List', ListSchema);
var List = mongoose.model('List');
var sampleList = new List({name:'Sample List'});
I created a new TaskSchema object with basic info a task might have. A Mongoose virtual attribute is setup to conveniently combine the name and priority of the Task. I only specified a getter here but virtual setters are supported as well.
I also defined a simple task method called isHighPriority to demonstrate how methods work with this setup.
In the ListSchema definition you’ll notice how the tasks key is configured to hold an array of TaskSchema objects. The task key will become an instance of DocumentArray which provides special methods for dealing with embedded Mongo documents.
For now I only passed the ListSchema object into mongoose.model and left the TaskSchema out. Technically it's not necessary to turn the TaskSchema into a formal model since we won’t be saving it in it’s own collection. Later on I’ll show you how it doesn’t harm anything if you do and it can help to organize all your models in the same way especially when they start spanning multiple files.
With the List model setup let’s add a couple tasks to it and save them to Mongo.
var List = mongoose.model('List');
var sampleList = new List({name:'Sample List'});
sampleList.tasks.push(
{name:'task one', priority:1},
{name:'task two', priority:5}
);
sampleList.save(function(err) {
if (err) {
console.log('error adding new list');
console.log(err);
} else {
console.log('new list successfully saved');
}
});
The tasks attribute on the instance of our List model (sampleList) works like a regular JavaScript array and we can add new tasks to it using push. The important thing to notice is the tasks are added as regular JavaScript objects. It’s a subtle distinction that may not be immediately intuitive.
You can verify from the Mongo shell that the new list and tasks were saved to mongo.
db.lists.find()
{ "tasks" : [
{
"_id" : ObjectId("4dd1cbeed77909f507000002"),
"priority" : 1,
"name" : "task one"
},
{
"_id" : ObjectId("4dd1cbeed77909f507000003"),
"priority" : 5,
"name" : "task two"
}
], "_id" : ObjectId("4dd1cbeed77909f507000001"), "name" : "Sample List" }
Now we can use the ObjectId to pull up the Sample List and iterate through its tasks.
List.findById('4dd1cbeed77909f507000001', function(err, list) {
console.log(list.name + ' retrieved');
list.tasks.forEach(function(task, index, array) {
console.log(task.name);
console.log(task.nameandpriority);
console.log(task.isHighPriority());
});
});
If you run that last bit of code you’ll get an error saying the embedded document doesn’t have a method isHighPriority. In the current version of Mongoose you can’t access methods on embedded schemas directly. There’s an open ticket to fix it and after posing the question to the Mongoose Google Group, manimal45 posted a helpful work-around to use for now.
List.findById('4dd1cbeed77909f507000001', function(err, list) {
console.log(list.name + ' retrieved');
list.tasks.forEach(function(task, index, array) {
console.log(task.name);
console.log(task.nameandpriority);
console.log(task._schema.methods.isHighPriority.apply(task));
});
});
If you run that code you should see the following output on the command line.
Sample List retrieved
task one
task one (1)
true
task two
task two (5)
false
With that work-around in mind let’s turn the TaskSchema into a Mongoose model.
mongoose.model('Task', TaskSchema);
var Task = mongoose.model('Task');
var ListSchema = new Schema({
name: String,
tasks: [Task.schema]
});
mongoose.model('List', ListSchema);
var List = mongoose.model('List');
The TaskSchema definition is the same as before so I left it out. Once its turned into a model we can still access it’s underlying Schema object using dot notation.
Let’s create a new list and embed two Task model instances within it.
var demoList = new List({name:'Demo List'});
var taskThree = new Task({name:'task three', priority:10});
var taskFour = new Task({name:'task four', priority:11});
demoList.tasks.push(taskThree.toObject(), taskFour.toObject());
demoList.save(function(err) {
if (err) {
console.log('error adding new list');
console.log(err);
} else {
console.log('new list successfully saved');
}
});
As we’re embedding the Task model instances into the List we’re calling toObject on them to convert their data into plain JavaScript objects that the List.tasks DocumentArray is expecting. When you save model instances this way your embedded documents will contain ObjectIds.
The complete code example is available as a gist. Hopefully these work-arounds help smooth things over as Mongoose continues to develop. I’m still pretty new to Mongoose and MongoDB so please feel free to share better solutions and tips in the comments. Happy data modeling!
Schema is an object that defines the structure of any documents that will be stored in your MongoDB collection; it enables you to define types and validators for all of your data items.
Model is an object that gives you easy access to a named collection, allowing you to query the collection and use the Schema to validate any documents you save to that collection. It is created by combining a Schema, a Connection, and a collection name.
Originally phrased by Valeri Karpov, MongoDB Blog
I don't think the accepted answer actually answers the question that was posed. The answer doesn't explain why Mongoose has decided to require a developer to provide both a Schema and a Model variable. An example of a framework where they have eliminated the need for the developer to define the data schema is django--a developer writes up their models in the models.py file, and leaves it to the framework to manage the schema. The first reason that comes to mind for why they do this, given my experience with django, is ease-of-use. Perhaps more importantly is the DRY (don't repeat yourself) principle--you don't have to remember to update the schema when you change the model--django will do it for you! Rails also manages the schema of the data for you--a developer doesn't edit the schema directly, but changes it by defining migrations that manipulate the schema.
One reason I could understand that Mongoose would separate the schema and the model is instances where you would want to build a model from two schemas. Such a scenario might introduce more complexity than is worth managing--if you have two schemas that are managed by one model, why aren't they one schema?
Perhaps the original question is more a relic of the traditional relational database system. In world NoSQL/Mongo world, perhaps the schema is a little more flexible than MySQL/PostgreSQL, and thus changing the schema is more common practice.
To understand why? you have to understand what actually is Mongoose?
Well, the mongoose is an object data modeling library for MongoDB and Node JS, providing a higher level of abstraction. So it's a bit like the relationship between Express and Node, so Express is a layer of abstraction over regular Node, while Mongoose is a layer of abstraction over the regular MongoDB driver.
An object data modeling library is just a way for us to write Javascript code that will then interact with a database. So we could just use a regular MongoDB driver to access our database, it would work just fine.
But instead we use Mongoose because it gives us a lot more functionality out of the box, allowing for faster and simpler development of our applications.
So, some of the features Mongoose gives us schemas to model our data and relationship, easy data validation, a simple query API, middleware, and much more.
In Mongoose, a schema is where we model our data, where we describe the structure of the data, default values, and validation, then we take that schema and create a model out of it, a model is basically a wrapper around the schema, which allows us to actually interface with the database in order to create, delete, update, and read documents.
Let's create a model from a schema.
const tourSchema = new mongoose.Schema({
name: {
type: String,
required: [true, 'A tour must have a name'],
unique: true,
},
rating: {
type: Number,
default: 4.5,
},
price: {
type: Number,
required: [true, 'A tour must have a price'],
},
});
//tour model
const Tour = mongoose.model('Tour', tourSchema);
According to convetion first letter of a model name must be capitalized.
Let's create instance of our model that we created using mongoose and schema. also, interact with our database.
const testTour = new Tour({ // instance of our model
name: 'The Forest Hiker',
rating: 4.7,
price: 497,
});
// saving testTour document into database
testTour
.save()
.then((doc) => {
console.log(doc);
})
.catch((err) => {
console.log(err);
});
So having both schama and modle mongoose makes our life easier.
Think of Model as a wrapper to schemas. Schemas define the structure of your document , what kind of properties can you expect and what will be their data type (String,Number etc.). Models provide a kind of interface to perform CRUD on schema. See this post on FCC.
Schema basically models your data (where you provide datatypes for your fields) and can do some validations on your data. It mainly deals with the structure of your collection.
Whereas the model is a wrapper around your schema to provide you with CRUD methods on collections. It mainly deals with adding/querying the database.
Having both schema and model could appear redundant when compared to other frameworks like Django (which provides only a Model) or SQL (where we create only Schemas and write SQL queries and there is no concept of model). But, this is just the way Mongoose implements it.
I know I have to define Schema's in Mongoose, but I have a case where I'm connecting to a MongoDB via
dsn = "mongodb://#{config.database.username}:#{config.database.password}##{config.database.host}/{config.database.name}"
mongoose.connect(dsn, (err) -> throw err if err)
And most of my writes will be using Models the way I'm supposed to. But there is this one read that I have to do from a Collection and it's Schema-less. Meaning, it's unprocessed data that was stored by another process. How can I successfully read from that then write to other collections using my Schemas?
If I use mongoose, can I not do this?
To start with you can just make a blank schema for it.
var OtherSchema = new Schema({}, {collection: 'your-collection-name'});
Mongoose.model('Other', OtherSchema);
// ..