Suppose I have a User Schema which has around 30 fields, and other 3 schemas also.
UserSchema.js
user_schema = new Schema({
user_id: { type: String},
.........//30 properties
});
ctrs_schema = new Schema({
.........10 properties
});
ids_schema = new Schema({
.........5 properties
});
comments_schema = new Schema({
.........10 properties
});
Now I am writing a route which will change the gender of the user, Now in order to do it I can use UserSchema.js but that will load all of the schemas into my route, whereas if I would have created a new file which had only one schema with two fields, then all schemas will not get loaded into the memory for the route.
UserGenderSchema.js
gender_schema = new Schema({
user_id: { type: String},
gender: { type: String}
});
I know there are pros and cons of both of the ways
Pros -
I have to edit only in single file if I would have to change something
for any field.
Cons -
All Schemas are Loading for all routes which are unnecesary. Memory
Wastage.
Will, there be any less memory usage between both of the ways on the threads?
Can anyone Please tell me which architecture will be better or what you are implementing in your project and why?
Thanks
It's better to keep user related fields in just one schema, cause mongo has been there because of its non-relational structure and it gained its performance by keeping relational structures away, so if you create a schema for each field and then create a reference in each of them to point out to the user they are related to, you are kind of using mongo to build a heavily relational structure and mongo is not good as it should be in this situation, so if later on your application you want to somehow show all the information of the user or try to update multiple fields of user or try to show more information of the user in one of your routes or something, you will end up having some serious performance issues. as a conclusion, the cost of loading all the schema to touch only one field is not as much as the cost of breaking down your data structure.
Related
I recently started taking a course on Node JS and Mongo DB. During the course, I created a model class:
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const IdeaSchema = new Schema({
title: {
type: String,
required: true
},
details: {
type: String,
required: true
},
date: {
type: Date,
default: Date.now
}
});
mongoose.model('ideas', IdeaSchema);
This created a 'table' (correct me if I'm wrong) in the MongoDB database, and editing the class changed the design of the table. So, what happens if I delete the class? Does the 'table' get deleted? Why? And if not, how can I delete it?
Everything in Mongoose starts with a Schema. Each schema maps to a MongoDB collection and defines the shape of the documents within that collection. It's just a blueprint of the house, deleting the Schema doesn't mean you are deleting the actual physical house or in mongoDB the collection.
You can change your schema anytime, without having any changes / effect on your old documents. You can still retrieve it and add new documents with new added fields based on the new modified schema.
To easily manage your mongoDb data, you can install tools like Robo3T see https://robomongo.org If you are new in mongo, this kind of tools will save you time and help you manage your dbs and collections easily. Everytime you change / add new schemas, you can use the tool to delete db, delete collections, insert documents, etc.
Or if you are just accessing mongo via terminal, this mongo shell commands will help you https://www.opentechguides.com/how-to/article/mongodb/118/mongodb-cheatsheat.html
For more details on mongoose schema, see https://mongoosejs.com/docs/guide.html
your table didn't get deleted. just your schema changed and added your new field in same table.
I have to make a page for entering some data about people on specific regions. Each user will have prvileges on a specific region.
So I will make two collections into a DB. One for the users and one for the regions.
The users will have access only to one region (or in some cases to more) and specific acceses, like read or write, or read&write, and so on.
How can I model my schema so I can add more regions?
That's how I did it:
var mongoose = require("mongoose"),
passportLocalMongoose = require("passport-local-mongoose");
let userSchema = new mongoose.Schema({
username:
{type: String,
unique: true
},
password: String,
privileges:
{
regiune: [Number],
read: [Number],
write: [Number],
edit: [Number]
}
});
userSchema.plugin(passportLocalMongoose);
module.exports = mongoose.model("User", userSchema);
but I don't know how I should edit this model so it can have multiple regions with different privileges.
For regular users they might have one region with read, write and edit, let's say. But if I want another user to have access to region x with read, write privileges and region y with write privileges I won't be able to do it using this schema.
So how should I edit this? (also, I'm a beginner in DBs so if there is another better way to set privileges please let me know)
Thanks!
Actually, since the privileges field belongs to each user, you can easily have multiple regions with different privileges
Turn privileges into an array and each particular permission into a number and you have it!
privileges:
[{
regiune: Number,
read: Number,
write: Number,
edit: Number
}]
An array of objects is the idiomatic way to model dependent objects in Mongo. People generally recommend not having an unbounded array as subdocument, but I bet your list of regions is relatively finite.
You will probably want to introduce some mechanism for ensuring that you have unique regiunes in the array. Mongo 4.10 introduced a plugin for this: https://www.npmjs.com/package/mongoose-unique-array that you can use like this:
privileges:
[{
regiune: { type: Number, unique: true },
read: Number,
write: Number,
edit: Number
}]
If you wanted to access your permissions as an object, by key: region1, region3, etc. Then you might declare privileges as a mixed type, meaning it can be any arbitrary json structure (but you lose type information). Or you might make it a separate model, collection with a back ref to the user object. The user object could also have a list of references to this new model (but it would again be an array and not an object).
I have a Node.js API method that creates a Track document and then creates Task document (which has DBRef to a Track to work on) to actually convert and upload it to s3 (specific operations doesn't really matter). I use Mongoose to perform operations on my MongoDB.
So my question is there any way to avoid having "stalled" tracks that have Track document created and no Task for them? Creating a Task without Track is fine though, but in that case we don't have DBRef until we create a Track document and then update a Task document which may fail due to server restart or something.
In traditional RDBMS that kind of problem is easy to solve since I would do that in a single transaction.
Thanks in advance.
Update:
My schemas look like that:
var Track = new Schema({
name: String,
...
});
var Task = new Schema({
track: {type: Schema.Types.ObjectId, ref: 'Track'},
created: {type: Date, default: Date.now}
});
What is the best way to propagate updates when you have a denormalized Schema? Should it be all done in the same function?
I have a schema like so:
var Authors = new Schema({
...
name: {type: String, required:true},
period: {type: Schema.Types.ObjectId, ref:'Periods'},
quotes: [{type: Schema.Types.ObjectId, ref: 'Quotes'}]
active: Boolean,
...
})
Then:
var Periods = new Schema({
...
name: {type: String, required:true},
authors: [{type: Schema.Types.ObjectId, ref:'Authors'}],
active: Boolean,
...
})
Now say I want to denormalize Authors, since the period field will always just use the name of the period (which is unique, there can't be two periods with the same name). Say then that I turn my schema into this:
var Authors = new Schema({
...
name: {type: String, required:true},
period: String, //no longer a ref
active: Boolean,
...
})
Now Mongoose doesn't know anymore that the period field is connected to the Period schema. So it's up to me to update the field when the name of a period changes. I created a service module that offers an interface like this:
exports.updatePeriod = function(id, changes) {...}
Within this function I go through the changes to update the period document that needs to be updated. So here's my question. Should I, then, update all authors within this method? Because then the method would have to know about the Author schema and any other schema that uses period, creating a lot of coupling between these entities. Is there a better way?
Perhaps I can emit an event that a period has been updated and all the schemas that have denormalized period references can observe it, is that a better solution? I'm not quite sure how to approach this issue.
Ok, while I wait for a better answer than my own, I will try to post what I have been doing so far.
Pre/Post Middleware
The first thing I tried was to use the pre/post middlewares to synchronize documents that referenced each other. (For instance, if you have Author and Quote, and an Author has an array of the type: quotes: [{type: Schema.Types.ObjectId, ref:'Quotes'}], then whenever a Quote is deleted, you'd have to remove its _id from the array. Or if the Author is removed, you may want all his quotes removed).
This approach has an important advantage: if you define each Schema in its own file, you can define the middleware there and have it all neatly organized. Whenever you look at the schema, right below you can see what it does, how its changes affect other entities, etc:
var Quote = new Schema({
//fields in schema
})
//its quite clear what happens when you remove an entity
Quote.pre('remove', function(next) {
Author.update(
//remove quote from Author quotes array.
)
})
The main disadvantage however is that these hooks are not executed when you call update or any Model static updating/removing functions. Rather you need to retrieve the document and then call save() or remove() on them.
Another smaller disadvantage is that Quote now needs to be aware of anyone that references it, so that it can update them whenever a Quote is updated or removed. So let's say that a Period has a list of quotes, and Author has a list of quotes as well, Quote will need to know about these two to update them.
The reason for this is that these functions send atomic queries to the database directly. While this is nice, I hate the inconsistency between using save() and Model.Update(...). Maybe somebody else or you in the future accidently use the static update functions and your middleware isn't triggered, giving you headaches that you struggle to get rid of.
NodeJS Event Mechanisms
What I am currently doing is not really optimal but it offers me enough benefits to actually outweight the cons (Or so I believe, if anyone cares to give me some feedback that'd be great). I created a service that wraps around a model, say AuthorService that extends events.EventEmitter and is a Constructor function that will look roughly like this:
function AuthorService() {
var self = this
this.create = function() {...}
this.update = function() {
...
self.emit('AuthorUpdated, before, after)
...
}
}
util.inherits(AuthorService, events.EventEmitter)
module.exports = new AuthorService()
The advantages:
Any interested function can register to the Service
events and be notified. That way, for instance, when a Quote is
updated, the AuthorService can listen to it and update the Authors
accordingly. (Note 1)
Quote doesn't need to be aware of all the documents that reference it, the Service simply triggers the QuoteUpdated event and all the documents that need to perform operations when this happens will do so.
Note 1: As long as this service is used whenever anyone needs to interact with mongoose.
The disadvantages:
Added boilerplate code, using a service instead of mongoose directly.
Now it isn't exactly obvious what functions get called when you
trigger the event.
You decouple producer and consumer at the cost of legibility (since
you just emit('EventName', args), it's not immediately obvious
which Services are listening to this event)
Another disadvantage is that someone can retrieve a Model from the Service and call save(), in which the events won't be triggered though I'm sure this could be addressed with some kind of hybrid between these two solutions.
I am very open to suggestions in this field (which is why I posted this question in the first place).
I'm gonna speak more from an architectural point of view than a coding point of view since when it comes right down to it, you can pretty-much achieve anything with enough lines of code.
As far as I've been able to understand, your main concern has been keeping consistency across your database, mainly removing documents when their references are removed and vice-versa.
So in this case, rather than wrapping the whole functionality in extra code I'd suggest going for atomic Actions, where an Action is a method you define yourself that performs a complete removal of an entity from the DB (both document and reference).
So for example when you wanna remove an author's quote, you do something like removing the Quote document from the DB and then removing the reference from the Author document.
This sort of architecture ensures that each of these Actions performs a single task and performs it well, without having to tap into events (emitting, consuming) or any other stuff. It's a self-contained method for performing its own unique task.
Hi is there a way to repopulate a mongo collection from a json array using mongoose?
I have this schema.
var UserNote_Schema = new Schema({
user_id: {type:ObjectId, ref:require('../User/User_Schema.js')},
note: {type:String},
created: {type:Date},
last_mod: {type:Date}
});
and would like to populate it with this json and have it save to the collection.
{"_id":"506b8922a6bd97d322000011",
"user_id":"504aaeef898f541527000012",
"note":"a note","created":"2012-03-10T00:38:58.000Z",
"last_mod":"2012-03-10T00:38:58.000Z"}
My goal is to reset my database to a certain state after running some tests which add remove and change some of the data within.
However, if there is more convenient way of achieving a database reset from a previous state in mongoose/mongo I'd much appreciate any pointers. Keeping the versioning info would also be nice.
thanks