I'm new to mongodb and I'm working on my personal project which is a project management system for college projects. I have 3 users admin(the department), students, and advisor. The admin is the one who registers both the students and the advisors. All the users have some common fields like name, email, and password. And different fields of there own.
Their roles:-
- Admin -> adds the students and advisors
- Student -> choose projects and work on the projects in a team
- Advisor -> advice students based on their progress on their project
My problem is in designing the model should I use one userSchema and add all the users in one collection or create different collections for each user.
My second confusion is if I create different collections for each user how can I deal with authentication and authorization? I'm using node js for the backend.
Can I get some guidance and suggestion?
It's your choice.
Here, I would suggest to use a single Collection for all the 3 types of Users.
Since everyone would be having same functionalities like name, email, password etc. on registration, single Schema would work for sure.
So create on schema of suppose 'User' and then use one 'tag' selector to identify the admin, advisor and student.
I would do something like this:
var userSchema = new mongoose.Schema({
email: {
type: String,
unique: true,
required: true,
},
name: {
type: String,
required: true,
},
tag : {
type: String
},
Addtasks: [
{
topic: String,
words: Number,
keywords: String,
website: String,
otherdetails: String,
exampleRadios: String,
deadline: Date,
Date: String,
fileName: String
},
],
});
module.exports = mongoose.model('User', userSchema);
With this create one page for authentication as auth.js separately and write the single authentication code there using passport module authentication. With that said now you can use one authentication validation for all 3 dashboards.
For handlebars as front-end use this to check the user if he/she has logged in or not.
{{if #user}}
<html>
<head>
.
.
.
...
following this you can achieve this.
Related
Suppose the following User Schema in MongoDB (using Mongoose/Nodejs):
var UserSchema = new Schema({
email: {
type: String,
unique: true,
required: 'User email is required.'
},
password: {
type: String,
required: 'User password is required.'
},
token: {
type: String,
unique: true,
default: hat
},
created_at: {
type: Date,
default: Date.now
},
});
// mongoose-encrypt package
UserSchema.plugin(encrypt, {
secret: 'my secret',
encryptedFields: ['email', 'password', 'token', 'created_at']
});
Now assume I want to return the user object from an API endpoint. In fact, suppose I want to return user objects from multiple API endpoints. Possibly as a standalone object, possibly as a related model.
Obviously, I don't want password to be present in the returned structure - and in many cases I wouldn't want token to be returned either. I could do this manually on every endpoint, but I'd prefer a no-thought solution - being able to simply retrieve the user, end of story, and not worry about unsetting certain values after the fact.
I mainly come from the world of Laravel, where things like API Resources (https://laravel.com/docs/5.6/eloquent-resources) exist. I already tried implementing the mongoose-hidden package (https://www.npmjs.com/package/mongoose-hidden) to hide the password and token, but unfortunately it seems as though that breaks the encryption package I'm using.
I'm new to Nodejs and MongoDB in general - is there a good way to implement this?
How to protect the password field in Mongoose/MongoDB so it won't return in a query when I populate collections?
You can use this: Users.find().select("-password"),
but this is done whenever you send the queried item to the user (res.json()...) so you can do your manipultions with this field included and then remove it from the user before you send it back (this is using the promise approach, the best practice).
And if you want your changes to be used as default you can add "select: false" into the schema object's password field.
Hope this helps :)
I have begun diving into the server side of things lately, and am working on an app where I need to think about how I plan my models.
My users are teachers, and in the dashboard will have the ability to create a list of students. My schema's will contain more directives to prevent duplicates being created, but I have simplified them here. Here's what I have attempted so far:
// Teacher Model
const Teacher = new Schema({
fname: String,
lname: String,
email: String,
})
// Student Model
const Student = new Schema({
fname: String,
lname: String,
full: String,
uuid: String
grades: {
classwork: Array,
quizzes: Array,
tests: Array
}
})
Here's where my inexperience with backend work comes into play. This setup doesn't really make sense to me. Say when I go and save a student, it will create a new student under the student collection in the database. This is not ideal, as the student should be stored in a way that is strictly accessible to the teacher who created it.
I was thinking about creating a new key in my Teachers Schema called "students"(which would be an array) that would push a student into it each time one was created.
It's definitely important that I plan this properly, as the teacher is going to have much more ability in the future, like creating assignments, grading students etc. I'd like to design this with best practices in mind, to ensure the teachers data is safe from other users.
I don't agree with #Lazyexpert. MongoDB is a non-relational database and
you can store until 16Mb of data per document. It is really enough for what you need
The maximum BSON document size is 16 megabytes. The maximum document size helps ensure that a single document cannot use excessive amount of RAM or, during transmission, excessive amount of bandwidth. To store documents larger than the maximum size, MongoDB provides the GridFS API.
i.e: https://docs.mongodb.com/manual/reference/limits/
So I suggest you just add the datas of each student directly in your teacher.
You can find some tips here : https://www.safaribooksonline.com/library/view/50-tips-and/9781449306779/ch01.html
So your model would looks something like that :
const Teacher = new Schema({
fname: String,
lname: String,
email: String,
students : [
{
fname: String,
lname: String,
full: String,
uuid: String
grades: {
classwork: Array,
quizzes: Array,
tests: Array
},
},
],
})
And if you absolutely want a collection Student as well, then use a "post" middleware on the "save" action on your student schema. Something like this :
StudentSchema.post('save', function(doc) {
Teacher.findOneAndUpdate({_id: <your teacher id>}, <your student object>, callback);
});
i.e: mongoosejs.com/docs/api.html#schema_Schema-post
Good luck :)
Using nested array in mongo model is not so smooth.
I can suggest to think about size of this array.
If there is a chance that your array can grow - don't use it.
My suggestion for your database design is simple.
Add teacherId to the student model.
This way, when you need to fetch students list according to the certain teacher - you can easily query by teacherId.
So your student schema modified would look this way:
const Student = new Schema({
teacherId: {
type: mongoose.Schema.Types.ObjectId,
index: true,
required: true
},
fname: String,
lname: String,
full: String,
uuid: String
grades: {
classwork: Array,
quizzes: Array,
tests: Array
}
});
I've looked around left and right, I wrote some demo code, wrote some tests to implement a school management system.
What I want to know from people more used to mongoose development is how would be the best practice to create this schema in a way that made it possible to add as many address, and contact fees as I want from this single document.
I made my own solution, but I don't know if it is the most elegant and feasible way, I want an opinion from seasoned people.
Should I create separate models for address, email and phone numbers?
I created this schema. It still has some pseudo-code, but for giving the general idea is fine.
var student = {
name: String,
surname: String,
currentClass: {
type: mongoose.Schema.Types.ObjectId,
ref: "Classes"
},
birthday: {
year: Number,
month: Number,
day: Number
},
address: [{
name: String,
zip: Number,
address: String,
city: String,
state: String,
complement: String
}]
accountable: {
name: String,
surname: String,
email: [{
type: String,
required: true,
lowercase: true
}],
phone: [String, String]
}
My sollution was, by using html, creating a new "email" or "address" fields as the user requested by clickinking in the propper button. This generated a new input field with a name that followed a pattern like:
email1, email2, email3, email4
And so, when the user sent the data, if we were creating a new student I would first create the array with the data and send it to mongoose.
In case of updating, I would get the already created emails and add it to the new array with the newEmails.concat(old_emails)
To design the database, there are many situations for it:
1 to 1, 1 to many, many to many.
One to one: you should to put the strong object inside the other, for example:a person can have only one passport, then we should put passport object inside the person object.
One to Many, 3 cases for one to many.
One to Few:few is less than 100 objects,Then you should add the few as list in the one object, for example:
A person can have multiple addresses as in your example above.
One to Many:many is Thousands, then you should put the primary keys of the many in a list inside the the one object.
One to Too many: then do not do the previous solution, but instead add the primary of the one in every objects of the many.
And finally, Many to Many: you should put them as list in both sides.
Check the below references:
https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-1
https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-2
https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-3
Morover, for this part phone: [String, String], you should make it phone: [String]
Here is my current Schema
Brand:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var BrandSchema = new mongoose.Schema({
name: { type: String, lowercase: true , unique: true, required: true },
photo: { type: String , trim: true},
email: { type: String , lowercase: true},
year: { type: Number},
timestamp: { type : Date, default: Date.now },
description: { type: String},
location: { },
social: {
website: {type: String},
facebook: {type: String },
twitter: {type: String },
instagram: {type: String }
}
});
Style:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var StyleSchema = new mongoose.Schema({
name: { type: String, lowercase: true , required: true},
});
Product
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var ProductSchema = new mongoose.Schema({
name: { type: String, lowercase: true , required: true},
brandId : {type: mongoose.Schema.ObjectId, ref: 'Brand'},
styleId: {type: mongoose.Schema.ObjectId, ref: 'Style'},
year: { type: Number },
avgRating: {type: Number}
});
Post:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var PostSchema = new mongoose.Schema({
rating: { type: Number},
upVote: {type: Number},
brandId : {type: mongoose.Schema.ObjectId, ref: 'Brand'},
comment: {type: String},
productId: {type: mongoose.Schema.ObjectId, ref: 'Style'},
styleId: {type: mongoose.Schema.ObjectId, ref: 'Style'},
photo: {type: String}
});
I'm currently making use of the mongoose populate feature:
exports.productsByBrand = function(req, res){
Product.find({product: req.params.id}).populate('style').exec(function(err, products){
res.send({products:products});
});
};
This works, however, being a noob --- i've started reading about performance issues with the mongoose populate, since it's really just adding an additional query.
For my post , especially, it seems that could be taxing. The intent for the post is to be a live twitter / instagram-like feed. It seems that could be a lot of queries, which could greatly slow my app down.
also, I want to be able to search prodcuts / post / brand by fields at some point.
Should i consider nesting / embedding this data (products nested / embedded in brands)?
What's the most efficient schema design or would my setup be alright -- given what i've specified I want to use it for?
User story:
There will be an Admin User.
The admin will be able to add the Brand with the specific fields in the Brand Schema.
Brands will have associated Products, each Product will have a Style / category.
Search:
Users will be able to search Brands by name and location (i'm looking into doing this with angular filtering / tags).
Users will be able to search Products by fields (name, style, etc).
Users will be able to search Post by Brand Product and Style.
Post:
Users will be able to Post into a feed. When making a Post, they will choose a Brand and a Product to associate the Post with. The Post will display the Brand name, Product name, and Style -- along with newly entered Post fields (photo, comment, and rating).
Other users can click on the Brand name to link to the Brand show page. They can click on the Product name to link to a Product show page.
Product show page:
Will show Product fields from the above Schema -- including associated Style name from Style schema. It will also display Post pertaining to the specific Product.
Brand show page:
Will simply show Brand fields and associated products.
My main worry is the Post, which will have to populate / query for the Brand , Product, and Style within a feed.
Again, I'm contemplating if I should embed the Products within the Brand -- then would I be able to associate the Brand Product and Style with the Post for later queries? Or, possibly $lookup or other aggregate features.
Mongodb itself does not support joins. So, mongoose populate is an attempt at external reference resolution. The thing with mongodb is that you need to design your data so that:
most of you queries need not to refer multiple collections.
after getting data from query, you need not to transform it too much.
Consider the entities involved, and their relations:
Brand is brand. Doesn't depend on anything else.
Every Product belongs to a Brand.
Every Product is associated with a Style.
Every Post is associated with a Product.
Indirectly, every Post is associated to a Brand and Style, via product.
Now about the use cases:
Refer: If you are looking up one entity by id, then fetching 1-2 related entities is not really a big overhead.
List: It is when you have to return a large set of objects and each object needs an additional query to get associated objects. This is a performance issue. This is usually reduced by processing "pages" of result set at a time, say 20 records per request. Lets suppose you query 20 products (using skip and limit). For 20 products you extract two id arrays, one of referred styles, and other of referred brands. You do 2 additional queries using $in:[ids], get brands and styles object and place them in result set. That's 3 queries per page. Users can request next page as they scroll down, and so on.
Search: You want to search for products, but also want to specify brand name and style name. Sadly, product model only holds ids for style and brand. Same issue with searching Posts with brand and product. Popular solution is to maintain a separate "search index", a sort of table, that stores data exactly the way it will be searched for, with all searchable fields (like brand name, style name) at one place. Maintaining such search collections in mongodb manually can be a pain. This is where ElasticSearch comes in. Since you are already using mongoose, you can simply add mongoosastic to your models. ElasticSearch's search capabilities are far greater than a DB Storage engine will offer you.
Extra Speed: There is still some room for speeding things up: Caching. Attach mongoose-redis-cache and have frequent repeated queries served, in-memory from Redis, reducing load on mongodb.
Twitter like Feeds: Now if all Posts are public then listing them up for users in chronological order is a trivial query. However things change when you introduce "social networking" features. Then you need to list "activity feeds" of friends and followers. There's some wisdom about social inboxes and Fan-out lists in mongodb blog.
Moral of the story is that not all use cases have only "db schema query" solutions. Scalability is one of such cases. That's why other tools exist.
I've a Nodejs app and I use Passeport to let my users connect throught Facebook or Google via Oauth. I'm storing basic data after connection such as : uid, firstname etc.
How can I be assured that the uid I'm receiving will be always unique for a given user ? I mean, data is coming either from Facebook or Google, so why couldn't I face a nasty duplicate uid for different users someday ?
The id you are getting via OAuth is unique when you are using either Facebook or Google. But if you want to use both then I would suggest make your dbSchema (if you are using mongoose) like this -
var userSchema = new mongoose.Schema({
local: {
username: String,
password: String
},
facebook: {
id: String,
token: String,
email: String,
name: String
},
google: {
id: String,
token: String,
email: String,
name: String
}
});
The UID is unique only within the scope of that single provider’s list of users.
This article suggests you combine the provider’s name plus the UID to get a unique value.
Combination of provider’s name and uID will uniquely identify user inside our app.
I would use the provider’s domain name such as “google.com” or “com.google”.