I've been enjoying the new Sails relationships in v0.10, but my biggest challenge currently is looking up models by their associations. If I were to have populated a manual association, say, an array of IDs, this would be pretty easy. However, I can't seem to find the right way of handling the lookups with a Sails association.
I've provided some sample code that outlines two models, a Company and a User. Companies can have many Users, and a User can have only one Company. This is a pretty straight forward one-to-many relationship, and the goal is to find all Companies that match a User ID.
## Company.js
name:
type: 'string'
required: true
users:
collection: 'User'
via: 'company'
## User.js
company:
model: 'Company'
required: true
last_name:
type: 'string'
required: true
first_name:
type: 'string'
required: true
## Lookup Users by Company ID of '2'
User.find(where: company: 2).exec(console.log)
# Result
# [] - Array of users matching that company ID
## ---- The Problem / Question ----
## Lookup Companies by User ID '1'
Company.find(where: users: contains: 1).exec(console.log)
# Result
# Error (E_UNKNOWN) :: Encountered an unexpected error:
# error: column company.users does not exist
# Details:
# { error: 'E_UNKNOWN',
# summary: 'Encountered an unexpected error',
# status: 500,
# raw: 'error: column company.users does not exist' }
I'd appreciate any thoughts on the best way to handle this lookup!
In the case of querying "all companies whose list of users contains #1", you're trying to do a subquery, which Waterline does not currently support. Furthermore, it's a slightly silly example since every user can only have one company, so you should only ever expect one result. In any case the correct method would simply be to look up User #1 and populate its company:
User.findOne(1).populate('company').exec(function(err, user) {
console.log(user.company);
});
I'll leave it to you to translate that to Coffeescript ;)
To find all of a company's users, you'd do something similar:
Company.findOne(123).populate('users').exec(...)
You can filter the populated results, but it's not the same as a subquery:
Company.findOne(123).populate('users', {where: {id: [1,2,3]}}).exec(...)
This would get you company #123 and populate its users array only with users with ID values of 1, 2 or 3.
Related
I'm new to mongodb and I'm working on my personal project which is a project management system for college projects. I have 3 users admin(the department), students, and advisor. The admin is the one who registers both the students and the advisors. All the users have some common fields like name, email, and password. And different fields of there own.
Their roles:-
- Admin -> adds the students and advisors
- Student -> choose projects and work on the projects in a team
- Advisor -> advice students based on their progress on their project
My problem is in designing the model should I use one userSchema and add all the users in one collection or create different collections for each user.
My second confusion is if I create different collections for each user how can I deal with authentication and authorization? I'm using node js for the backend.
Can I get some guidance and suggestion?
It's your choice.
Here, I would suggest to use a single Collection for all the 3 types of Users.
Since everyone would be having same functionalities like name, email, password etc. on registration, single Schema would work for sure.
So create on schema of suppose 'User' and then use one 'tag' selector to identify the admin, advisor and student.
I would do something like this:
var userSchema = new mongoose.Schema({
email: {
type: String,
unique: true,
required: true,
},
name: {
type: String,
required: true,
},
tag : {
type: String
},
Addtasks: [
{
topic: String,
words: Number,
keywords: String,
website: String,
otherdetails: String,
exampleRadios: String,
deadline: Date,
Date: String,
fileName: String
},
],
});
module.exports = mongoose.model('User', userSchema);
With this create one page for authentication as auth.js separately and write the single authentication code there using passport module authentication. With that said now you can use one authentication validation for all 3 dashboards.
For handlebars as front-end use this to check the user if he/she has logged in or not.
{{if #user}}
<html>
<head>
.
.
.
...
following this you can achieve this.
We have user and news model, in the news model we have e viewsCount field, I want to increment this view count when a GET request is made by a User.
When a specific user makes a GET request, the view count will increment one, every user just one view.
const NEWSModel = new Schema({
viewesCount: { type: Number },
Publisher: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
required: true
},
LikesCount: { type: Number },
DislikeCount: { type: Number },
Comments: CommenTs
});
Every user can view the news as many times as wants, but just can make one view. How can I do that?
you Can change your model like and then whenever you get a news just push the user id to the viewedBy field.
news.viewedBy.push(user id)
viewedBy: [{
type: mongoose.Schema.Types.ObjectId,
ref: "User"
}]
}); ```
If you have not a lot of users, you can add additional field to news model like users_viewed which would be array of unique user ids.
And make additional check before incrementing views count.
If user, who requested news is already in this users_viewed array, you skip any additional actions.
If don't, increment views counter.
But if you do have a lot of users, it's better to store views counter in Redis to skip request to database and increment in memory counter.
The logic for storing and showing data would be the same, but you'll reduce load on your database and speed up the whole process.
[UPDATE] According to your comment, about number of users.
To make things work you can use this package.
First of all, after request for a news from a client, you can store all the news data in your cache (to reduce number of requests to your database).
Now you have few possible ways to handle number of views.
I think, the easiest to implement would be to add user unique identifier to SET. And return number of users in SET using SCARD;
In this solution you wouldn't need to check if user already watched the news, because set data structure holds only unique values (the same reason why do we need to use user's unique identifier).
And you just use 2 redis requests, which is pretty good for heavy load services.
You can have another field called viewedBy of type array in which you can store users ids. Then it will be easier to check if a user already viewed your post or to count them.
File: news.model.js
const News = new Schema({
viewedBy: [{
type: mongoose.Schema.Types.ObjectId,
ref: "User"
}],
// other properties...
});
File: news.controller.js
const user = User.find({...}); // get current user
const news = News.find({...}); // get a news
/*
Update views count by adding the current user id if it's not already added
Thanks to '$addToSet', the update() function will do nothing if the user id it's already there)
*/
news.update({ $addToSet: { viewedBy: user._id } });
// Getting the views count
console.log('Total views:', news.viewedBy.length);
More about $addToSet: https://docs.mongodb.com/manual/reference/operator/update/addToSet/
I have a standard mongoose User Scheme, and a 'Uniform' scheme that holds dress items in the following way:
mongoose.Schema({
user: { type: mongoose.Schema.Types.ObjectId, ref: 'User' },
items: { type: [String], unique: true }
})
Thing is, I want the items in the 'items' array to be unique only in the scope of the user. Meaning that I don't want a user to have serval uniforms with the same item. But I don't have any problem with other users having the same items as my user.
The current scheme prevents ANY uniform to have the same values inside 'items', I want it to be just within the scope of the user.
Can that even be done?
(In ruby on rails, the line that creates this behavior is: validates_uniqueness_of :items, scope: [:user])
If User and Uniform are different schemas (different collections) then having uniqueness achieved between keys of different collections cannot be done in mongodb.
We can have uniqueness achieved by combining different keys of a same collection using Compound Index
Here is my current Schema
Brand:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var BrandSchema = new mongoose.Schema({
name: { type: String, lowercase: true , unique: true, required: true },
photo: { type: String , trim: true},
email: { type: String , lowercase: true},
year: { type: Number},
timestamp: { type : Date, default: Date.now },
description: { type: String},
location: { },
social: {
website: {type: String},
facebook: {type: String },
twitter: {type: String },
instagram: {type: String }
}
});
Style:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var StyleSchema = new mongoose.Schema({
name: { type: String, lowercase: true , required: true},
});
Product
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var ProductSchema = new mongoose.Schema({
name: { type: String, lowercase: true , required: true},
brandId : {type: mongoose.Schema.ObjectId, ref: 'Brand'},
styleId: {type: mongoose.Schema.ObjectId, ref: 'Style'},
year: { type: Number },
avgRating: {type: Number}
});
Post:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var PostSchema = new mongoose.Schema({
rating: { type: Number},
upVote: {type: Number},
brandId : {type: mongoose.Schema.ObjectId, ref: 'Brand'},
comment: {type: String},
productId: {type: mongoose.Schema.ObjectId, ref: 'Style'},
styleId: {type: mongoose.Schema.ObjectId, ref: 'Style'},
photo: {type: String}
});
I'm currently making use of the mongoose populate feature:
exports.productsByBrand = function(req, res){
Product.find({product: req.params.id}).populate('style').exec(function(err, products){
res.send({products:products});
});
};
This works, however, being a noob --- i've started reading about performance issues with the mongoose populate, since it's really just adding an additional query.
For my post , especially, it seems that could be taxing. The intent for the post is to be a live twitter / instagram-like feed. It seems that could be a lot of queries, which could greatly slow my app down.
also, I want to be able to search prodcuts / post / brand by fields at some point.
Should i consider nesting / embedding this data (products nested / embedded in brands)?
What's the most efficient schema design or would my setup be alright -- given what i've specified I want to use it for?
User story:
There will be an Admin User.
The admin will be able to add the Brand with the specific fields in the Brand Schema.
Brands will have associated Products, each Product will have a Style / category.
Search:
Users will be able to search Brands by name and location (i'm looking into doing this with angular filtering / tags).
Users will be able to search Products by fields (name, style, etc).
Users will be able to search Post by Brand Product and Style.
Post:
Users will be able to Post into a feed. When making a Post, they will choose a Brand and a Product to associate the Post with. The Post will display the Brand name, Product name, and Style -- along with newly entered Post fields (photo, comment, and rating).
Other users can click on the Brand name to link to the Brand show page. They can click on the Product name to link to a Product show page.
Product show page:
Will show Product fields from the above Schema -- including associated Style name from Style schema. It will also display Post pertaining to the specific Product.
Brand show page:
Will simply show Brand fields and associated products.
My main worry is the Post, which will have to populate / query for the Brand , Product, and Style within a feed.
Again, I'm contemplating if I should embed the Products within the Brand -- then would I be able to associate the Brand Product and Style with the Post for later queries? Or, possibly $lookup or other aggregate features.
Mongodb itself does not support joins. So, mongoose populate is an attempt at external reference resolution. The thing with mongodb is that you need to design your data so that:
most of you queries need not to refer multiple collections.
after getting data from query, you need not to transform it too much.
Consider the entities involved, and their relations:
Brand is brand. Doesn't depend on anything else.
Every Product belongs to a Brand.
Every Product is associated with a Style.
Every Post is associated with a Product.
Indirectly, every Post is associated to a Brand and Style, via product.
Now about the use cases:
Refer: If you are looking up one entity by id, then fetching 1-2 related entities is not really a big overhead.
List: It is when you have to return a large set of objects and each object needs an additional query to get associated objects. This is a performance issue. This is usually reduced by processing "pages" of result set at a time, say 20 records per request. Lets suppose you query 20 products (using skip and limit). For 20 products you extract two id arrays, one of referred styles, and other of referred brands. You do 2 additional queries using $in:[ids], get brands and styles object and place them in result set. That's 3 queries per page. Users can request next page as they scroll down, and so on.
Search: You want to search for products, but also want to specify brand name and style name. Sadly, product model only holds ids for style and brand. Same issue with searching Posts with brand and product. Popular solution is to maintain a separate "search index", a sort of table, that stores data exactly the way it will be searched for, with all searchable fields (like brand name, style name) at one place. Maintaining such search collections in mongodb manually can be a pain. This is where ElasticSearch comes in. Since you are already using mongoose, you can simply add mongoosastic to your models. ElasticSearch's search capabilities are far greater than a DB Storage engine will offer you.
Extra Speed: There is still some room for speeding things up: Caching. Attach mongoose-redis-cache and have frequent repeated queries served, in-memory from Redis, reducing load on mongodb.
Twitter like Feeds: Now if all Posts are public then listing them up for users in chronological order is a trivial query. However things change when you introduce "social networking" features. Then you need to list "activity feeds" of friends and followers. There's some wisdom about social inboxes and Fan-out lists in mongodb blog.
Moral of the story is that not all use cases have only "db schema query" solutions. Scalability is one of such cases. That's why other tools exist.
I'm trying to add private messaging between users into my data model. I've been going back and forth between two possible ways of doing this.
1) Each user has an array of user_id, chat_id pairs which correspond to chats they are participating in. Chat model just stores chat_id and array of messages.
2) Don't store chats with user at all and just have the Chat model store a pair of user_ids and array of messages.
The issue with option (1) is whenever a user joins or starts a chat, I would need to look first through the array for the user to see if the user_id, chat_id pair already exists. And then do a second find for the chat_id in Chat. If it doesn't exist, I would need to create the user_id, chat_id pair in two different places for both users who are participating.
With option (2) I would search through the Chat model for the user_id1, user_id2 pair, and if I find it I'm done, if not I would create a new Chat record for that pair and done.
Based on this option (2) does seem like the better way of handling this. However, I'm running into issues figuring out how to model the "pair" of user ids in a way that they are easily searchable in the chat model. i.e. how do I make sure I can find the chat record even if the user_ids are passed in the wrong order, i.e. user_id2, user_id1. What would be the best way to model this in Mongoose?
var chatSchema = mongoose.Schema({
messages: [{
text: {
type: String,
max: 2000
},
sender: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}
}],
participant1: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}]
participant2: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}]
});
If it's something like above, how would I search for a participant pair? Could I order the participant IDs in some way so that they are always participant1 < participant2 for example, making search simpler?
Well, there is no correct answer to this question, But definitely, the approaches you have mentioned are not the best at all!
Firstly, when you are thinking about designing a "chat" model, you need to take into account that there would be millions of messages between the users, so you need to care about performance when you want to fetch the chats.
Storing the messages into an array is not a good idea at all, your model's size will be large by the time and you have to consider that MongoDB's document size limit is currently 16 MB per document.
https://docs.mongodb.com/manual/reference/limits/
Secondly, You have to consider pagination aspect because it will affect the performance when the chat is large, when you retrieve the chat between 2 users you won't request all the chats since the beginning of the time, you will just request the most recent ones, and then you can request the older ones if the user scroll the chat, this aspect is very important and can't be neglected due to its effect on performance.
My approach will be to store each message in a separated document
First of all, storing each message in a single document will boost your performance during fetching the chats, and the document size will be very small.
This is a very simple example, you need to change the model according to your needs, it is just to represent the idea:
const MessageSchema = mongoose.Schema({
message:{
text: { type:String, required:true }
// you can add any other properties to the message here.
// for example, the message can be an image ! so you need to tweak this a little
}
// if you want to make a group chat, you can have more than 2 users in this array
users:[{
user: { type:mongoose.Schema.Types.ObjectId, ref:'User', required:true }
}]
sender: { type:mongoose.Schema.Types.ObjectId, ref:'User', required:true },
read: { type:Date }
},
{
timestamps: true
});
you can fetch the chats by this query:
Message.find(({ users: { "$in" : [#user1#,#user2#]} })
.sort({ updatedAt: -1 })
.limit(20)
Easy and clean!
as you see, pagination becomes very easy with this approach.
A few suggestions.
First - why store Participant1 and 2 as arrays? There is one specific sender, and one (or more) recipients (depending on if you want group messages).
Consider the following Schema:
var ChatSchema = new Schema({
sender : {
type : mongoose.Schema.Types.ObjectId,
ref : 'User'
},
messages : [
{
message : String,
meta : [
{
user : {
type : mongoose.Schema.Types.ObjectId,
ref : 'User'
},
delivered : Boolean,
read : Boolean
}
]
}
],
is_group_message : { type : Boolean, default : false },
participants : [
{
user : {
type : mongoose.Schema.Types.ObjectId,
ref : 'User'
},
delivered : Boolean,
read : Boolean,
last_seen : Date
}
]
});
This schema allows one chat document to store all messages, all participants, and all statuses related to each message and each participant.
the Boolean is_group_message is just a shorter way to filter which are direct / group messages, maybe for client side viewing or server-side processing. Direct messages are obviously easier to work with query-wise, but both are pretty simple.
the meta array lists the delivered/read status, etc, for each participant of a single message. If we weren't handling group messages, this wouldn't need to be an array, but we are, so that's fine.
the delivered and read properties on the main document (not the meta subdocument) are also just shorthand ways of telling if the last message was delivered/read or not. They're updated on each write to the document.
This schema allows us to store everything about a chat in one document. Even group chats.