I started learning some NodeJS, and how to make a REST API from Academind on YouTube, and learned what a relational and non-relational database is, etc.
With MongoDB, writes are rather cheap, so I want to minimize the number of reads that I do. At the moment I am trying to see how I could make an API, that will be for an app that's similar to discord's, although it'll be for fun.
Is this the right way to make a Schema?
const mongoose = require('mongoose')
const userSchema = mongoose.Schema({
_id: mongoose.Schema.Types.ObjectId,
name: { type: String, required: true, unique: true},
email: { type: String, required: true },
password: { type: String, required: true }, // TODO: Hashing, etc
guilds: [{
_id: mongoose.Schema.Types.ObjectId,
name: {type: String, required: true},
channels: [{
_id: mongoose.Schema.Types.ObjectId,
name: {type: String, required: true},
// Only the X most recent messages
messages: [{
_id: mongoose.Schema.Types.ObjectId,
message: {type: String, required: true},
user: {
_id: mongoose.Schema.Types.ObjectId,
name: {type: String, required: true}
}
}]
}],
// Only an X amount of users
users: [{
_id: mongoose.Schema.Types.ObjectId,
name: {type: String, required: true}
}]
}]
})
module.exports = mongoose.model('User', userSchema)
And then for the Guilds,
const mongoose = require('mongoose')
const guildSchema = mongoose.Schema({
_id: mongoose.Schema.Types.ObjectId,
name: {type: String, required: true},
channels: [{
_id: mongoose.Schema.Types.ObjectId,
name: {type: String, required: true},
// Only an X amount of messages
messages: [{
_id: mongoose.Schema.Types.ObjectId,
message: {type: String, required: true},
user: {
_id: mongoose.Schema.Types.ObjectId,
name: {type: String, required: true}
}
}]
}],
// All the users
users: [{
_id: mongoose.Schema.Types.ObjectId,
name: {type: String, required: true}
}]
})
module.exports = mongoose.model('Guild', guildSchema)
Channel Schema
const mongoose = require('mongoose')
const channelSchema = mongoose.Schema({
_id: mongoose.Schema.Types.ObjectId,
name: { type: String, required: true },
guild: {
_id: mongoose.Schema.Types.ObjectId,
name: { type: String, required: true },
channels: [{
_id: mongoose.Schema.Types.ObjectId,
name: { type: String, required: true }
}],
// The users of the guild, or just the channel?
// Could add a users object outisde of the guild object
users: [{
_id: mongoose.Schema.Types.ObjectId,
name: { type: String, required: true }
}]
}
})
module.exports = mongoose.model('Channel', channelSchema)
And finally for the messages
const mongoose = require('mongoose')
const messageSchema = mongoose.Schema({
_id: mongoose.Schema.Types.ObjectId,
user: {
_id: mongoose.Schema.Types.ObjectId,
name: {type: String, required: true}
},
message: {type: String, required: true},
channel: {
guild: {
_id: mongoose.Schema.Types.ObjectId,
name: {type: String, required: true}
// Store more data for each message?
}
}
})
module.exports = mongoose.model('Message', messageSchema)
I am not sure if this is how a non-relational schema should look like. If it's not, how would I go about to store the data that I need?
Also, let's say that I POST a message on channel X on guild Y with the users A B and C, how would I go about to update all the entries, to add a message?
I've only used the User.find({_id: id}).exec().then().catch() so far, so I am not sure how to go about to update them.
Thanks in advance!
The messages collection should be on its own, do not embed it into any collection. This is not a good idea to embed data that will grow without limit.
The idea to store the last 5 messages into other collection looks painful to implement.
Embed denormalised data from all collections into the users collection seems like a problem when you will have to update guilds, channels, guilds users.
You may embed channels into guilds. Channels would not grow without a limit, should be a reasonable amount, less than 100 of channels per guild and probably it always used with a guild that they belong to. If not, consider not to embed channels into guilds.
The power of mongodb is to build the schema that reflects how your app is using data. I would recommend starting with normalized data. And when problems with creating, reading, updating, deleting data will occur then make appropriate changes in your mongoose schema to solve the problem. Premature optimization will only hurts in the long run.
As always an answer depends on details. Since I do not know all details I would recommend three part article by William Zola, Lead Technical Support Engineer at MongoDB. part 1 part 2 part 3
Related
I have a list of users, each user have a list of exercises. I want to aggregate the basic user's info and their exercises to be displayed.
I did the following:
const userSchema = new mongoose.Schema({
name: {
type: String,
required: [true, 'Invalid username'],
unique: [true, 'This user already exists'],
},
exercises: {
type: [mongoose.Schema.Types.ObjectId],
ref: 'Exercise'
}
})
User = mongoose.model('User', userSchema);
And
const exerciseSchema = new mongoose.Schema({
user: {type: mongoose.Schema.Types.ObjectId, ref: 'User', required: true},
description: {type:String, required: true},
duration: {type:Number, required: true},
date: {type:Date, required: true},
});
Exercise = mongoose.model('Exercise', exerciseSchema);
However, the aggregation part displays only the user's info with an empty array of exercises:
User.
find().
populate({
path: 'exercises',
model: 'Exercise'
}).
exec().
then(docs => res.json(docs).status(200)).
catch(err => res.json(err).status(500))
})
Gives:
[{
"exercises":[],
"_id":"6047b61bc7a4f702f477085b",
"name":"John Smith",
"__v":0}
]
Use the following aggregate query to get users and is exercise data.
User.aggregate([{
"$lookup":{
"localField":"exercises",
"foreignField":"_id",
"from":"Exercise",
"as" :"userExercises"
}
}])
You will get each user with its exercise data.
The problem is:
I have a collection of photos schema and likes schema, and inside photos there is an array of like ObjectIds called likeArray which is used to populate data from likes colletion.
But when i delete a like from likes collection, the ObjectId of that like in the likeArray still exists.
I tried to find the index of like._id in the likeArray and use likeArray.splice(index, 1) but couldn't work.
Can someone know the solution?
Here's the photo schema:
var Photo = mongoose.model('Photo', new mongoose.Schema({
photo_url: String,
photo_description: String,
user:{
id: {
type: mongoose.Schema.Types.ObjectId,
ref: 'user'
},
username: String,
profile_photo: String
},
comments: [
{
type: mongoose.Schema.Types.ObjectId,
ref: 'comment'
}
],
likes: [
{
type: mongoose.Schema.Types.ObjectId,
ref: 'like'
}
],
created_at: {type: Date, default: Date.now}
}));
Here's the Like schema:
var Like = mongoose.model('Like', new mongoose.Schema({
user_id: {
type: mongoose.Schema.Types.ObjectId,
ref: 'user'
},
photo_id: {
type: mongoose.Schema.Types.ObjectId,
ref: 'photo'
},
created_at: {type: Date, default: Date.now}
}));
Instead of splice you can use $pull operator. Here's how it'd look in mongo shell.
db.likes.remove({_id: like_oid});
db.photos.update({likes: like_oid}, {$pull: {likes: like_oid}}, { multi: true });
I have a problem with a mongoose population and I don't know what I should do.
I got two schemas:
var userSchema = new userSchema({
username: { type: String, required: true, unique: true },
password: { type: String, required: true },
mods: [{ type: mongoose.Schema.Types.ObjectId, ref: 'users'}]
});
var dataSchema = mongoose.Schema({
title: { type: String, required: true, unique: true },
description: { type: String, required: true },
owner: {type: mongoose.Schema.Types.ObjectId, required: true}
});
So one user can have several data packages.
Some users are moderated by other users.
Whats the query for a moderator, that all his own data packages and the ones of the users he is moderating are listed?
You see that I have a SQL background and there's definitely another way to do it with MongoDB.
Thanks for your help!
I'm not clear understand what queries do you need but first you need set ref property in 'owner' field in dataSchema. As about population it's look like this:
//if you use callback
users.find({/*your query*/}).populate('mods')
.exec((err, result)=>{/*your code*/});
//if you use promise
users.find({/*your query*/}).populate('mods').exec()
.then(result=>{/*your code*/})
.catch(err=>{throw err});
I have 2 schemas:
var pollSchema = new mongoose.Schema({
title: String,
created: {
type: Date, default: Date.now
},
options: [{
label: String,
count: {
type: Number, default: 0
},
backgroundColor: {
type: String, default: '#fff'
}
}],
author:{
id:{
type: mongoose.Schema.Types.ObjectId,
ref: "User"
},
username: String
}
});
var userSchema = new Schema({
username: {type: String, unique:true},
email: {type: String, unique:true, lowercase: true},
password: String
});
Now each poll will store data of it's author.
Questions:
How can I redesign my schemas - so I will be able to find all the polls belong to particular user?
Or should I leave the schemas the same and find another approach?
you can still find all the polls belonging to a particular user . You have the author.id for that.
Also you can keep an array as var userSchema = new Schema({
username: {type: String, unique:true},
email: {type: String, unique:true, lowercase: true},
password: String,
polls: []
});
And every time a user polls, push the userId inside the polls array, which you can later populate or get the count.
I recently started a project using mongodb and nodejs to build a restful web service. Unfortunately mongodb is very new to me, and coming from the relational databases world I'm asking my self a lot of questions.
Let me explain you my problem :
The goal is to build a sort of content management system with social features like a user can post topics that can be shared and commented.
I have 2 possibilities to do this the one using a reference to get topics posted by a user, the second using topics as embedded document of user instead of reference.
So basically I can have these 2 schemas :
var UserSchema = new Schema({
username: {
type: String,
unique: true,
required: true
},
password: {
type: String,
required: true
},
name: {
type: String
},
first_name: String,
phone: String,
topics: [Topic.schema]
});
var TopicSchema = new Schema({
_creator: {
type: String,
ref: 'User'
},
description: String,
comments: [Comments.schema],
shared_with: [{
type: Schema.ObjectId,
ref: 'User'
}] //[{ type: String, ref: 'User'}]
});
var CommentSchema = new Schema({
_creator: {
type: String,
require: true
},
text: {
type: String,
required: true
},
});
and
var UserSchema = new Schema({
username: {
type: String,
unique: true,
required: true
},
password: {
type: String,
required: true
},
name: {
type: String
},
first_name: String,
phone: String,
topics: [{ type: Schema.ObjectId, ref: 'Topics'}]
});
var TopicSchema = new Schema({
_creator: {
type: String,
ref: 'User'
},
description: String,
comments: [Comments.schema],
shared_with: [{
type: Schema.ObjectId,
ref: 'User'
}] //[{ type: String, ref: 'User'}]
});
var CommentSchema = new Schema({
_creator: {
type: String,
require: true
},
text: {
type: String,
required: true
},
});
So the first schema uses 1 collection of user document and the second use 1 collection for the user and 1 collection for the topics, this implies to make for example, 2 finds queries to retrieve a user and it's topics but it is also easyer to query directly the topics.
Here is the request I use to retrieve a specific topic with some user info with the first schema :
User.aggregate([
{$match: {
"topics._id":{$in:[mongoose.Types.ObjectId('56158c314861d2e60d000003')]}
}},
{ $unwind:"$topics" },
{$match: {
"topics._id":{$in:[mongoose.Types.ObjectId('56158c314861d2e60d000003')]}
}},
{ $group: {
_id: {
_id:"$_id",
name:"$name",
first_name:"$first_name"
},
topics:{ "$push": "$topics"}
}}
]);
So the question is, what do youh think ? Which is the good schema in your opinion ?
Thanks in advance.
Better solution: using a reference to get topics posted by a user
For this database use, one typically needs to consider the MMAPV1 document size limit (16MB). Putting user, topic, and comments in one document allows the document to grow without bound. If each topic is a page of text (1K), then each user could have about 16,000 topics before the limit is reached. That seems huge, but what happens if you decide to put images, videos, sounds in the topic as the product matures? Converting from an embedded to a normalized schema later would be a lot more work than a simple design choice today.
Similarly, if the comments could grow to cause a topic to exceed the 16MB limit, they should be in a separate collection. Unlikely? Probably. But if you are writing something that will become, say, the Huffington Post - check out comments on their popular articles.
Here is mongo's advice on data model design