use mongoose schema over multiple microservices - node.js

my application is split into multiple microservices that are running on heroku dynos (they can't access each others files). Sometimes, there are multiple microservices working with one collection. Therefore, both of the microservices need the according mongoose schema.
However, not both microservices need the full schema. For example, microservice A needs the full schema whereas microservice B only needs a few fields of that schema.
Example schema inside microservice A:
var AccountSchema = mongoose.Schema({
email: { type: String, required: true, unique: true },
password: { type: String, required: true },
phone: { type: String, required: true, unique: true },
forename: { type: String, required: true },
surname: { type: String, required: true },
middleInitals: { type: String, required: false },
failedLoginAttempts: { type: Number, required: true, default: 0 },
lockUntil: { type: Number },
createdAt: { type: Date, default: Date.now }
})
Example Schema inside microservice B:
var AccountSchema = mongoose.Schema({
email: { type: String, required: true, unique: true },
password: { type: String, required: true },
failedLoginAttempts: { type: Number, required: true, default: 0 },
lockUntil: { type: Number },
createdAt: { type: Date, default: Date.now }
})
My approach
I would just go ahead and create a new schema in each microservice, containing only the needed fields. However, I am not sure if there will be any problems when multiple microservices register a new schema to the MongoDB database? For example, both microservices would attempt to create an index for the unique field. Will there be any performance issues?
Does anybody have a different approach I could use? Is this even a valid approach to go with?
Thanks in advance :)

It's a valid approach. you can have 2 schemas pointing to the same collection. i have tested it and it works.
Mongoose is an Object Data Modeling (ODM) library, and you can have 2 objects looking at the same collection /(Table or view in SQL) - no problem with that.
No reason for performance problems, so long you got the right index. no relation to Object Data Modeling.
You might want to add some type key, so you can find only type1/type2 accounts on get request. On find, you can restrict getting the right fields with projection.
I think you should have only 2 keys in the index – email + password. If you have phone index and microservice B: don't include a phone –you will have a violation at the unique index of phone.
But if you really want a unique phone index you can make an override. You can generate temp unique value for phone for mircoservice B (using auto-Generated or duplicate the email value), you will ignore this value on mircoservice B and only find/ update / present phone in microsaervice A, where you have a valid phone. When user change from accountB type to accountA type – you must make sure to replace the wrong phone number with a valid one.
I see no problem in 2 schemas to same collection - you just need to mange your indexes the right way – to avoid collisions, and to insure you can differentiate the different accounts type in the collection.

As far as I can see there is no reason you simply can't use the same schema, maybe use some sort of privilege system and have it only return the right information between these separate micro services. You could have a fetch request tell it from which service its coming from and return a which items using a simple conditional.

Related

Mongodb Relationship: Posts and Comments (ref v sub-documents)

I know there are a lot of similar questions, but they're too old and since Mongodb has evolved alot for last 5-6 years I am looking for a good schema design.
Goal: I want to have a post with comments by users.
What I have designed so far is:
Separate post model:
const projectSchema = new mongoose.Schema({
user: { type: mongoose.Schema.Types.ObjectId, required: true, ref: 'User' },
title: { type: String, required: true },
image: { type: String, default: undefined },
description: { type: String, required: true, minLength: 200, maxlength: 500 },
comments: [{
type: mongoose.Schema.Types.ObjectId, ref: 'Comment'
}],
state: { type: Boolean, default: true },
collaborators: { type: Array, default: [] },
likes: { type: Array, default: [] }
})
And a separate comments model:
const commentSchema = new mongoose.Schema({
comment: { type: String, required: true },
project: { type: String, required: true, ref: 'Project' },
user: { type: String, required: true, ref: 'User' }
})
The reason I am going for the relational approach is because if the comments increase to say 10,000 in number, it will increase the size of schema by alot.
This way, no matter how many comments we can populate them using their IDs, also, we will have different collection for comments iself.
Reference : one-to-many
Is this a good approach for my project?
The way I am querying the comments from one single post:
const project = await Project.findById(
new mongoose.Types.ObjectId(req.params.projectId)
).populate({
path: 'comments',
populate: { path: 'user' }
}).lean()
Whether it's a good design depends how many comments per post do you expect, and what query will be performed on your app.
There's a good blog from mongodb.com on how to design your database schema
The common design is:
One to Few (Use embed)
One to Many (Use embed reference)
One to squillions (The usual relational database one-to-many approach)
Summary is:
So, even at this basic level, there is more to think about when designing a MongoDB schema than when designing a comparable relational schema. You need to consider two factors:
Will the entities on the “N” side of the One-to-N ever need to stand alone?
What is the cardinality of the relationship: is it one-to-few; one-to-many; or one-to-squillions?
Based on these factors, you can pick one of the three basic One-to-N schema designs:
Embed the N side if the cardinality is one-to-few and there is no need to access the embedded object outside the context of the parent object
Use an array of references to the N-side objects if the cardinality is one-to-many or if the N-side objects should stand alone for any reasons
Use a reference to the One-side in the N-side objects if the cardinality is one-to-squillions
There is also a blog about advanced schema design which is worth the read.
You seems to be using the two-way referencing approach.
The difference between yours and one-to-squillions is you are not only storing post id reference on comment document, but also storing comment ids as reference in post document, while one-to-squillions will only stores project id reference in comment document.
Using your approach will be better if you need to get comment ids of a post. But the disadvantage is you need to run two queries when deleting or creating a comment, one to delete / create comment id from post, and the other one to delete / create the comment document it self. It's also will be slower to find "which post belongs to given comment id".
While using one-to-squillions would gives you worse performance when performing a query to get comments by post id. But you can mitigate this by properly indexing your comment collection.

how to dynamically query in mongoose?

so I'm new to Programming and Mongoose, just learning the basics.
and now I get the task to make a dynamic query with criteria such as
query => gets all the data I input into the query in the parameter
example:
query: [ios, android] => i get all ios data, and android
query_else => gets all the data I input other than the parameter in the query
example:
query_else: [ios, android] => I got all the data OTHER ios and android
if I try to use .find, I can only get 1 specific data I'm looking for, but it can't if I enter 2 queries in it the results are []
maybe it's not the answer I want to look for, but how to think about solving this case, because of my lack of knowledge about mongoose and my lack of knowledge about coding that makes me deadlock thinking
Thank you in advance
account log schema in activity-log collection:
const actLogSchema = new Schema(
{
account_id: {
type: Schema.Types.ObjectId,
ref: 'account'
},
date_created: { type: String, default: Date.now() },
ip: String,
location: String,
device: String,
type: String,
label: String,
action: String,
description: String
},
{ versionKey: false }
);
i assume query is part of {date_created, ip, location, device, type, action, label, description}
and query_else is equal to query but the different it is values.. like the example above

Sort Keys in Response Object from Mongoose in ExpressJS and NodeJS

I have been working on my own small API, so far everything works super fine, i just have a small cosmetic issue I can#t seem to find an answer to.
I defined a schema like so in mongoose:
const ArtistSchema = new Schema({
stageName: {
type: String,
unique: true,
required: true,
minlength: 3,
maxlength: 255
},
realName: {
type: String,
unique: true,
required: true,
minlength: 5,
maxlength: 255
},
birthday: {
type: Date,
required: true
},
debutDate: {
type: Date,
required: true
},
company: {
type: String,
minlength: 5,
maxlength: 255,
required: function () {
return this.active;
}
},
active: {
type: Boolean,
default: true
},
music: [AlbumSchema],
createdAt: {
type: Date,
default: Date.now
}
});
I can create an entry in the database, with no problem either. I use this function on app.post
create(req, res, next) {
const artistProps = req.body;
Artist.create(artistProps)
.then(artist => res.send(artist))
.catch(next);
},
This works good, but the res.send(artist)) actually returns the object with no key order.. or in a pattern I cannot recognize. I want the response to be the same sorting as i defined in the schema, beause now it returns it:
active, stagename, realname, label, music, birthday
while it should be stagename, realname, birthday, debutDate etc..
I hope someone can help me out here. I know i can sort the VALUEs of a specific key with sort (like sort stageName alphabetically) but I really cant find anything for the keys .
Express' res.send method recognizes that artist is an Object, and calls JSON.stringify on it to convert the Object to a JSON string before sending. Simplifying a bit, the JSON.stringify method iterates through your artist object keys in the order they were created. (Here's a link to the more complicated ordering explanation.) That explains the current behavior.
Others may chime in with their own proposals for how you might do what you're aiming for, but here's a simple one to try first:
First, do your own JSON.stringify, using a "replacer" to create
the output order that you want:
const artistString = JSON.stringify(artist, ["realName", "stageName", ...])
// '{"realName": "Paul David Hewson", "stageName": "Bono", ...}'
Then, use res.json(artistString), rather than res.send, to send your JSON string with the
correct Content-Type header. (res.send will assume you want
Content-Type: “text/html”.)
There are definitely more complicated approaches, including creating a function that gets keys, sorts them, and returns a replacer; or writing your own .toJSON() substitute for JSON.stringify. You may need to implement one of these approaches, because you have nested objects; the behavior of the replacer can be a bit wonky in this case. You might be able to list the nested properties immediately after the parent, like:
["realName", "type", ...]
but since you have the same names for some nested properties, this may or may not work for you. You might have to stringify the insides before you stringify the outsides (gah!).
In any case, hopefully my suggestion can be a first step.

MongoDB - Limit number of records in related model

I have just started learning Node and MongoDB and I'm building an application for my wedding to try out these technologies.
Part of the application will focus on the assignment of guests to tables for the wedding breakfast. So far I have a Guest schema like so:
const guestSchema = new mongoose.Schema({
firstname: {
type: String,
trim: true,
},
surname: {
type: String,
trim: true,
},
attending: String,
allergies: String,
menu: String
});
const tableSchema = new mongoose.Schema({
name: String,
capacity: Number,
**guests: [
{ type: mongoose.Schema.ObjectId, ref: 'Guest' }
]**
I am aware that I can create a relationship between guests and tables as shown in bold but I'd like to limit the number of 'related' guests to a table according to the capacity of the table. E.g - tableOne might have 5 people, tableTwo might have 7, etc. I know that I could do this in Node by using my capacity field and performing a query before I relate a guest to a table, but I was wondering if there was a way to do this on the database side as I'd imagine this would be better practice?
All thoughts welcome,
Thanks

Mongoose best practices for document creation with unknown number of items in a field

I've looked around left and right, I wrote some demo code, wrote some tests to implement a school management system.
What I want to know from people more used to mongoose development is how would be the best practice to create this schema in a way that made it possible to add as many address, and contact fees as I want from this single document.
I made my own solution, but I don't know if it is the most elegant and feasible way, I want an opinion from seasoned people.
Should I create separate models for address, email and phone numbers?
I created this schema. It still has some pseudo-code, but for giving the general idea is fine.
var student = {
name: String,
surname: String,
currentClass: {
type: mongoose.Schema.Types.ObjectId,
ref: "Classes"
},
birthday: {
year: Number,
month: Number,
day: Number
},
address: [{
name: String,
zip: Number,
address: String,
city: String,
state: String,
complement: String
}]
accountable: {
name: String,
surname: String,
email: [{
type: String,
required: true,
lowercase: true
}],
phone: [String, String]
}
My sollution was, by using html, creating a new "email" or "address" fields as the user requested by clickinking in the propper button. This generated a new input field with a name that followed a pattern like:
email1, email2, email3, email4
And so, when the user sent the data, if we were creating a new student I would first create the array with the data and send it to mongoose.
In case of updating, I would get the already created emails and add it to the new array with the newEmails.concat(old_emails)
To design the database, there are many situations for it:
1 to 1, 1 to many, many to many.
One to one: you should to put the strong object inside the other, for example:a person can have only one passport, then we should put passport object inside the person object.
One to Many, 3 cases for one to many.
One to Few:few is less than 100 objects,Then you should add the few as list in the one object, for example:
A person can have multiple addresses as in your example above.
One to Many:many is Thousands, then you should put the primary keys of the many in a list inside the the one object.
One to Too many: then do not do the previous solution, but instead add the primary of the one in every objects of the many.
And finally, Many to Many: you should put them as list in both sides.
Check the below references:
https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-1
https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-2
https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-3
Morover, for this part phone: [String, String], you should make it phone: [String]

Resources