How to model a collection in nodejs+mongodb - node.js

Hello I am new to nodejs and mongodb.
I have 3 models:
"user" with fields "name phone"
"Shop" with fields "name, address"
"Member" with fields "shop user status". (shop and user hold the "id" of respective collections).
Now when I create "shops" api to fetch all shop, then I need to add extra field "isShopJoined" which is not part of the model. This extra field will true if user who see that shop is joined it otherwise it will be false.
The problem happens when I share my model with frontend developers like Android/iOS and others, They will not aware of that extra field until they see the API response.
So is it ok if I add extra field in shops listing which is not part of the model? Or do I need to add that extra field in model?

Important note
All the code below has NOT been tested (yet, I'll do it when I can setup a minimal environment) and should be adapted to your project. Keep in mind that I'm no expert when it comes to aggregation with MongoDB, let alone with Mongoose, the code is only here to grasp the general idea and algorithm.
If I understood correctly, you don't have to do anything since the info is stored in the Member collection. But it forces the front-end to do an extra-request (or many extra-requests) to have both the list of Shops and to check (one by one) if the current logged user is a Member of the shop.
Keep in mind that the front-end in general is driven by the data (and so, the API/back-end), not the contrary. The front-end will have to adapt to what you give it.
If you're happy with what you have, you can just keep it that way and it will work, but that might not be very effective.
Assuming this:
import mongoose from "mongoose";
const MemberSchema = new mongoose.Schema({
shopId: {
type: ObjectId,
ref: 'ShopSchema',
required: true
},
userId: {
type: ObjectId,
ref: 'UserSchema',
required: true
},
status: {
type: String,
required: true
}
});
const ShopSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
address: {
//your address model
}
});
const UserSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
phone: {
type: String,
required: true,
},
// Add something like this
shopsJoined: {
type: Array,
default: [],
required: true
}
});
You could tackle this problem via 2 ways:
MongoDB Aggregates
When retrieving (back-end side) the list of shops, if you know the user that made the request, instead of simply returning the list of Shops, you could return an aggregate of Shops and Members resulting in an hybrid document containing both the info of Shops and Models. That way, the front-end have all the info it needs with one back-end request.
Important note
The following code might not work as-is and you'll have to adapt it, I currently have nothing to test it against. Keep in mind I'm not very familiar with aggregates, let alone with Mongoose, but you'll get the general idea by looking the code and comments.
const aggregateShops = async (req, res, next) => {
try {
// $lookup will merge the "Model" and "Shop" documents into one
// $match will return only the results matching the condition
const aggreg = await Model.aggregate({$lookup: {
from: 'members', //the name of the mongodb collection
localField: '_id', //the "Shop" field to match with foreign collection
foreignField: 'shopId', //the "Member" field to match with local collection
as: 'memberInfo' //the field name in which to store the "Member" fields;
}, {
$match: {memberInfo: {userId: myUserId}}
}});
// the result should be an array of object looking like this:
/*{
_id: SHOP_OBJECT_ID,
name: SHOP_NAME,
address: SHOP_ADDRESS,
memberInfo: {
shopId: SHOP_OBJECT_ID,
userId: USER_OBJECT_ID,
status: STATUS_JOINED_OR_NOT
}
}*/
// send back the aggregated result to front-end
} catch (e) {
return next(e);
}
}
Drop the Members collection and store the info elsewhere
Instinctively, I would've gone this way. The idea is to either store an array field shopsJoined in the User model, or a membersJoined array field in the Shops model. That way, the info is retrieved no matter what, since you still have to retrieve the Shops and you already have your User.
// Your PATCH route should look like this
const patchUser = async (req, res, next) => {
try {
// How you chose to proceed here is up to you
// I tend to facilitate front-end work, so get them to send you (via req.body) the shopId to join OR "un-join"
// They should already know what shops are joined or not as they have the User
// For example, req.body.shopId = "+ID" if it's a join, or req.body.shopId = "-ID" if it's an un-join
if (req.body.shopId.startsWith("+")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $push: { shopsJoined: req.body.shopId } }
);
} else if (req.body.shopId.startsWith("-")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $pull: { shopsJoined: req.body.shopId } }
);
} else {
// not formatted correctly, return error
}
// return OK here depending on the framework you use
} catch (e) {
return next(e);
}
};
Of course, the above code is for the User model, but you can do the same thing for the Shop model.
Useful links:
MongoDB aggregation pipelines
Mongoose aggregates
MongoDB $push operator
MongoDB $pull operator

Yes you have to add the field to the model because adding it to the response will be only be a temporary display of the key but what if you need that in the future or in some list filters, so its good to add it to the model.
If you are thinking that front-end will have to be informed so just go it, and also you can set some default values to the "isShopJoined" key let it be flase for the time.

Related

Mongoose Insert Multiple Related Records at once

Currently I have the following, effective inserting one lyric per song at a time:
SongSchema.statics.addLyric = function(songId, title, content) {
const Lyric = mongoose.model('lyric');
return this.findById(songId).then(song => {
const lyric = new Lyric({ title, content, song });
song.lyrics.push(lyric);
return Promise.all([lyric.save(), song.save()]).then(([lyric, song]) => song);
});
};
However I would like to update this to something like the following, where I pass multiple lyrics at once in an array...
SongSchema.statics.addLyric = function(songId, lyrics) {
...
};
Is it possible to insert all the lyrics at once and still return the updated song to graphql
Fair point. Instead of doing findById() then it's probably better to use findByIdAndUpdate() instead, and also do the creation inline and "chain" the Promise instead:
SongSchema.statics.addLyrics = function(songId, lyrics) {
const Lyric = mongoose.model('lyric');
return Lyric.insertMany(lyrics).then( lyrics =>
this.findByIdAndUpdate(
songId,
{ "$push": { "lyrics": { "$each": lyrics } } },
{ "new": true }
);
};
That uses $each as a modifier to $push which accepts and array and does an "atomic" operation to update the document. It's a lot more efficient and safer than fetching the document "then" modifying it before updating back.
Also of course insertMany() does your 'array' of lyrics as a single write as opposed to "many".
An alternate approach would be to create instances based on a Array.map() and save() in parallel.
SongSchema.statics.addLyrics = function(songId, lyrics) {
const Lyric = mongoose.model('lyric');
lyrics = lyrics.map(lyric => new Lyric(lyric));
return Promise.all([
this.findByIdAndUpdate(
songId,
{ "$push": { "lyrics": { "$each": lyrics } } },
{ "new": true }
),
...lyrics.map(lyric => lyric.save())
]).then(([song, ...lyrics]) => song);
};
But the first approach really has less overhead, and Promise.all() is not going to respond until "all" promises are resolved anyway. So you really don't gain anything by not doing the operations in series.
The alternate case of course is to instead of keeping an "array" of related ObjectId values within the Song, you would instead simply record the songId within the Lyric entry.
So the schema would then become something like:
const lyricSchema = new Schema({
title: String,
content: String,
songId: { type: Schema.Types.ObjectId, ref: 'Song' }
})
Then the insertion is simply
lyricSchema.statics.addLyrics = function(songId, lyrics) {
return this.insertMany(lyrics.map(lyric => ({ ...lyric, songId })))
}
And in the Song schema, instead of keeping an array like this:
songs: [{ type: Schema.Types.ObjectId, ref: 'Lyric' }]
Remove that and replace with a virtual,
SongSchema.virtual.('songs', {
ref: 'Lyric',
localField: '_id',
foreignField: 'songId'
});
And that means there is no need to touch the Song model at all as you can simply insert the related data without needing to update an array.
Modern MongoDB versions really should be using $lookup to "query" this information anyway, and the maintaining of "arrays" within a parent is a bit of an "anti-pattern" which generally should be avoided.
The "virtual" is therefore "optional", and just a way to enable populate() as a convenience.
The best way I know of is like this:
lyrics = lyrics.map((lyric. i) => {lyric, songId: songI[i]});
SongSchema.collection.insertMany(lyrics, {'ordered': false},
(err, data) => {
/*Do your stuff*/
});
Ordered true will throw an error if there is any repeated unique values and stop, if false, the error is still thrown but the insertion goes on for the non repeated ones.
You do not need to create a new Schema, but this way Momgoose validations will be skipped too.

Mongoose query nested document returns empty array

I have these schemas:
var Store = mongoose.model('Store', new Schema({
name: String
}));
var Client = mongoose.model('Cllient', new Schema({
name: String,
store: { type: Schema.ObjectId, ref: 'Store' }
}));
var Order = mongoose.model('Order', new Schema({
number: String,
client: { type: Schema.ObjectId, ref: 'Client' }
}));
I'm trying to code the Url handler of the API that returns the order details, which looks like this:
app.get('/api/store/:storeId/order/:orderId', function (...));
I'm passing the store id in the Url to quickly check if the logged user has permissions on the store. If not, it returns a 403 status. That said, I think this storeId and the orderId are enough data to get the order, so I'm trying to do a query on a nested document, but it just doesn't work.
Order.findOne(
{ 'client.store': req.params.storeId, _id: req.params.orderId },
function (err, order) { ... });
But the order object is null;
Even when I perform a find, it returns an empty array:
Order.find(
{ 'client.store': req.params.storeId },
function (err, results) { ... });
I know that I could as well pass the cliendId to the Url and check first if the client belongs to the store, and then retrieve the order from the client, but I think the client part is redundant, don't you think? I should be able to get the order in a secure way by using only these two fields.
What am I doing wrong here?
Ok, I found it. The secret was in the match option of populate. The final code looks like this:
Order
.findOne({ _id: req.params.orderId })
.populate({ path: 'client', match: { store: req.params.storeId } })
.exec(function (err, order) { ... });

What is populate in SailsJs?

I recently began to develop on sailsJs and not understanding the subtleties
Please explain to me what is populate in SailsJs and who can please do simple example
Thanks in advance ?
whats is ?
User.find({ name: 'foo' })
.populate('pets', { name: 'fluffy' })
.exec(function(err, users) {
if(err) return res.serverError(err);
res.json(users);
});
populate is used for associations. When your model is something like this:
// User.js
module.exports = {
attributes: {
name: {
type: "string"
},
pet: {
model: "pet"
}
}
}
Here pet attribute of user collection is a reference to pet table. In user table it will store only the id column of pet. However, when you do a populate while find, then it will fetch the entire record of the pet entry and display it here. This is just for one to one association. You can have many to one associations as well as many to many. See this documentation for more details
sailsjs project is use the wateline ORM. you can see the document. if you want to use 'populate()', you need define Associations in the model.
.populate()
populate is used with associations to include any related values specified in a model definition. If a collection attribute is defined in a many-to-many, one-to-many or many-to-many-through association the populate option also accepts a full criteria object. This allows you to filter associations and run limit and skip on the results.
as you example, you need do like this:
User.js
// A user may have many pets
var User = Waterline.Collection.extend({
identity: 'user',
connection: 'local-postgresql',
attributes: {
firstName: 'string',
lastName: 'string',
// Add a reference to Pets
pets: {
collection: 'pet',
via: 'owner'
}
}
});
Pet.js
// A pet may only belong to a single user
var Pet = Waterline.Collection.extend({
identity: 'pet',
connection: 'local-postgresql',
attributes: {
breed: 'string',
type: 'string',
name: 'string',
// Add a reference to User
owner: {
model: 'user'
}
}
});
you can ready the doc, and you can use it very easy
To resume what is .populate() (used by waterline) is a little what join is used by SQL.
.populate() allows you to join the tables in your database.
The link identifier is to be defined in your model.
In other words, to associate a user (who is in the "User" table) with a dog (who is in the "Dog" table), you use populate.
To resume your example:
You are looking for the user => User.find ({name: my_user})
You are looking for the dog named "fluffy" => {name: 'fluffy'}
You are looking for the dog 'fluffy' which is associated with your user (belongs) => populate ('pets')
Which give:
User.find({ name: 'foo' })
.populate('pets', { name: 'fluffy' })
.exec(function(err, users) {
if(err) return res.serverError(err);
res.json(users);
}
This association ("pets"), you define it in your models "User" and "Pets" like the example above.
Populate is all about displaying the content of the (id) on which it was refreed
"abc": [{
type: ObjectId,
ref: "xyz"
}],

Create unique autoincrement field with mongoose [duplicate]

This question already has answers here:
Mongoose auto increment
(15 answers)
Closed 2 years ago.
Given a Schema:
var EventSchema = new Schema({
id: {
// ...
},
name: {
type: String
},
});
I want to make id unique and autoincrement. I try to realize mongodb implementation but have problems of understanding how to do it right in mongoose.
My question is: what is the right way to implement autoincrement field in mongoose without using any plugins and so on?
const ModelIncrementSchema = new Schema({
model: { type: String, required: true, index: { unique: true } },
idx: { type: Number, default: 0 }
});
ModelIncrementSchema.statics.getNextId = async function(modelName, callback) {
let incr = await this.findOne({ model: modelName });
if (!incr) incr = await new this({ model: modelName }).save();
incr.idx++;
incr.save();
return incr.idx;
};
const PageSchema = new Schema({
id: { type: Number , default: 0},
title: { type: String },
description: { type: String }
});
PageSchema.pre('save', async function(next) {
if (this.isNew) {
const id = await ModelIncrement.getNextId('Page');
this.id = id; // Incremented
next();
} else {
next();
}
});
Yes, here's the "skinny" on that feature.
You need to have that collection in your mongo database. It acts as concurrent key allocation single record of truth if you want. Mongo's example shows you how to perform an "atomic" operation to get the next key and ensure that even there are concurrent requests you will be guaranteed to have the unique key returned without collisions.
But, mongodb doesn't implement that mechanism natively, they show you how to do it. They only provide for the _id to be used as unique document key. I hope this clarifies your approach.
To expand on the idea, go ahead and add that mongo suggested implementation to your defined Mongoose model and as you already guessed, use it in Pre-save or better yet pre-init event to ensure you always generate an id if you work with a collection server side before you save it to mongo.
You can use this.
This package every time generate unique value for this.
Package Name : uniqid
Link : https://www.npmjs.com/package/uniqid
Ignore all the above. Here is the solution
YourModelname.find().count(function(err, count){
req["body"].yourID= count + 1;
YourModelname.create(req.body, function (err, post) {
if (err) return next(err);
res.json(req.body);
});
});

Hide embedded document in mongoose/node REST server

I'm trying to hide certain fields on my GET output for my REST server. I have 2 schema's, both have a field to embed related data from eachother into the GET, so getting /people would return a list of locations they work at and getting a list of locations returns who works there. Doing that, however, will add a person.locations.employees field and will then list out the employees again, which obviously I don't want. So how do I remove that field from the output before displaying it? Thanks all, let me know if you need any more information.
/********************
/ GET :endpoint
********************/
app.get('/:endpoint', function (req, res) {
var endpoint = req.params.endpoint;
// Select model based on endpoint, otherwise throw err
if( endpoint == 'people' ){
model = PeopleModel.find().populate('locations');
} else if( endpoint == 'locations' ){
model = LocationsModel.find().populate('employees');
} else {
return res.send(404, { erorr: "That resource doesn't exist" });
}
// Display the results
return model.exec(function (err, obj) {
if (!err) {
return res.send(obj);
} else {
return res.send(err);
}
});
});
Here is my GET logic. So I've been trying to use the query functions in mongoose after the populate function to try and filter out those references. Here are my two schema's.
peopleSchema.js
return new Schema({
first_name: String,
last_name: String,
address: {},
image: String,
job_title: String,
created_at: { type: Date, default: Date.now },
active_until: { type: Date, default: null },
hourly_wage: Number,
locations: [{ type: Schema.ObjectId, ref: 'Locations' }],
employee_number: Number
}, { collection: 'people' });
locationsSchema.js
return new Schema({
title: String,
address: {},
current_manager: String, // Inherit person details
alternate_contact: String, // Inherit person details
hours: {},
employees: [{ type: Schema.ObjectId, ref: 'People' }], // mixin employees that work at this location
created_at: { type: Date, default: Date.now },
active_until: { type: Date, default: null }
}, { collection: 'locations' });
You should specify the fields you want to fetch by using the select() method. You can do so by doing something like:
if( endpoint == 'people' ){
model = PeopleModel.find().select('locations').populate('locations');
} else if( endpoint == 'locations' ){
model = LocationsModel.find().select('employees').populate('employees');
} // ...
You can select more fields by separating them with spaces, for example:
PeopleModel.find().select('first_name last_name locations') ...
Select is the right answer but it also may help to specify it in your schema so that you maintain consistency in your API and I've found it helps me to not remember to do it everywhere I perform a query on the object.
You can set certain fields in your schema to never return by using the select: true|false attribute on the schema field.
More details can be found here: http://mongoosejs.com/docs/api.html#schematype_SchemaType-select
SOLUTION!
Because this was so hard for me to find i'm going to leave this here for anybody else. In order to "deselect" a populated item, just prefix the field with "-" in your select. Example:
PeopleModel.find().populate({path: 'locations', select: '-employees'});
And now locations.employee's will be hidden.
If you remember from you SQL days, SELECT does a restriction on the table(s) being queried. Restrict is one of the primitive operations from the relational model and continues to be a useful feature as the relational model has evolved. blah blah blah.
In mongoose, the Query.select() method allows you to perform this operation with some extra features. Particularly, not only can you specify what attributes (columns) to return, but you can also specify what attributes you want to exclude.
So here's the example:
function getPeople(req,res, next) {
var query = PeopleModel.find().populate({path: 'locations', select: '-employees'});
query.exec(function(err, people) {
// error handling stuff
// process and return response stuff
});
}
function getLocations(req,res, next) {
var query = LocationModel.find().populate({path: 'employees', select: '-locations'});
query.exec(function(err, people) {
// error handling stuff
// processing and returning response stuff
});
}
app.get('people', getPeople);
app.get('locations', getLocations);
Directly from the Mongoose Docs:
Go to http://mongoosejs.com/docs/populate.html and search for "Query conditions and other options"
Query conditions and other options
What if we wanted to populate our fans array based on their age,
select just their names, and return at most, any 5 of them?
Story
.find(...)
.populate({
path: 'fans',
match: { age: { $gte: 21 }},
select: 'name -_id',
options: { limit: 5 }
})
.exec()
I just wanted to remark, for the simplicity of the endpoint you may be able to get away with this way to define the endpoints. However, in general this kind of dispacher pattern is not necessary and may pose problems later in development when developing with Express.

Resources