I have the following mongoose schema used in my MEAN app:
// schema
var categorySchema = new Schema ({
sId: String,
name: String,
parentId: String,
merchants: {},
attributes: {}, /* used to generate pivots and then discarded. */
pivots: [],
_id: {type: String, select: false},
/*other elements in db also not returned by using select: false*/
});
here's the problem. I have a mongodb that is not created by my app, rather its actual schema is defined elsewhere. I have access to this data but want it in a completely different format then what is actually in the database. This is working great by using:
categorySchema.options.toJSON = {
transform: function(doc, ret, options) {
however the schema doesn't represent the full API contract because the "attributes" field in the Schema is deleted in the transform. Pivots aren't in the database but are needed in the schema for mongoose to return it. Thankfully I like this, I want the schema to reflect exactly what I am returning, not what is in the database because frankly, it's a mess and I'm heavily transforming it, so I can give it to other engineers and use it for automated testing.
How do I get attributes out of the schema but still able to use in the transform?
turns out mongoose has function transforms. So I can do:
merchants: { type: {}, get: objToArr},
and that function is called.
just be sure to set:
Schema.set('toObject', { getters: true });
Schema.set('toJSON', { getters: true });
to true.
Related
I am trying to create a user document through this way:
// create the document ---------------
const id = mongoose.Types.ObjectId()
let userDoc = await Admin.create({ ...req.body, _id: id, by: id })
Schema:
adminSchema = new mongoose.Schema({
// some other fields, firstName, lastName ... etc
by: {
type: mongoose.Schema.ObjectId,
ref: 'Admin',
required: [true, "the 'by' field is required"],
immutable: true,
}
}, { _id: false })
Model:
const Admin = mongoose.model('Admin', adminSchema, 'users')
My schema doesn't have an _id property.
Now I want to have the _id field and the by field has the same value, which is a server-side generated id.
Mongoose is throwing this error:
Error: MongooseError: document must have an _id before saving at
.../node_modules/mongoose/lib/model.js:291:18
update:
I updated my question, I added the schema options, and now I know the reason why this error is happening. it's because of the _id: false schema option that I have set. But I need this option because I don't want to see _ids in the responses that I send to the clients. is there a workaround? because this option looks like its doing two unrelated things
Using Mongoose 6.4
I solved this by removing the _id: false schema type option.
and to remove the _id from the responses without having to pollute the routes with _.omit()s or deletes everywhere, I added the following schema type options to the schema:
toObject: {
virtuals: true,
transform(doc, ret) {
delete ret._id
},
},
Now the real question is, why does simply adding the option _id: false results in the Mongoose error when you're generating the id on the server-side without the help of Mongoose?
Error: MongooseError: document must have an _id before saving at
.../node_modules/mongoose/lib/model.js:291:18
I partially answered my own question, but for this one... I really don't know.
Based on your comment, if you want the response the user receives to not contain the _id you can:
Get the document
Remove the _id property and return this object without the _id (Or create a new object to avoid problems).
A brief example could be:
let responseDoc = await Admin.findOne({ _id: id });
delete responseDoc["_id"]
// responseDoc is now ready for use. Note that if I am not mistaken it is still a document.
Hello I am new to nodejs and mongodb.
I have 3 models:
"user" with fields "name phone"
"Shop" with fields "name, address"
"Member" with fields "shop user status". (shop and user hold the "id" of respective collections).
Now when I create "shops" api to fetch all shop, then I need to add extra field "isShopJoined" which is not part of the model. This extra field will true if user who see that shop is joined it otherwise it will be false.
The problem happens when I share my model with frontend developers like Android/iOS and others, They will not aware of that extra field until they see the API response.
So is it ok if I add extra field in shops listing which is not part of the model? Or do I need to add that extra field in model?
Important note
All the code below has NOT been tested (yet, I'll do it when I can setup a minimal environment) and should be adapted to your project. Keep in mind that I'm no expert when it comes to aggregation with MongoDB, let alone with Mongoose, the code is only here to grasp the general idea and algorithm.
If I understood correctly, you don't have to do anything since the info is stored in the Member collection. But it forces the front-end to do an extra-request (or many extra-requests) to have both the list of Shops and to check (one by one) if the current logged user is a Member of the shop.
Keep in mind that the front-end in general is driven by the data (and so, the API/back-end), not the contrary. The front-end will have to adapt to what you give it.
If you're happy with what you have, you can just keep it that way and it will work, but that might not be very effective.
Assuming this:
import mongoose from "mongoose";
const MemberSchema = new mongoose.Schema({
shopId: {
type: ObjectId,
ref: 'ShopSchema',
required: true
},
userId: {
type: ObjectId,
ref: 'UserSchema',
required: true
},
status: {
type: String,
required: true
}
});
const ShopSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
address: {
//your address model
}
});
const UserSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
phone: {
type: String,
required: true,
},
// Add something like this
shopsJoined: {
type: Array,
default: [],
required: true
}
});
You could tackle this problem via 2 ways:
MongoDB Aggregates
When retrieving (back-end side) the list of shops, if you know the user that made the request, instead of simply returning the list of Shops, you could return an aggregate of Shops and Members resulting in an hybrid document containing both the info of Shops and Models. That way, the front-end have all the info it needs with one back-end request.
Important note
The following code might not work as-is and you'll have to adapt it, I currently have nothing to test it against. Keep in mind I'm not very familiar with aggregates, let alone with Mongoose, but you'll get the general idea by looking the code and comments.
const aggregateShops = async (req, res, next) => {
try {
// $lookup will merge the "Model" and "Shop" documents into one
// $match will return only the results matching the condition
const aggreg = await Model.aggregate({$lookup: {
from: 'members', //the name of the mongodb collection
localField: '_id', //the "Shop" field to match with foreign collection
foreignField: 'shopId', //the "Member" field to match with local collection
as: 'memberInfo' //the field name in which to store the "Member" fields;
}, {
$match: {memberInfo: {userId: myUserId}}
}});
// the result should be an array of object looking like this:
/*{
_id: SHOP_OBJECT_ID,
name: SHOP_NAME,
address: SHOP_ADDRESS,
memberInfo: {
shopId: SHOP_OBJECT_ID,
userId: USER_OBJECT_ID,
status: STATUS_JOINED_OR_NOT
}
}*/
// send back the aggregated result to front-end
} catch (e) {
return next(e);
}
}
Drop the Members collection and store the info elsewhere
Instinctively, I would've gone this way. The idea is to either store an array field shopsJoined in the User model, or a membersJoined array field in the Shops model. That way, the info is retrieved no matter what, since you still have to retrieve the Shops and you already have your User.
// Your PATCH route should look like this
const patchUser = async (req, res, next) => {
try {
// How you chose to proceed here is up to you
// I tend to facilitate front-end work, so get them to send you (via req.body) the shopId to join OR "un-join"
// They should already know what shops are joined or not as they have the User
// For example, req.body.shopId = "+ID" if it's a join, or req.body.shopId = "-ID" if it's an un-join
if (req.body.shopId.startsWith("+")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $push: { shopsJoined: req.body.shopId } }
);
} else if (req.body.shopId.startsWith("-")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $pull: { shopsJoined: req.body.shopId } }
);
} else {
// not formatted correctly, return error
}
// return OK here depending on the framework you use
} catch (e) {
return next(e);
}
};
Of course, the above code is for the User model, but you can do the same thing for the Shop model.
Useful links:
MongoDB aggregation pipelines
Mongoose aggregates
MongoDB $push operator
MongoDB $pull operator
Yes you have to add the field to the model because adding it to the response will be only be a temporary display of the key but what if you need that in the future or in some list filters, so its good to add it to the model.
If you are thinking that front-end will have to be informed so just go it, and also you can set some default values to the "isShopJoined" key let it be flase for the time.
From node, I access a database with objects like
animals: [
{
name: monkey,
diet: banana,
tame: false,
},
{
name: donkey,
diet: carrot,
tame: true,
}
// [...]
]
I'd like to give access to most of the data to the clients, but make sure that the tame property is not exposed.
Using node and lodash's pick(), I could somehow whitelist the data, e.g.,
// retrieve data
// [...]
// whitelist
return {
name: _.pick(animal, 'name'),
diet: _.pick(animal, 'diet'),
};
but this is somewhat tedious, particularly if the selection of keys depends on other factors (e.g., the user who tries to access the data).
What's are good whitelisting/redaction patterns/modules for node?
A lot of this depends on the Database you use. Most databases allow you to select only specific columns/fields in the query itself. MongoDB also does this.
If you use mongoose you can actually enforce this per model:
function filter(document, animal) {
delete animal.tame;
return animal;
};
var options = {
toJSON: {transform: filter},
toObject: {transform: filter}
};
var animalSchema = new Schema({
name: { type: String, trim: true, required: true },
tame: { type: boolean, required: true },
secret: { type: String, required: true, select: false }
},options);
var Animal = mongoose.model('Animal', animalSchema);
var dog = new Animal({name:"rex", tame:true, secret:"rexrex"});
dog.save();
dog.toJSON(); // will not have "tame" property
dog.toJSON({transform: filter}) // dynamic filter
dog.toObject(); // will not have "tame" property
Animal.findOne(); // result objects will not have "secret" property
As you see, you can:
Set a transform function to execute on document when exporting to json or object.
Mark a field with select:false , and it will not show up in any mongoose Model based queries. (You still can do a custom query though.)
In case you are processing a lot of objects, consider writing a Transform Stream. Then you can:
Animal.find().stream().pipe(myTransformStream).pipe(clientResponse)
I am struggling to insert a document inside another document. I've looked at all the entries like this but they aren't quite what I am looking for.
Here is the scenario:
I have a common document that has its own schema. Lets call it a related record:
(function(){
'use strict';
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var relatedRecordSchema = new Schema({
params: {
recordId: Schema.Types.ObjectId,
recordType: String,
recordTitle: String
},
metadata: {
dateCreated: {type: Date, default: Date.now}
}
},{ _id : false });
mongoose.model('RelatedRecord', relatedRecordSchema);
})();
I have no trouble inserting this in an ARRAY inside document that require it. I.e its configured this way:
//Embedded
relationships: {
following: [mongoose.model('RelatedRecord').schema],
followers: [mongoose.model('RelatedRecord').schema],
blocked: [mongoose.model('RelatedRecord').schema]
}
This works perfectly.
The scenario that does not work is where there is a single related record, lets say the source of a notification:
var notificationSchema = new Schema({
params: {
title: String,
imageUrl: String,
source: mongoose.model('RelatedRecord').schema
},
metadata: {
dateCreated: { type: Date, default: Date.now },
dateViewed: Date
}
});
So when I am creating the notification I try and assign the previously prepared RelatedRecord
returnObj.params.source = relatedRecord;
The record appears during a debug to be inserted (it is inside a _docs branch but far deeper than I would expect) but when the object is saved (returnObj.save()) the save routine is abandoned without error, meaning it does not enter into the callback at all.
So it looks to me that i'm confusing mongoose as the dot assignment is forcing the subdoc into the wrong location.
So the question is simple:
How do I set that subdocument?
What the question isn't:
No I don't want to populate or advice on how you would solve this problem differently. We have sensible reasons for doing things how we are doing them.
Cheers
b
As Hiren S correctly pointed out:
1) Sub-Docs = array, always. Its in the first line in the docs :|
2) By setting the type to mixed, assignment of the object worked.
I'm a dumdum.
Using node.js, mongoose (3.5+), mongodb. Have got two collections in the DB:
var AuthorSchema = new mongoose.Schema({
name: { type: String },
});
var StorySchema = new mongoose.Schema({
title: { type: String },
author: { type: type: Schema.Types.ObjectId },
});
What I would like to do is retrieve an author and populate it with a subcollection (say, "stories") that is looked up from Story and match the author. Yes, much like a SQL join.
All of the examples out there work on the AuthorSchema having an array of objectids that reference StorySchema objects - that works fine. But I want to go the opposite direction; partly due to minimizing insert/updates. If I follow the example, adding a new store requires adding a new Story document and updating the Author. I want to just insert a new Story that references the Author.
I suspect that populate() is the right way to go, but can't get it to work. I'm doing something like this:
Author.find({name: 'Asimov').populate({
path: 'stories',
model: 'Story',
match: {'author': this['_id']},
}).exec(function(err, authors) {
console.log(authors);
})
But this doesn't return any stories member in the returned authors. Is this not a populate() solution? Do I really need to structure the schemas differently? Or is there some other feature of mongoose/mongo that would do what I'm looking for.
In the story schema, do this:
author: { type: type: Schema.Types.ObjectId, ref:'Author' }, //or whatever the model name is
then you can run
Story.find({}).populate('author').exec(function(err,stories) {...});