Mongoose: what's up with "_doc"? - node.js

It seems Mongoose is doing something really funky internally.
var Foo = new mongoose.model('Foo', new mongoose.Schema({a: String, b: Number}));
var foo = new Foo({a: 'test'; b: 42});
var obj = {c: 1};
foo.goo = obj; // simple object assignment. obj should be
// passed by reference to foo.goo. recall goo
// is not defined in the Foo model schema
console.log(foo.goo === obj); // comparison directly after the assignment
// => false, doesn't behave like normal JS object
Essentially, any time you try to deal with properties of a Mongoose model that aren't
a) defined in the model's schema or
b) defined as the same type (array, obj, ..) ... the model doesn't even behave like a normal Javascript object.
Switching line 4 to foo._doc.goo = obj makes the console output true.
edit: trying to reproduce weirdness
example 1:
// Customer has a property 'name', but no property 'text'
// I do this because I need to transform my data slightly before sending it
// to client.
models.Customer.find({}, function(err, data) {
for (var i=0, len=data.length; i<len; ++i) {
data[i] = data[i]._doc; // if I don't do this, returned data
// has no 'text' property
data[i].text = data[i].name;
}
res.json({success: err, response:data});
});

_doc exist on the mongoose object.
Because mongooseModel.findOne returns the model itself, the model has structure (protected fields).
When you try to print the object with console.log it gives you only the data from the database, because console.log will print the object public fields.
If you try something like JSON.stringify then you get to see inside the mongoose model object. (_doc, state ...)
In the case where you want to add more fields in the object and it's not working
const car = model.findOne({_id:'1'})
car.someNewProp = true // this will not work
If later you set the property to the object car and you didn't specify in the Model Schema before then Mongoose model is validating if this field exists and if it's the valid type.
If the validation fails then the property will not be set.

Update
Maybe I misunderstood your original question, but now it looks like the nature of your question changed, so the below information isn't relevant, but I'm leaving it. :)
I tested your code and it works fine for me. Mongoose doesn't execute any special code when you set properties that aren't part of the schema (or a few other special properties). JavaScript currently doesn't support calling code for properties that don't yet exist (so Mongoose can't get in the way of the set of the goo property for example).
So, when you set the property:
foo.goo = { c: 1 };
Mongoose isn't involved. If your console.log was something other than the code you displayed, I could see that it might report incorrectly.
Additionally, when you send the results back as JSON, JSON.stringify is being called, which calls toString on your Mongoose Model. When that happens, Mongoose only uses the properties defined on the schema. So, no additional properties are being sent back by default. You've changed the nature of the data array though to directly point at the Mongoose data, so it avoids that problem.
Details about normal behavior
When you set the property goo using Mongoose, quite a few things happen. Mongoose creates property getters/setters via the Object.defineProperty (some docs). So, when you set the goo property, which you've defined as a [String], a few things happen:
Mongoose code is called prior to the value being set onto the object instance (unlike a simple JavaScript object)
Mongoose creates an array (optionally) to store the data (a MongooseArray) which will contain the array data. In the example you provided, since you didn't pass an array, it will be created.
Mongoose will attempt to cast your data to the right type
It will call toString on the data passed as part of the cast.
So, the results are that the document now contains an array with a toString version of the object you passed.
If you checked the contents of the goo property, you'd see that it's now an array with a single element, which is a string that contains [object Object]. If you'd picked a more basic type or matched the destination property storage type, you would see that a basic equality check would have worked.

you can use toJSON() instead of _doc

Try using lean
By default, Mongoose queries return an instance of the Mongoose Document class. Documents are much heavier than vanilla JavaScript objects, because they have a lot of internal state for change tracking. Enabling the lean option tells Mongoose to skip instantiating a full Mongoose document and just give you the POJO.
https://mongoosejs.com/docs/tutorials/lean.html

Had same problem. Instead of updating my model.
const car = model.findOne({_id:'1'})
let temp = JSON.stringify(car);
let objCar = JSON.parse(temp);
objCar.color = 'Red'; //now add any property you want
this solves my problem

I was stuck on this today... Drove me nuts. Not sure if the below is a good solution (and OP has mentioned it too), but this is how I overcame this issue.
My car object:
cars = [{"make" : "Toyota"}, {"make" : "Kia"}];
Action:
console.log("1. Cars before the color: " + car);
cars.forEach(function(car){
car.colour = "Black"; //color is NOT defined in the model.
});
console.log("2. Cars after the color: " + car);
Problematic console output:
1. Cars before the color: [{"make" : "Toyota"}, {"make" : "Kia"}];
2. Cars after the color: [{"make" : "Toyota"}, {"make" : "Kia"}]; //No change! No new colour properties :(
If you try to pass in this property that was undefined in the model, via doc (e.g. car._doc.color = "black"), it will work (this colour property will be assigned to each car), but you can't seem to access it via EJS (frontend) for some reason.
Solution:
(Again, not sure if this is the best way... but it worked for me): Add in this new property (colour) in the car model.
var carSchema = mongoose.Schema({
make: String,
color: String //New property.
})
With the model redefined, everything worked as normal / expected (no _doc 'hacks' needed etc.) and I lived another day; hope it helps someone else.

There is some weirdness with Mongoose models and you have to check that Mongoose doesn't already have a model created in it's models array.
Here is my solution:
import mongoose from 'mongoose';
createModel = (modelName="foo", schemaDef, schemaOptions = {})=> {
const { Schema } = mongoose;
const schema = Schema(schemaDef, schemaOptions);
const Model = mongoose.models[modelName] || mongoose.model(modelName, schema);
return Model;
}
I use my own mongoose model class and base class for my models. I made this and it should work for you.

For those using spread(...) and/ can't see a solution, here's an example of #entesar's answer
Instead of spread or ._doc in:
import User from "./models/user";
...
async function createUser(req, res) {
const user = await User.create(req.body);
res.status(201).json({
message: "user created",
data: {
...user // OR user._doc,
token: "xxxxxxxx",
},
});
}
...
Use this
import User from "./models/user";
...
async function createUser(req, res) {
const user = await User.create(req.body);
res.status(201).json({
message: "user created",
data: {
...user.toJSON(),
token: "xxxxxxxx",
},
});
}
...
Ps: took me a while to understand the answer.

You should add .lean() on the find to have it skip all the Model "magic".

Related

Dynamic Mongoose Field Type

I am developing an app where a user could store his model on a database using mongoDB and mongoose. Taken from mongoose tutorial the type of the field has to be defined. For example here we have to define that the name is a string.
const personSchema = new mongoose.Schema({
name: String
});
const Person = mongoose.model('Person', personSchema);
Is there any way to make it dynamic to user's input. I want to create a form where a user will enter a field name and select one of the field types that Mongoose offers [String,Number,Date etc], but I cannot figure any way to implement it. To be honest I don't know even if this is a good approach. An alternative would be to pass everything as a String and serialise the input in order to store it. I want to achieve something like that:
const {fieldName,fieldType} = userInput;
const customSchema = new mongoose.Schema({
fieldName: fieldType
});
const CustomModel = mongoose.model('CustomSchema', customSchema);
Is this possible or should I implement another approach? An alternative would be to pass everything as a String and serialise the input in order to store it.
Thank you in advance!
If I understand you correctly it should work like that:
User defines the model to store
Schema is created using the data provided by the user
User can pass the data to store using the previously created model which will validate the user's input later
In fact, I'm working on a project that has the same functionality. Here is how we did it.
A user sends the model and we store it as a string since we need to have the ability to create the model once again.
When the user passes new data to store using the created model we get the string from mongo and parse it to create the schema. This operation is relatively easy (but depends on what you want to achieve as it can get tricky if you want to have some advanced validation) as you have to just create an object with correct values from mongoose. Something like this for every field that the user has defined.
export const fieldConverter = ({name, type}) => {
switch (type) {
case 'String':
return { [name]: String };
case 'Number':
return { [name]: Number };
...
}
When you have your object ready then you can create a model out of it.
The line with accessing your model from mongoose.models is important as the mongoose will cache the model and throw an error if you try to create it once again.
const DatasetModel =
mongoose.models["your-model-name"] ??
mongoose.model("your-model-name", new mongoose.Schema(schema));
Now when you have the model the rest is just like with the normally created one.
This approach worked for us so I'm adding this as inspiration maybe it will help you. If you have any specific questions about the implementation feel free to ask I will be happy to help.
There is also a Mixed type in mongoose if you don't need the validation later. You can check it here: https://mongoosejs.com/docs/schematypes.html#mixed
You can use Schema.Types.Mixed, An "anything goes" SchemaType. Mongoose will not do any casting on mixed paths.
let customSchema = new Schema({custom: Schema.Types.Mixed})
Read more about it here
After some research I figure at that mongoose type can also be strings. For example
const personSchema = new mongoose.Schema({
name: "String"
});
const Person = mongoose.model('Person', personSchema);
Mongoose will handle it

How can I dynamically generate Mongoose discriminators (at runtime?)

TL;DR: Is there a safe way to dynamically define a mongoose discriminator at runtime?
I have an app with a MongoDB collection where users have some control over the underlying schema.
I could add one or two fixed, required fields and just use mongoose.Mixed for the remainder that users can change, but I'd like to make use of Mongoose's validation and discriminators if I can.
So, what I've got is a second collection Grid where the users can define the shape they'd like their data to take, and in my main model Record, I've added a function to dynamically generate a discriminator from the definition in the second collection.
The code for my Record model looks like this:
const mongoose = require("mongoose")
const recordSchema = new mongoose.Schema({
fields: {
type: Array,
required: true
}
}, {
discriminatorKey: "grid"
})
const Record = mongoose.model("Record", recordSchema)
module.exports = grid => {
// Generate a mongoose-compatible schema from the grid's field definitions
const schema = grid.fields.map(field => {
if(field.type === "string") return { [field.name]: String }
if(field.type === "number") return { [field.name]: Number }
if(field.type === "checkbox") return { [field.name]: Boolean }
return { [field.name]: mongoose.Mixed }
})
return Record.discriminator(grid._id, new mongoose.Schema(schema))
}
This is inside an Express app, and I use the model in my middleware handlers something like this:
async (req, res) => {
const grid = await Grid.findById(req.params.id)
const Record = await GenerateRecordModel(grid)
const records = await Record.find({})
res.json({
...grid,
records
})
}
This works great on the first request, but after that I get an error Discriminator with name “ ” already exists.
I guess this is because only one discriminator with its name per model can exist.
I could give every discriminator a unique name whenever the function is called:
return Record.discriminator(uuidv4(), new mongoose.Schema(schema), grid._id)
But I imagine that this isn't a good idea because discriminators seem to persist beyond the lifetime of the request, so am I laying the groundwork for a memory leak?
I can see two ways forward:
COMPLICATED? Define all discriminators when the app boots up, rather than just when a HTTP request comes in, and write piles of extra logic to handle the user creating, updating or deleting the definitions over in the Grid collection.
SIMPLER? Abandon using discriminators, just use mongoose.Mixed so anything goes as far as mongoose is concerned, and write any validation myself.
Any ideas?

Add a new attribute to existing json object in node.js

I have an object like this
==================records=========={ Id: 5114a3c21203e0d811000088,
userId: 'test',
sUserId: test,
userName: 'test',
url: 'test',
Title: 'test'
}
I need to add a new field Name : 'test' to the above record, I tried giving records.Name = name, it didn't work.
Helps please
Thanks,
Prats
I am assuming you are trying to add a property to a returned Mongoose Document to reuse it somewhere else. Documents returned by Mongoose are not JSON objects directly, you'll need to convert them to an object to add properties to them. The following should work:
//... record is a mongoose Document
var r = record.toObject();
r.Name = 'test';
console.log("Record ",r);
Those finding this problem, OP mentioned in a comment below the original question that the solution for this problem is:
records.set('Name', 'test')
This adds a new attribute called Name having value test.
Just use,
var convertedJSON = JSON.parse(JSON.stringify(mongooseReturnedDocument);
and Then,
convertedJSON.newProperty = 'Hello!'
'Hello!' can be anything, a number, a object or JSON Object Literal.
Cheers! :)
I experienced a similar problem, and hope that my hours of existential frustration help others in the same situation. My inclination was to believe that documents returned via Mongoose are read-only. This is actually not true.
However, you cannot assign a property to your document that is not also in your Schema.
So, unless your schema has the following:
{
Name: {String}
}
you will be constantly frustrated by trying to assign Name to your document.
Now, there are workarounds in the answers above that also worked for me, but they do not get to the root of the problem:
myDocument.toObject();
JSON.parse(JSON.stringify(myDocument);
These will work, but in my opinion they just hide the problem. The real issue is that Mongoose is smarter than we realized and is enforcing your schema, as it should.
You could also use the lean() method, e.g. const users = await Users.find().lean().exec()
From the mongoose documentation:
Documents returned from queries with the lean option enabled are plain
javascript objects, not MongooseDocuments. They have no save method,
getters/setters or other Mongoose magic applied
My variant.
If schema defined with {strict:false} you can simply add new property by
recorcd.newProp = 'newvalue';
If schema defined with {strict:true} you can either convert Mongoose object to object as mentioned earlier or use command
record.set('newProp','newValue',{strict:false})
See http://mongoosejs.com/docs/api.html#document_Document-schema
If you have loaded this object into records, both records.Name = "test" or records['Name'] = "test" will work. You have either not loaded the object correctly, or are inserting an undefined value into it.
To test: add console.log(records.userId), this should print 'test' to the terminal.
Also add console.log(name). If you get ReferenceError: name is not defined, you obviously cannot do: records.Name = name

Underscore.js _.extend function does not copy mongoose model properties when they are not defined in the schema?

Why can I not use underscore(_) extend to update mongoose models where the properties are not defined in the schema definition. Is there a way to get around this?
Node model:
var mongoose = require('mongoose')
, Schema = mongoose.Schema
var NodeSchema = new Schema({
label: {type : String, default : 'none'}
}, { strict: false })
mongoose.model('Node', NodeSchema)
Node Controller:
var node = new Node();
node = _.extend(node, {"EXTENDNOTinSchema":"TRUE"});
console.log("extend: " + node);
node.set("SETNOTinSchema","TRUE");
console.log("set: " + node);
Console Output:
extend: { __v: 0,
_id: 50bb05656880a68976000001,
label: 'none' }
set: { __v: 0,
_id: 50bb05656880a68976000001,
label: 'none'
SETNOTinSchema: TRUE}
This is happening because if something is not in the schema then Mongoose cannot use 'defineProperty', and this treats the assignment like any other.
So first off, just to be clear.
node = _.extend(node, {"EXTENDNOTinSchema":"TRUE"});
is identical to this:
node['EXTENDNOTinSchema'] = 'TRUE';
Which is entirely different from this, in the general case.
node.set("SETNOTinSchema","TRUE");
The trick is that Mongoose is smart, and using the defineProperty function I mentioned above, it can bind a function to get called for things like this:
node['INSCHEMA'] = 'something';
But for things that are not in the schema, it cannot do this, so the assignment works like a normal assignment.
The part that tripping you up I think is that console.log is doing some hidden magic. If you look at the docs, console.log will call the inspect method of an object that is passed to it. In the case of Mongoose, it's models do not store attributes directly on the model object, they are stored on an internal property. When you assign to a property being watched with defineProperty or call set, it stores the value on the internal object. When you log the model, inspect prints out the internal model contents, making it seem like the model values are stored right on the object.
So when you do
console.log(node);
what you are really seeing is
console.log(node.somehiddenproperty);
So the answer to your question is really, if you have a bunch of values that are not in the schema, you cannot use _.extend. Instead, just use set because it takes an object anyway.
node.set({"EXTENDNOTinSchema":"TRUE"});

How do you turn a Mongoose document into a plain object?

I have a document from a mongoose find that I want to extend before JSON encoding and sending out as a response. If I try adding properties to the doc it is ignored. The properties don't appear in Object.getOwnPropertyNames(doc) making a normal extend not possible. The strange thing is that JSON.parse(JSON.encode(doc)) works and returns an object with all of the correct properties. Is there a better way to do this?
Mongoose Models inherit from Documents, which have a toObject() method. I believe what you're looking for should be the result of doc.toObject().
http://mongoosejs.com/docs/api.html#document_Document-toObject
Another way to do this is to tell Mongoose that all you need is a plain JavaScript version of the returned doc by using lean() in the query chain. That way Mongoose skips the step of creating the full model instance and you directly get a doc you can modify:
MyModel.findOne().lean().exec(function(err, doc) {
doc.addedProperty = 'foobar';
res.json(doc);
});
JohnnyHK suggestion:
In some cases as #JohnnyHK suggested, you would want to get the Object as a Plain Javascript.
as described in this Mongoose Documentation there is another alternative to query the data directly as object:
const docs = await Model.find().lean();
Conditionally return Plain Object:
In addition if someone might want to conditionally turn to an object,it is also possible as an option argument, see find() docs at the third parameter:
const toObject = true;
const docs = await Model.find({},null,{lean:toObject});
its available on the functions: find(), findOne(), findById(), findOneAndUpdate(), and findByIdAndUpdate().
NOTE:
it is also worth mentioning that the _id attribute isn't a string object as if you would do JSON.parse(JSON.stringify(object)) but a ObjectId from mongoose types, so when comparing it to strings cast it to string before: String(object._id) === otherStringId
the fast way if the property is not in the model :
document.set( key,value, { strict: false });
A better way of tackling an issue like this is using doc.toObject() like this
doc.toObject({ getters: true })
other options include:
getters: apply all getters (path and virtual getters)
virtuals: apply virtual getters (can override getters option)
minimize: remove empty objects (defaults to true)
transform: a transform function to apply to the resulting document before returning
depopulate: depopulate any populated paths, replacing them with their original refs (defaults to false)
versionKey: whether to include the version key (defaults to true)
so for example you can say
Model.findOne().exec((err, doc) => {
if (!err) {
doc.toObject({ getters: true })
console.log('doc _id:', doc._id)
}
})
and now it will work.
For reference, see: http://mongoosejs.com/docs/api.html#document_Document-toObject
To get plain object from Mongoose document, I used _doc property as follows
mongooseDoc._doc //returns plain json object
I tried with toObject but it gave me functions,arguments and all other things which i don't need.
The lean option tells Mongoose to skip hydrating the result documents. This makes queries faster and less memory intensive, but the result documents are plain old JavaScript objects (POJOs), not Mongoose documents.
const leanDoc = await MyModel.findOne().lean();
not necessary to use JSON.parse() method
You can also stringify the object and then again parse to make the normal object.
For example like:-
const obj = JSON.parse(JSON.stringify(mongoObj))
I have been using the toObject method on my document without success.
I needed to add the flattenMap property to true to finally have a POJO.
const data = document.data.toObject({ flattenMaps: true });

Resources