So I've been trying to save CSP reports into Mongoose with a Mixed schema and have ran into a snag of sorts.
If I try to save anything using the "schema-less" way, it only saves the default _v and _id fields
ViolationSchema = new Schema({});
Violation = mongoose.model('CSPViolation', ViolationSchema);
... wait for POST ...
new Violation( req.body ).save( callback );
// { _id : <some_id>, _v : <some_hash> }
If I set a field in the schema to be Mixed and add a .markModified() to the field, it will save.
ViolationSchema = new Schema({ report : { type : Mixed } });
Violation = mongoose.model('CSPViolation', ViolationSchema);
... wait for POST ...
var v = new Violation( { report : req.body } );
v.markModified('report');
v.save( callback );
// report saved under v.report.<actual_report>
I thought about using native MongoDB-style collection.insert, however it doesn't look like the model has an insert method (nor the schema for that matter).
I suppose I could also go over each key in the report I'm saving and manually mark it as modified, but I'd like to avoid that just to store a report such as this.
Any ideas how I can blindly save a mixed schema type using Mongoose?
It looks like this can be done by setting { strict : false } on the schema. This ensures that Mongoose will save any fields that weren't declared in the original schema.
Normally this isn't something you would enable on 95% of your data, it just fits perfectly with what I'm trying to do currently.
Example
ViolationSchema = new Schema({ type: Mixed }, { strict : false });
Violation = mongoose.model('CSPViolation', ViolationSchema);
... wait for POST ...
new Violation( req.body ).save( callback );
// Saves with full data
Related
UserEventsInfo = new mongoose.Schema({
name: String,
username: String,
event_movie:[String],
event_tour:[String],
event_restaurant:[String],
event_lifetimeevents:[String]
},{strict : false});
I am able to insert new key-value pair other than defined in the schema
but when I try to read the value of that key. I can't. I am using the following code.
UserEventsDetails.find({username:username},function(err,docs){
if(!docs.length)
{
res.send('datanotavailable');
}
else{
res.send(docs[0][eventname]);
}
});
Here eventname is a variable.
When I add that key in the schema it returns the value i.e. work's fine.
Otherwise it is not returning any value.
Looks like there was an issue submitted like this to mongoose. Here is there response:
The benefit we see in a schemaless database is the ability for our data model to evolve as fast as our features require it, without a linear impact on performance and slower deployment cycles with needless migrations.
If you don't want your data to be normalized and validated prior to saving, then you don't need a tool like Mongoose, you can use the driver directly.
After a little digging there is a way to do this, but you will need to have a field with type Schema.Types.Mixed. So it would look like this:
var schema = new Schema({
mixed: Schema.Types.Mixed,
});
var Thing = mongoose.model('Thing', schema);
var m = new Thing;
m.mixed = { any: { thing: 'i want' } };
m.save(callback);
To do a find on a mixed this SO question answers that.
****EDIT
forgot to link the documentation of mixed types
I have a schema with a field in which I can store anything : new Schema({settings : {}}).
I have a database with
I want to keep this ability to add data without adding new fields, but for some of the fields have default values if they are not present.
I can do the following :
new Schema({
settings : {
key : { type : String, default: "abc" }
// I want to be able to add data that contains more than just "key"
}
});
I just want to make sure that when requesting the data from this schema, I will still get all the data, and not just the keys explicitly defined ?
It seems to work, but I want to make sure that I can still :
read all the data
still write arbitrary data (ie. not necessarily defined in the schema)
Are there rules on mongo/mongoose that would prevent me from doing one of these two things (I'm very unsure for the writing part) ? If there is such a "feature", how can it be done ?
Note : I saw this question. Correct me if I am wrong, but the fields are not implicit (like in the first case with {}), and have to be defined (it's actually the opposite question).
Edit : I now saw also this question that addresses my concerns (even if the accepted solution sounds more like a workaround to me). But in my case I already have data stored so (1 - disable strict) would mean writing a lot of validation code to be safe (because a lot of keys, this is the biggest collection of the app), and (2 - mixed schemas) would require to migrate the data of this specific sub-element... In short : I would still welcome a solution to my particular problem.
I think you will want to build your own custom validation here rather than rely on the defauly schema type validation methods. Luckily, mongoose has a facility for this:
var async = require('async'),
mongoose = require('mongoose'),
Schema = mongoose.Schema;
mongoose.connect('mongodb://localhost/test');
var testSchema = new Schema({
settings: {}
});
var Test = mongoose.model( 'Test', testSchema, "test" );
testSchema.path('settings').validate(function(value) {
return Object.keys(value).indexOf("key") != -1;
},'Error "settings" must contain "key" as "settings.key"');
//var test = new Test({ settings: { "key": "something" } });
var test = new Test({ settings: { } });
test.save(function(err) {
try {
if (err) throw err;
console.log(test);
} catch (e) {
console.log(e);
}
});
So basically, I have set up the validate function there for the "settings" path in the schema to look for the presence of "key" with it's own object. Where that "key" does not exist, an exception is reported in the errors.
Like any such errors, it will be returned within the err object when you .save() the object, thus blocking the write. It can be then be acted on to handle the error however you want, with the message that was defined reported.
So that is a self contained "test" where you can alternately uncomment the valid data for the object and successfully save that object without errors being reported.
Alternately you can do a "pre save" to fill in some default data:
testSchema.pre("save",function(next) {
this.settings = (this.settings) ? this.settings : {};
if (Object.keys(this.settings).indexOf("key") == -1)
this.setting.key = "abc";
next();
});
Which fills in a default if it is not already there.
Try to use like this way
new Schema({
settings : {}
});
var modelObj = new myModel();
modelObj.settings.key = "keyval";
modelObj.settings.key1 = "keyval";
modelObj.settings.key2 = "keyval";
modelObj.settings.key3 = "keyval";
modelObj.save(function(err){
//handle
});
I have a Mongoose schema that looks like this:
ManifestSchema = new Schema({
entries: [{
order_id: String,
line_item: {}, // <-- resolved at run time
address: {},// <-- resolved at run time
added_at: Number,
stop: Number,
}]
}, {collection: 'manifests', strict: true });
and somewhere in the code I have this:
Q.ninvoke(Manifests.findById(req.params.id), 'exec')
.then(function(manifest)
{
// ... so many things, like resolving the address and the item information
entry.line_item = item;
entry.address = order.delivery.address;
})
The issue that I faced is that without defining address and line_item in the schema, when I resolved them at run time, they wouldn't returned to the user because they weren't in the schema...so I added them...which cause me another unwanted behavior: When I saved the object back, both address and line_item were saved with the manifest object, something that I would like to avoid.
Is there anyway to enable adding fields to the schema at run time, but yet, not saving them on the way back?
I was trying to use 'virtuals' in mongoose, but they really provide what I need because I don't create the model from a schema, but it rather returned from the database.
Call toObject() on your manifest Mongoose instance to create a plain JavaScript copy that you can add extra fields to for the user response without affecting the doc you need to save:
Q.ninvoke(Manifests.findById(req.params.id), 'exec')
.then(function(manifest)
{
var manifestResponse = manifest.toObject();
// ... so many things, like resolving the address and the item information
entry.line_item = item;
entry.address = order.delivery.address;
})
It seems Mongoose is doing something really funky internally.
var Foo = new mongoose.model('Foo', new mongoose.Schema({a: String, b: Number}));
var foo = new Foo({a: 'test'; b: 42});
var obj = {c: 1};
foo.goo = obj; // simple object assignment. obj should be
// passed by reference to foo.goo. recall goo
// is not defined in the Foo model schema
console.log(foo.goo === obj); // comparison directly after the assignment
// => false, doesn't behave like normal JS object
Essentially, any time you try to deal with properties of a Mongoose model that aren't
a) defined in the model's schema or
b) defined as the same type (array, obj, ..) ... the model doesn't even behave like a normal Javascript object.
Switching line 4 to foo._doc.goo = obj makes the console output true.
edit: trying to reproduce weirdness
example 1:
// Customer has a property 'name', but no property 'text'
// I do this because I need to transform my data slightly before sending it
// to client.
models.Customer.find({}, function(err, data) {
for (var i=0, len=data.length; i<len; ++i) {
data[i] = data[i]._doc; // if I don't do this, returned data
// has no 'text' property
data[i].text = data[i].name;
}
res.json({success: err, response:data});
});
_doc exist on the mongoose object.
Because mongooseModel.findOne returns the model itself, the model has structure (protected fields).
When you try to print the object with console.log it gives you only the data from the database, because console.log will print the object public fields.
If you try something like JSON.stringify then you get to see inside the mongoose model object. (_doc, state ...)
In the case where you want to add more fields in the object and it's not working
const car = model.findOne({_id:'1'})
car.someNewProp = true // this will not work
If later you set the property to the object car and you didn't specify in the Model Schema before then Mongoose model is validating if this field exists and if it's the valid type.
If the validation fails then the property will not be set.
Update
Maybe I misunderstood your original question, but now it looks like the nature of your question changed, so the below information isn't relevant, but I'm leaving it. :)
I tested your code and it works fine for me. Mongoose doesn't execute any special code when you set properties that aren't part of the schema (or a few other special properties). JavaScript currently doesn't support calling code for properties that don't yet exist (so Mongoose can't get in the way of the set of the goo property for example).
So, when you set the property:
foo.goo = { c: 1 };
Mongoose isn't involved. If your console.log was something other than the code you displayed, I could see that it might report incorrectly.
Additionally, when you send the results back as JSON, JSON.stringify is being called, which calls toString on your Mongoose Model. When that happens, Mongoose only uses the properties defined on the schema. So, no additional properties are being sent back by default. You've changed the nature of the data array though to directly point at the Mongoose data, so it avoids that problem.
Details about normal behavior
When you set the property goo using Mongoose, quite a few things happen. Mongoose creates property getters/setters via the Object.defineProperty (some docs). So, when you set the goo property, which you've defined as a [String], a few things happen:
Mongoose code is called prior to the value being set onto the object instance (unlike a simple JavaScript object)
Mongoose creates an array (optionally) to store the data (a MongooseArray) which will contain the array data. In the example you provided, since you didn't pass an array, it will be created.
Mongoose will attempt to cast your data to the right type
It will call toString on the data passed as part of the cast.
So, the results are that the document now contains an array with a toString version of the object you passed.
If you checked the contents of the goo property, you'd see that it's now an array with a single element, which is a string that contains [object Object]. If you'd picked a more basic type or matched the destination property storage type, you would see that a basic equality check would have worked.
you can use toJSON() instead of _doc
Try using lean
By default, Mongoose queries return an instance of the Mongoose Document class. Documents are much heavier than vanilla JavaScript objects, because they have a lot of internal state for change tracking. Enabling the lean option tells Mongoose to skip instantiating a full Mongoose document and just give you the POJO.
https://mongoosejs.com/docs/tutorials/lean.html
Had same problem. Instead of updating my model.
const car = model.findOne({_id:'1'})
let temp = JSON.stringify(car);
let objCar = JSON.parse(temp);
objCar.color = 'Red'; //now add any property you want
this solves my problem
I was stuck on this today... Drove me nuts. Not sure if the below is a good solution (and OP has mentioned it too), but this is how I overcame this issue.
My car object:
cars = [{"make" : "Toyota"}, {"make" : "Kia"}];
Action:
console.log("1. Cars before the color: " + car);
cars.forEach(function(car){
car.colour = "Black"; //color is NOT defined in the model.
});
console.log("2. Cars after the color: " + car);
Problematic console output:
1. Cars before the color: [{"make" : "Toyota"}, {"make" : "Kia"}];
2. Cars after the color: [{"make" : "Toyota"}, {"make" : "Kia"}]; //No change! No new colour properties :(
If you try to pass in this property that was undefined in the model, via doc (e.g. car._doc.color = "black"), it will work (this colour property will be assigned to each car), but you can't seem to access it via EJS (frontend) for some reason.
Solution:
(Again, not sure if this is the best way... but it worked for me): Add in this new property (colour) in the car model.
var carSchema = mongoose.Schema({
make: String,
color: String //New property.
})
With the model redefined, everything worked as normal / expected (no _doc 'hacks' needed etc.) and I lived another day; hope it helps someone else.
There is some weirdness with Mongoose models and you have to check that Mongoose doesn't already have a model created in it's models array.
Here is my solution:
import mongoose from 'mongoose';
createModel = (modelName="foo", schemaDef, schemaOptions = {})=> {
const { Schema } = mongoose;
const schema = Schema(schemaDef, schemaOptions);
const Model = mongoose.models[modelName] || mongoose.model(modelName, schema);
return Model;
}
I use my own mongoose model class and base class for my models. I made this and it should work for you.
For those using spread(...) and/ can't see a solution, here's an example of #entesar's answer
Instead of spread or ._doc in:
import User from "./models/user";
...
async function createUser(req, res) {
const user = await User.create(req.body);
res.status(201).json({
message: "user created",
data: {
...user // OR user._doc,
token: "xxxxxxxx",
},
});
}
...
Use this
import User from "./models/user";
...
async function createUser(req, res) {
const user = await User.create(req.body);
res.status(201).json({
message: "user created",
data: {
...user.toJSON(),
token: "xxxxxxxx",
},
});
}
...
Ps: took me a while to understand the answer.
You should add .lean() on the find to have it skip all the Model "magic".
i have a model schema as :
var A = new Schema ({
a: String,
b : [ { ba: Integer, bb: String } ]
}, { collection: 'a' } );
then
var M = mongoose.model("a", A);
var saveid = null;
var m = new M({a:"Hello"});
m.save(function(err,model){
saveid = model.id;
}); // say m get the id as "1"
then
m['b'].push({ba:235,bb:"World"});
m.save(function(err,model){
console.log(model.id); //this will print 1, that is the id of the main Document only.
//here i want to find the id of the subdocument i have just created by push
});
So my question is how to find the id of the subdocument just pushed in one field of the model.
I've been looking for this answer as well, and I'm not sure that I like accessing the last document of the array. I do have an alternative solution, however. The method m['b'].push will return an integer, 1 or 0 - I'm assuming that is based off the success of the push (in terms of validation). However, in order to get access to the subdocument, and particularly the _id of the subdocument - you should use the create method first, then push.
The code is as follows:
var subdoc = m['b'].create({ ba: 234, bb: "World" });
m['b'].push(subdoc);
console.log(subdoc._id);
m.save(function(err, model) { console.log(arguments); });
What is happening is that when you pass in the object to either the push or the create method, the Schema cast occurs immediately (including things like validation and type casting) - this means that this is the time that the ObjectId is created; not when the model is saved back to Mongo. In fact, mongo does not automatically assign _id values to subdocuments this is a mongoose feature. Mongoose create is documented here: create docs
You should also note therefore, that even though you have a subdocument _id - it is not yet in Mongo until you save it, so be weary of any DOCRef action that you might take.
The question is "a bit" old, but what I do in this kind of situation is generate the subdocument's id before inserting it.
var subDocument = {
_id: mongoose.Types.ObjectId(),
ba:235,
bb:"World"
};
m['b'].push(subDocument);
m.save(function(err,model){
// I already know the id!
console.log(subDocument._id);
});
This way, even if there are other database operations between the save and the callback, it won't affect the id already created.
Mongoose will automatically create an _id for each new sub document, but - as far as I know - doesn't return this when you save it.
So you need to get it manually. The save method will return the saved document, including the subdocs. As you're using push you know it will be the last item in the array, so you can access it from there.
Something like this should do the trick.
m['b'].push({ba:235,bb:"World"});
m.save(function(err,model){
// model.b is the array of sub documents
console.log(model.b[model.b.length-1].id);
});
If you have a separate schema for your subdocument, then you can create the new subdocument from a model before you push it on to your parent document and it will have an ID:
var bSchema = new mongoose.Schema({
ba: Integer,
bb: String
};
var a = new mongoose.Schema({
a: String,
b : [ bSchema ]
});
var bModel = mongoose.model('b', bSchema);
var subdoc = new bModel({
ba: 5,
bb: "hello"
});
console.log(subdoc._id); // Voila!
Later you can add it to your parent document:
m['b'].push(subdoc)
m.save(...