I'm working with some old data where some of the schema has a "mixed" type.
Basically sometimes a value will be a referenced ObjectID, but other times it'll be some text (super poor design).
I unable to correctly populate this data because of the times a non-ObjectID appears.
So, for my actual question: Is it possible to create a populate (on a collection) that is conditional; I need to be able to tell the populate to skip those other values.
Yes, you can do that check the middleware function on the Mongoose API reference
http://mongoosejs.com/docs/middleware.html
What you need to do is before you populate those data, you validate the data if is is Object ID or not, if it is Object ID, you call next() to pass the next function, else you just return, this will skip it
Example
xSchema.pre('validate', function(next){
var x = this;
var checkXType = typeof x.id;
if (checkXType === String) {
return;
} else {
next();
}
});
Related
When using Mongoose (with bluebird in my case, but using callbacks to illustrate), the following codes all return a document from the collection:
model.findOne({}, function(err, document) {
//returns a document
})
model.findOne(null, function(err, document) {
//returns a document
})
model.findOne([], function(err, document) {
//returns a document
})
I would like to know if and how I can disable this kind of behaviour, as it is becoming a liability to my code where I infer queries from data a user feeds into the system. Especially the null query returning a valid document worries me.
As of right now I check the input for being an non-empty, non-array, non-null object, but it's becoming a bit cumbersome at scale.
What would be the best way to exclude this behaviour?
Not sure if it is the best way to go about it, but right now I've settled on using a pre-hook on the model itself which checks for the _conditions property of the 'this' object (which I inferred from printing seems to hold the query object) to not be empty.
Inserting a self-defined object in the next functionality causes the Promise to reject in which the query was originally called from.
( _ is the underscore package)
//model.js
//model is a mongoose.Schema type in the following code
model.pre('findOne', function(next) {
var self = this
if (_.isEmpty(self._conditions)) {
next(mainErrors.malformedRequest)
} else {
next()
}
})
I have a data feed from a 3rd party server that I am pulling in and converting to JSON. The data feed will never have my mongoDB's auto-generated _ids in it, but there is a unique identifier called vehicle_id.
The function below is what is handling taking the data-feed generated json object fresh_event_obj and copying its values into a mongo document if there is a mongo document with the same vehicle_id.
function update_vehicle(fresh_event_obj) {
console.log("Updating Vehicle " + fresh_event_obj.vehicleID + "...");
Vehicle.find({ vehicleID: fresh_event_obj.vehicleID }, function (err, event_obj) {
if (err) {
handle_error(err);
} else {
var updated = _.merge(event_obj[0], fresh_event_obj);
updated.save(function (err) {
if (err) {
handle_error(err)
} else {
console.log("Vehicle Updated");
}
});
}
});
}
The structures of event_obj[0] and fresh_event_obj are identical, except that event_obj[0] has _id and __v while the "normal" object doesn't.
When I run _.merge on these two, or even my own recursive function that just copies values from the latter to the former, nothing in the updated object is different from the event_obj[0], despite fresh_event_obj having all new values.
Does anyone have any idea what I'm doing wrong? I feel it is obvious and I'm just failing to see it.
The problem is that if you don't have properties defined in your schema, and if they don't already exist, you can't create them with
doc.prop = value
even if you have {strict:false} in your schema.
The only way to set new properties is to do
doc.set('prop', value)
(You still have to have {strict:false} in your schema if that property doesn't exist in your schema)
As for having too many properties to be defined in schema, you can always use for-in loop to go through object properties
for(key in fresh_event_obj)
event_obj.set(key, fresh_event_obj[key]);
It seems Mongoose is doing something really funky internally.
var Foo = new mongoose.model('Foo', new mongoose.Schema({a: String, b: Number}));
var foo = new Foo({a: 'test'; b: 42});
var obj = {c: 1};
foo.goo = obj; // simple object assignment. obj should be
// passed by reference to foo.goo. recall goo
// is not defined in the Foo model schema
console.log(foo.goo === obj); // comparison directly after the assignment
// => false, doesn't behave like normal JS object
Essentially, any time you try to deal with properties of a Mongoose model that aren't
a) defined in the model's schema or
b) defined as the same type (array, obj, ..) ... the model doesn't even behave like a normal Javascript object.
Switching line 4 to foo._doc.goo = obj makes the console output true.
edit: trying to reproduce weirdness
example 1:
// Customer has a property 'name', but no property 'text'
// I do this because I need to transform my data slightly before sending it
// to client.
models.Customer.find({}, function(err, data) {
for (var i=0, len=data.length; i<len; ++i) {
data[i] = data[i]._doc; // if I don't do this, returned data
// has no 'text' property
data[i].text = data[i].name;
}
res.json({success: err, response:data});
});
_doc exist on the mongoose object.
Because mongooseModel.findOne returns the model itself, the model has structure (protected fields).
When you try to print the object with console.log it gives you only the data from the database, because console.log will print the object public fields.
If you try something like JSON.stringify then you get to see inside the mongoose model object. (_doc, state ...)
In the case where you want to add more fields in the object and it's not working
const car = model.findOne({_id:'1'})
car.someNewProp = true // this will not work
If later you set the property to the object car and you didn't specify in the Model Schema before then Mongoose model is validating if this field exists and if it's the valid type.
If the validation fails then the property will not be set.
Update
Maybe I misunderstood your original question, but now it looks like the nature of your question changed, so the below information isn't relevant, but I'm leaving it. :)
I tested your code and it works fine for me. Mongoose doesn't execute any special code when you set properties that aren't part of the schema (or a few other special properties). JavaScript currently doesn't support calling code for properties that don't yet exist (so Mongoose can't get in the way of the set of the goo property for example).
So, when you set the property:
foo.goo = { c: 1 };
Mongoose isn't involved. If your console.log was something other than the code you displayed, I could see that it might report incorrectly.
Additionally, when you send the results back as JSON, JSON.stringify is being called, which calls toString on your Mongoose Model. When that happens, Mongoose only uses the properties defined on the schema. So, no additional properties are being sent back by default. You've changed the nature of the data array though to directly point at the Mongoose data, so it avoids that problem.
Details about normal behavior
When you set the property goo using Mongoose, quite a few things happen. Mongoose creates property getters/setters via the Object.defineProperty (some docs). So, when you set the goo property, which you've defined as a [String], a few things happen:
Mongoose code is called prior to the value being set onto the object instance (unlike a simple JavaScript object)
Mongoose creates an array (optionally) to store the data (a MongooseArray) which will contain the array data. In the example you provided, since you didn't pass an array, it will be created.
Mongoose will attempt to cast your data to the right type
It will call toString on the data passed as part of the cast.
So, the results are that the document now contains an array with a toString version of the object you passed.
If you checked the contents of the goo property, you'd see that it's now an array with a single element, which is a string that contains [object Object]. If you'd picked a more basic type or matched the destination property storage type, you would see that a basic equality check would have worked.
you can use toJSON() instead of _doc
Try using lean
By default, Mongoose queries return an instance of the Mongoose Document class. Documents are much heavier than vanilla JavaScript objects, because they have a lot of internal state for change tracking. Enabling the lean option tells Mongoose to skip instantiating a full Mongoose document and just give you the POJO.
https://mongoosejs.com/docs/tutorials/lean.html
Had same problem. Instead of updating my model.
const car = model.findOne({_id:'1'})
let temp = JSON.stringify(car);
let objCar = JSON.parse(temp);
objCar.color = 'Red'; //now add any property you want
this solves my problem
I was stuck on this today... Drove me nuts. Not sure if the below is a good solution (and OP has mentioned it too), but this is how I overcame this issue.
My car object:
cars = [{"make" : "Toyota"}, {"make" : "Kia"}];
Action:
console.log("1. Cars before the color: " + car);
cars.forEach(function(car){
car.colour = "Black"; //color is NOT defined in the model.
});
console.log("2. Cars after the color: " + car);
Problematic console output:
1. Cars before the color: [{"make" : "Toyota"}, {"make" : "Kia"}];
2. Cars after the color: [{"make" : "Toyota"}, {"make" : "Kia"}]; //No change! No new colour properties :(
If you try to pass in this property that was undefined in the model, via doc (e.g. car._doc.color = "black"), it will work (this colour property will be assigned to each car), but you can't seem to access it via EJS (frontend) for some reason.
Solution:
(Again, not sure if this is the best way... but it worked for me): Add in this new property (colour) in the car model.
var carSchema = mongoose.Schema({
make: String,
color: String //New property.
})
With the model redefined, everything worked as normal / expected (no _doc 'hacks' needed etc.) and I lived another day; hope it helps someone else.
There is some weirdness with Mongoose models and you have to check that Mongoose doesn't already have a model created in it's models array.
Here is my solution:
import mongoose from 'mongoose';
createModel = (modelName="foo", schemaDef, schemaOptions = {})=> {
const { Schema } = mongoose;
const schema = Schema(schemaDef, schemaOptions);
const Model = mongoose.models[modelName] || mongoose.model(modelName, schema);
return Model;
}
I use my own mongoose model class and base class for my models. I made this and it should work for you.
For those using spread(...) and/ can't see a solution, here's an example of #entesar's answer
Instead of spread or ._doc in:
import User from "./models/user";
...
async function createUser(req, res) {
const user = await User.create(req.body);
res.status(201).json({
message: "user created",
data: {
...user // OR user._doc,
token: "xxxxxxxx",
},
});
}
...
Use this
import User from "./models/user";
...
async function createUser(req, res) {
const user = await User.create(req.body);
res.status(201).json({
message: "user created",
data: {
...user.toJSON(),
token: "xxxxxxxx",
},
});
}
...
Ps: took me a while to understand the answer.
You should add .lean() on the find to have it skip all the Model "magic".
I have a node.js application that pulls some data and sticks it into an object, like this:
var results = new Object();
User.findOne(query, function(err, u) {
results.userId = u._id;
}
When I do an if/then based on that stored ID, the comparison is never true:
if (results.userId == AnotherMongoDocument._id) {
console.log('This is never true');
}
When I do a console.log of the two id's, they match exactly:
User id: 4fc67871349bb7bf6a000002 AnotherMongoDocument id: 4fc67871349bb7bf6a000002
I am assuming this is some kind of datatype problem, but I'm not sure how to convert results.userId to a datatype that will result in the above comparison being true and my outsourced brain (aka Google) has been unable to help.
Mongoose uses the mongodb-native driver, which uses the custom ObjectID type. You can compare ObjectIDs with the .equals() method. With your example, results.userId.equals(AnotherMongoDocument._id). The ObjectID type also has a toString() method, if you wish to store a stringified version of the ObjectID in JSON format, or a cookie.
If you use ObjectID = require("mongodb").ObjectID (requires the mongodb-native library) you can check if results.userId is a valid identifier with results.userId instanceof ObjectID.
Etc.
ObjectIDs are objects so if you just compare them with == you're comparing their references. If you want to compare their values you need to use the ObjectID.equals method:
if (results.userId.equals(AnotherMongoDocument._id)) {
...
}
converting object id to string(using toString() method) will do the job.
The three possible solutions suggested here have different use cases.
Use .equals when comparing ObjectId on two mongoDocuments like this
results.userId.equals(AnotherMongoDocument._id)
Use .toString() when comparing a string representation of ObjectId to an ObjectId of a mongoDocument. like this
results.userId === AnotherMongoDocument._id.toString()
According to the above,i found three ways to solve the problem.
AnotherMongoDocument._id.toString()
JSON.stringify(AnotherMongoDocument._id)
results.userId.equals(AnotherMongoDocument._id)
The accepted answers really limit what you can do with your code. For example, you would not be able to search an array of Object Ids by using the equals method. Instead, it would make more sense to always cast to string and compare the keys.
Here's an example answer in case if you need to use indexOf() to check within an array of references for a specific id. assume query is a query you are executing, assume someModel is a mongo model for the id you are looking for, and finally assume results.idList is the field you are looking for your object id in.
query.exec(function(err,results){
var array = results.idList.map(function(v){ return v.toString(); });
var exists = array.indexOf(someModel._id.toString()) >= 0;
console.log(exists);
});
I faced exactly the same problem and i simply resolved the issue with the help of JSON.stringify() as follow:-
if (JSON.stringify(results.userId) === JSON.stringify(AnotherMongoDocument._id)) {
console.log('This is never true');
}
Mongoose from 5 to 6 migration guide:
"Mongoose now adds a valueOf() function to ObjectIds. This means you can now use == to compare an ObjectId against a string."
https://mongoosejs.com/docs/migrating_to_6.html#objectid-valueof
Here is an example that explains the issue and why it confusing for many. Only the first console log shows the object in its true form, and any other debuging/loging will be confusing because they look the same.
// Constructor for an object that has 'val' and some other stuffs
// related to to librery...
function ID(_val) {
this.val = _val;
this.otherStuff = "other stuffs goes here";
}
// function to help user get usefull infos from the Object
ID.prototype.toString = function toString() {
return `${this.val}`;
};
// Create new Object of type ID
const id = new ID('1234567');
console.log("my ID: ", id); // my ID: Object {
// val: "1234567",
// otherStuff: "other stuffs goes here"
// }
console.log("my ID: " + id); // my ID: 1234567
console.log(id === '1234567'); // false
console.log(id == '1234567'); // true
console.log(id.toString() === '1234567'); //true
console.log(`${id}` === '1234567'); // true
console.log(new ID('1234567') === id); // false
NodeJS + Express, MongoDB + Mongoose
I have a JSON feed where each record has a set of "venue" attributes (things like "venue name" "venue location" "venue phone" etc). I want to create a collection of all venues in the feed -- one instance of each venue, no dupes.
I loop through the JSON and test whether the venue exists in my venue collection. If it doesn't, save it.
jsonObj.events.forEach(function(element, index, array){
Venue.findOne({'name': element.vname}, function(err,doc){
if(doc == null){
var instance = new Venue();
instance.name = element.vname;
instance.location = element.location;
instance.phone = element.vphone;
instance.save();
}
}
}
Desired: A list of all venues (no dupes).
Result: Plenty of dupes in the venue collection.
Basically, the loop created a new Venue record for every record in the JSON feed.
I'm learning Node and its async qualities, so I believe the for loop finishes before even the first save() function finishes -- so the if statement is always checking against an empty collection. Console.logging backs this claim up.
I'm not sure how to rework this so that it performs the desired task. I've tried caolan's async module but I can't get it to help. There's a good chance I'm using incorrectly.
Thanks so much for pointing me in the right direction -- I've searched to no avail. If the async module is the right answer, I'd love your help with how to implement it in this specific case.
Thanks again!
Why not go the other way with it? You didn't say what your persistence layer is, but it looks like mongoose or possibly FastLegS. In either case, you can create a Unique Index on your Name field. Then, you can just try to save anything, and handle the error if it's a unique index violation.
Whatever you do, you must do as #Paul suggests and make a unique index in the database. That's the only way to ensure uniqueness.
But the main problem with your code is that in the instance.save() call, you need a callback that triggers the next iteration, otherwise the database will not have had time to save the new record. It's a race condition. You can solve that problem with caolan's forEachSeries function.
Alternatively, you could get an array of records already in the Venue collection that match an item in your JSON object, then filter the matches out of the object, then iteratively add each item left in the filtered JSON object. This will minimize the number of database operations by not trying to create duplicates in the first place.
Venue.find({'name': { $in: jsonObj.events.map(function(event){ return event.vname; }) }}, function (err, docs){
var existingVnames = docs.map(function(doc){ return doc.name; });
var filteredEvents = jsonObj.events.filter(function(event){
return existingVnames.indexOf(event.vname) === -1;
});
filteredEvents.forEach(function(event){
var venue = new Venue();
venue.name = event.vname;
venue.location = event.location;
venue.phone = event.vphone;
venue.save(function (err){
// Optionally, do some logging here, perhaps.
if (err) return console.error('Something went wrong!');
else return console.log('Successfully created new venue %s', venue.name);
});
});
});