Mongoose: Using addToSet with ObjectIds Results in Orphan Id - node.js

I am having a rather interesting problem using mongoDB's $addToSet to an array full of ObjectIds.
In my mongoose schema ("Happening"), I declare an array of ObjecIds called "expected", to be used by .populate().
expected: [{type: Schema.Types.ObjectId, ref: "User" }]
... which works nicely everywhere I use it. So far so good.
I then attempt to update the Happening.expected array using $addToSet as outlined here:
http://docs.mongodb.org/manual/reference/operator/addToSet/
like so:
app.get("/happening/yamobethere/:id", ensureLoggedIn("/login"),
function (req, res) {
// userId is the mongo ObjectId of the user record
var userId = req.session.user.id,
eventId = req.params.id;
models.Happening.update(
{_id: eventId}, {
$addToSet: {expected: userId}
},
function(err, updated){
if (err) {
res.json({"error": err});
}
res.json({"updated": updated});
});
});
... which always yields:
{updated: 1}
Now the docs lead me to expect the actual userId that I passed in, so the "1" is a bit odd. I expected it to be a fail, and in light of the weirdness that happens next, it appears to be a mongodb error of some sort percolating it's way back to me as results.
The weirdness is, when I check my database, I see that indeed a new ObjectId has been added: just not the one I passed in.
"expected" : [
ObjectId("51cb18623ade2b9f1e000004"),
ObjectId("51cdb7c12f0e58bdb3000001")
],
becomes
"expected" : [
ObjectId("51cb18623ade2b9f1e000004"),
ObjectId("51cdb7c12f0e58bdb3000001"),
ObjectId("51cdb80e09612bfab3000002")
],
The new ObjectId does not appear in any of my collections. It appears to be an orphan, but I'm a mongo noob, so I may be full of compost on this.
I did attempt to cast the userId as an ObjectId:
$addToSet: {expected: mongoose.Types.ObjectId.fromString(userId)}
but that changed nothing, and really should not be necessary, since the schema should handle it.
I'd really rather not resort to downloading the entire object, appending the value to the "expected" array, then sending the whole schmear back for an update.
Any help appreciated, folks. Thanks!
Update:
A colleague suggested the following technique:
var addMe = {$addToSet: {expected: userId}};
models.Happening.findByIdAndUpdate(eventId, addMe, function(err, me) {
if (err) {
return json(err);
}
res.json(200, me);
});
... which is a bit of an improvement, since it actually returns an object for me to inspect. Unfortunately, it also results in orphaned ObjecIds appearing in the array, rather than the existing userId value I specified.
Thanks again!

It appears that my passport strategy is returning the ObjectID of the rejected attempted creation of a new user in the db via data from oauth. So, the code is fine, my data is garbage.
Never trust anything, and be prepared to look like a boob. :-)
Thanks for the clarification on my return values JohnnyHK.

Related

Is collection.findOne({query}) not supposed to return the document itself?

I see a lot of tutorials/guides using collection.findOne({query}).field to get a value of a field in the document that is returned, but for me that doesn't seem to work and I wonder why. I did figure out another way to do it though. See below:
var rank = function(id) {
// My way of doing it
collection.findOne({ _id: id }, function(err, doc) {
console.log(doc.score); // Prints the actual score
});
// How some tutorials do it
var score = collection.findOne({ _id: id }).score;
console.log(score); // Gives a TypeError (Cannot read property 'score' of undefined)
};
// How some tutorials do it
Those tutorials are likely using mongodb's shell, not node.js api. The shell's api looks similar (all the same words, findOne, etc.), but it doesn't use callbacks. Shell's findOne does return the document inline.

findOneAndUpdate works part of the time. MEAN stack

I'm working with the mean stack I'm trying to update the following object:
{
_id : "the id",
fields to be updated....
}
This is the function that does the updating:
function updateById(_id, update, opts){
var deferred = Q.defer();
var validId = new RegExp("^[0-9a-fA-F]{24}$");
if(!validId.test(_id)){
deferred.reject({error: 'invalid id'});
} else {
collection.findOneAndUpdate({"_id": new ObjectID(_id)}, update, opts)
.then(function(result){
deferred.resolve(result);
},
function(err){
deferred.reject(err);
});
}
return deferred.promise;
}
This works with some of my objects, but doesn't work with others.
This is what is returned when it fails to update:
{
ok: 1,
value:null
}
When the function is successful in updating the object it returns this:
{
lastErrorObject: {}
ok: 1
value: {}
}
It seems like Mongo is unable to find the objects I'm trying to update when it fails. However, I can locate those objects within the Mongo shell using their _id.
Does anybody know why the driver would be behaving this way? Could my data have become corrupt?
Cheers!
I found the answer and now this question seems more ambiguous so I apologize if it was confusing.
The reason I was able to find some of the documents using ObjectID(_id) was because I had manually generated some _id fields using strings.
Now I feel like an idiot but, instead of deleting this question I decided to post the answer just in case someone is running into a similar issue. If you save an _id as a string querying the collection with the _id field changes.
querying collection with MongoDB generated _ids:
collection.findOneAndUpdate({"_id": new ObjectID(_id)}, update, opts)
querying collection with manually generated _ids:
collection.findOneAndUpdate({"_id": _id}, update, opts)
In the second example _id is a string.
Hope this helps someone!

Mongoose $push cannot push object into correct document

I have a mongoose schema like this:
A = {
_id: Schema.Types.ObjectId,
arrayA:[{
_id,
nestedArray: [Schema.Types.ObjectId]
}],
arrayB: [Schema.Types.ObjectId]
}
I would like to push an Object Id into nestedArray in specific arrayA object AND
arrayB should contains an specific Object Id by following code:
A.update({'arrayA._id': arrayAId, arrayB: {$in: [arrayContainsSomeArrayBIds]}},
{$push: {'arrayA.$.nestedArray': nestedArrayId}}, function(err) {
});
However, the Object Id is pushed into nestedArray of the last object in arrayA.
If arrayB: {$in: [arrayContainsSomeArrayBIds]} is removed, the Object Id can be pushed into correct object in arrayA.
mongoose version: 3.8.21
Can anyone help me to find out the problem?
Currently it is not possible in MongoDB to update an a array element with the positional operator, when the query document contains references to other arrays apart from the one being updated.
The below code, contains reference to two arrays fields: arrayA and arrayB, when
the update is issued on arrayA. This is invalid and would lead to undesired behavior.
A.update({'arrayA._id': arrayAId, arrayB: {$in: [arrayContainsSomeArrayBIds]}},
{$push: {'arrayA.$.nestedArray': nestedArrayId}}, function(err) {
});
From the docs,
Only one array field may appear in the query document.
The query document should only contain a single condition on the array field
being projected.
Multiple conditions may override each other
internally and lead to undefined behavior.
Under these requirements,
the following query is incorrect:
db.collection.find( { <array>: <value>, <someOtherArray>: <value2> },
{ "<array>.$": 1 } )
The solution is to modify your code to fire two queries:
Get the _ids of the documents, which match our condition.
Then perform the update.
Sample Code flow:
A.find({'arrayA._id': arrayAId, arrayB: {$in: [arrayContainsSomeArrayBIds]}},
function(err,data){
data.forEach(function(doc){
A.update({'arrayA._id': arrayAId,
"_id":doc._id},
{$push: {'arrayA.$.nestedArray': nestedArrayId}},
function(err) {
});
})
});

Saving subdocuments with mongoose

I have this:
exports.deleteSlide = function(data,callback){
customers.findOne(data.query,{'files.$':1},function(err,data2){
if(data2){
console.log(data2.files[0]);
data2.files[0].slides.splice((data.slide-1),1);
data2.files[0].markModified('slides');
data2.save(function(err,product,numberAffected){
if(numberAffected==1){
console.log("manifest saved");
var back={success:true};
console.log(product.files[0]);
callback(back);
return;
}
});
}
});
}
I get the "manifest saved" message and a callback with success being true.
When I do the console.log when I first find the data, and compare it with the console.log after I save the data, it looks like what I expect. I don't get any errors.
However, when I look at the database after running this code, it looks like nothing was ever changed. The element that I should have deleted, still appears?
What's wrong here?
EDIT:
For my query, I do {'name':'some string','files.name':'some string'}, and if the object is found, I get an array of files with one object in it.
I guess this is a subdoc.
I've looked around and it says the rules for saving subdocs are different than saving the entire collection, or rather, the subdocs are only applied when the root object is saved.
I've been going around this by grabbing the entire root object, then I do loops to find the actual subdoc I that I want, and after I manipulate that, I save the whole object.
Can I avoid doing this?
I'd probably just switch to using native drivers for this query as it is much simpler. (For that matter, I recently dropped mongoose on my primary project and am happy with the speed improvements.)
You can find documentation on getting access to the native collection elsewhere.
Following advice here:
https://stackoverflow.com/a/4588909/68567
customersNative.update(data.query, {$unset : {"slides.1" : 1 }}, function(err){
if(err) { return callback(err); }
customersNative.findAndModify(data.query, [],
{$pull: {'slides' : null } }, {safe: true, 'new' : true}, function(err, updated) {
//'updated' has new object
} );
});

Mongoose: CastError: Cast to ObjectId failed for value "[object Object]" at path "_id"

I am new to node.js, so I have a feeling that this will be something silly that I have overlooked, but I haven't been able to find an answer that fixes my problem. What I'm trying to do is create a path that will create a new child object, add it to the parent's array of children, then return the child object to the requester. The problem that I am running into is that if I pass the string id into findById, node crashes with
TypeError: Object {} has no method 'cast'
If I try to pass in an ObjectId instead, I get
CastError: Cast to ObjectId failed for value "[object Object]" at path "_id"
Here is a rough outline of my code:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var ObjectId = Schema.ObjectId; //Have also tried Schema.Types.ObjectId, mongoose.ObjectId
mongoose.connect('mongodb://user:password#server:port/database');
app.get('/myClass/:Id/childClass/create', function(request, result) {
var id = new ObjectId(request.params.Id);
MyClass.findById(id).exec( function(err, myClass) {
if (err || !myClass) { result.send("error: " + err + "<br>" + JSON.stringify(id) || ("object '" + request.params.Id + "' not found: " + id)); return; }
var child = ChildClass();
myClass.Children.addToSet(child);
myClass.save();
result.send(child);
});
});
If I execute this code with the path "/myClass/51c35e5ced18cb901d000001/childClass/create", this is the output of the code:
error: CastError: Cast to ObjectId failed for value "[object Object]" at path "_id"
{"path":"51c35e5ced18cb901d000001","instance":"ObjectID","validators":[],"setters":[],"getters":[],"_index":null}
I've tried using findOne and passing in {_id:id} instead, but this appears to be exactly what findById does. I've tried the different classes for ObjectId that I've seen listed on other sites. I've tried calling ObjectId() like a function instead of a constructor and that returns undefined. At this point, I'm running out of ideas and it doesn't seem that googling for an answer is helping. Any ideas on what I'm doing wrong?
Also, like I said, I'm new to node/Mongo/Mongoose/Express, so if there is a better way to accomplish my goal, please let me know. I appreciate all feedback.
EDIT:
After the workaround from Peter Lyons, I googled another error that I was running into and found findByIdAndUpdate, which works as expected and does exactly what I was hoping to do. I'm still not sure why findById and findOne were giving me such issues and I'm curious to know (maybe a bug report needs to be filed), so I'll leave this open in case someone else has an answer.
Short answer: use mongoose.Types.ObjectId.
Mongoose (but not mongo) can accept object Ids as strings and "cast" them properly for you, so just use:
MyClass.findById(req.params.id)
However, the caveat is if req.params.id is not a valid format for a mongo ID string, that will throw an exception which you must catch.
So the main confusing thing to understand is that mongoose.SchemaTypes has stuff you only use when defining mongoose schemas, and mongoose.Types has the stuff you use when creating data objects you want to store in the database or query objects. So mongoose.Types.ObjectId("51bb793aca2ab77a3200000d") works, will give you an object you can store in the database or use in queries, and will throw an exception if given an invalid ID string.
findOne takes a query object and passes a single model instance to the callback. And findById is literally a wrapper of findOne({_id: id}) (see source code here). Just find takes a query object and passes an array of matching model instances to the callback.
Just go slow. It's confusing but I can guarantee you you are getting confused and not hitting bugs in mongoose at this point. It's a pretty mature library, but it takes some time to get the hang of it.
The other suspect thing I see in your snippet is not using new when instantiating ChildClass. Beyond that, you'll need to post your schema code in order for us to help you tract down any CastErrors that remain.
I've faced this error, That was because the value you want to filter in the _id field is not in an ID format, one "if" should solve your error.
const mongoose = require('mongoose');
console.log(mongoose.Types.ObjectId.isValid('53cb6b9b4f4ddef1ad47f943'));
// true
console.log(mongoose.Types.ObjectId.isValid('whatever'));
// false
To solve it, always validate if the criteria value for search is a valid ObjectId
const criteria = {};
criteria.$or = [];
if(params.q) {
if(mongoose.Types.ObjectId.isValid(params.id)) {
criteria.$or.push({ _id: params.q })
}
criteria.$or.push({ name: { $regex: params.q, $options: 'i' }})
criteria.$or.push({ email: { $regex: params.q, $options: 'i' }})
criteria.$or.push({ password: { $regex: params.q, $options: 'i' }})
}
return UserModule.find(criteria).exec(() => {
// do stuff
})
For all those people stuck with this problem, but still couldn't solve it: I stumbled upon the same error and found the _id field being empty.
I described it here in more detail. Still have not found a solution except changing the fields in _id to not-ID fields which is a dirty hack to me. I'm probably going to file a bug report for mongoose. Any help would be appreciated!
Edit: I updated my thread. I filed a ticket and they confirmed the missing _id problem. It is going to be fixed in the 4.x.x version which has a release candidate available right now. The rc is not recommended for productive use!
If you are having this issue and you are performing a populate somewhere along the lines, see this Mongoose issue.
Update to Mongoose 4.0 and the issue has been fixed.
Had the same problem, I just coerced the id into a string.
My schema:
const product = new mongooseClient.Schema({
retailerID: { type: mongoose.SchemaTypes.ObjectId, required: true, index: true }
});
And then, when inserting:
retailerID: `${retailer._id}`
I had the same problem, turned out after I have updated my schema, I have forgotten I was calling the model using the old id, which was created by me; I have updated my schema from something like:
patientid: {
type: String,
required: true,
unique: true
},
to
patientid: { type: mongoose.SchemaTypes.ObjectId, ref: "Patient" },
It turned out, since my code is big, I was calling the findOne with the old id, therefore, the problem.
I am posting here just to help somebody else: please, check your code for unknown wrong calls! it may be the problem, and it can save your huge headacles!
My solution is that I want data from all docs, and I don't want _id, so
User.find({}, {_id:0, keyToShow:1, keyToNotShow:0})
I was receiving this error CastError: Cast to ObjectId failed for value “[object Object]” at path “_id” after creating a schema, then modifying it and couldn't track it down. I deleted all the documents in the collection and I could add 1 object but not a second. I ended up deleting the collection in Mongo and that worked as Mongoose recreated the collection.
For the record: I had this error trying to fill a subdocument in a wrong way:
{
[CastError: Cast to ObjectId failed for value "[object Object]" at path "_id"]
message: 'Cast to ObjectId failed for value "[object Object]" at path "_id"',
name: 'CastError',
type: 'ObjectId',
path: '_id'
value:
[ { timestamp: '2014-07-03T00:23:45-04:00',
date_start: '2014-07-03T00:23:45-04:00',
date_end: '2014-07-03T00:23:45-04:00',
operation: 'Deactivation' } ],
}
look ^ value is an array containing an object: wrong!
Explanation: I was sending data from php to a node.js API in this way:
$history = json_encode(
array(
array(
'timestamp' => date('c', time()),
'date_start' => date('c', time()),
'date_end' => date('c', time()),
'operation' => 'Deactivation'
)));
As you can see $history is an array containing an array. That's why mongoose try to fill _id (or any other field) with an array instead than a Scheme.ObjectId (or any other data type). The following works:
$history = json_encode(
array(
'timestamp' => date('c', time()),
'date_start' => date('c', time()),
'date_end' => date('c', time()),
'operation' => 'Deactivation'
));
I am not sure this will help but I resolved the issue by importing mongoose like below and implementing it as below
const mongoose = require('mongoose')
_id: new mongoose.Types.ObjectId(),
I also encountered this mongoose error
CastError: Cast to ObjectId failed for value \"583fe2c488cf652d4c6b45d1\" at path \"_id\" for model User
So I run npm list command to verify the mongodb and mongoose version in my local.
Heres the report:
......
......
├── mongodb#2.2.19
├── mongoose#4.7.2
.....
It seems there's an issue on this mongodb version so what I did is I uninstall and try to use different version such as 2.2.16
$ npm uninstall mongodb, it will delete the mongodb from your node_modules directory. After that install the lower version of mongodb.
$ npm install mongodb#2.2.16
Finally, I restart the app and the CastError is gone!!
I was having the same problem.Turns out my Node.js was outdated. After upgrading it's working.
just change the path it will work for example
app.get('/myClass/:Id/childClass/create', function(request, result) .....
change to
app.get('/myClass**es**/:Id/childClass/create', function(request, result) .....
I just added --es-- to the path (myClass) to become (myClasses)
now should work and will not see that error
For me, the ID was undefined (req.params.id returns undefined)
If you're finding a document by its "_id" for a model, say Drivers, the command below works quite well:
....
const driver = await Drivers.findById(<your id>);
....
Ensure you use the async-await and try-catch ES6 syntax
For me, I was using a put request without sending any data. I changed it to a post request and it worked.
This thread covers more about put requests.
just change the path it will work for example
app.get('/myClass/:Id/childClass/create', function(request, result)
change to
app.get('/myClass**es**/:Id/childClass/create', function(request, result)
I just added --es-- to the path (myClass) to become (myClasses)
now should work and will not see that error

Resources