MongooseJS messes up pre-imported data when document is updated - node.js

A piece of my Mongo document structure is:
{ "_id": ObjectId("xxxxxx..."),
"Country" : "UNITED KINGDOM",
"Line" : "something",
"Records" : [
{"rdata" : "foo", "rtype" : "X", "name" : "John"},
{"rdata" : "bar", "rtype" : "Y", "name" : "Bill"}
], ...
I'm using Mongoose to access the data via the following model:
var Record = new Schema({
rdata: String,
rtype: String,
name: String
}, {_id: false});
var ThingSchema = new Schema({
Country: String,
Line : String,
Records : [Record],
Let's say I want to update the "Line" property of one of my documents, from being "Line" : "something" to "Line" : "way more interesting" by sending a PUT request to the appropriate API URL. I can see that the data being sent is all right. This is what the API does:
exports.update = function(req, res) {
if(req.body._id) { delete req.body._id; }
Thing.findById(req.params.id, function (err, thing) {
if (err) { return handleError(res, err); }
if(!thing) { return res.send(404); }
var updated = _.merge(thing, req.body);
updated.save(function (err) {
if (err) { return handleError(res, err); }
return res.json(200, updated);
});
});
};
The API comes back with 200/OK - but I see the following updated data:
{ "_id": ObjectId("xxxxxx..."),
"Country" : "UNITED KINGDOM",
"Line" : "way more interesting", <-- updated correctly
"Records" : [
{"rdata" : "foo", "rtype" : "X", "name" : "John"},
{"rdata" : "foo", "rtype" : "X", "name" : "John"}
], ...
Notice, how the Records array got messed up by overwriting my second record by duplicating the first one. (If I switch on the automatic addition of '_id' to the subdocument by Mongoose, then even the "_id" fields will be the same on the two records within the array).
It may be relevant, that originally the Records were not added via Mongoose - but by importing a JSON document. Any suggestion as to how to start finding out why this is happening would be fantastic.

Try changing _.merge to _.extend, then call save directly on the thing document returned by the findById() method instead of the merged object updated:
exports.update = function(req, res) {
if(req.body._id) { delete req.body._id; }
Thing.findById(req.params.id, function (err, thing) {
if (err) { return handleError(res, err); }
if(!thing) { return res.send(404); }
_.extend(thing, req.body);
thing.save(function (err) {
if (err) { return handleError(res, err); }
return res.json(200, thing);
});
});
}
Another option is using the set method on the entity i.e. thing.set(req.body) before calling the save method on the thing object.
This answer by ShitalShah highlights the differences between merge and extend that is causing duplicates in your resulting object with merge but essentially:
Here's how extend/assign works: For each property in source, copy its
value as-is to destination. if property values themselves are objects,
there is no recursive traversal of their properties. Entire object
would be taken from source and set in to destination.
Here's how merge works: For each property in source, check if that
property is object itself. If it is then go down recursively and try
to map child object properties from source to destination. So
essentially we merge object hierarchy from source to destination.
While for extend/assign, it's simple one level copy of properties from
source to destination.
JSBin to illustrate the differences

Related

Updating nested array mongodb

I think there are multiple ways to do this, and that has me a little confused as to why I can't get it to work.
I have a schema and I would like to update Notes within it but I can't seem to do it. Additionally, if I want to return the notes how would I go about doing it?
schema :
{
_id : 1234
email : me#me.com
pass : password
stock : [
{
Ticker : TSLA
Market : Nasdaq
Notes : [
"Buy at 700",
"Sell at 1000"
]
},
{
Ticker : AAPL
Market : Nasdaq
Notes : [
"Buy at 110",
"Sell at 140"
]
},
]
}
Each user has a list of stocks, and each stock has a list of notes.
Here is what I have tried in order to add a note to the list.
router.post(`/notes/add/:email/:pass/:stock/:note`, (req, res) => {
var email = req.params.email
var pass = req.params.pass
var note = req.params.note
var tempStock = req.params.stock
userModel.findOne({email: email} , (err, documents)=>{
if (err){
res.send(err);
}
else if (documents === null){
res.send('user not found');
}else if (bcrypt.compareSync(pass , documents.pass)){
userModel.findOneAndUpdate({email : email , "stock.Ticker" : tempStock}, {$push : {Notes : note}} ,(documents , err)=>{
if(err){
res.send(err);
}else {
res.send(documents.stock);
}
})
}
})
})
Thanks :)
Currently, you are pushing the new note into a newly created Notes property inside the model instead of into the Notes of the concrete stock. I am not completely aware of the mongoose semantics but you need something like this:
userModel.findOneAndUpdate({ email: email, "stock.Ticker": tempStock }, { $push: { "stock.$.Notes": note } }, (documents, err) => {
$ gives you a reference to the currently matched element from the stock array.
For the second part, I am not sure what you mean by
Additionally, if I want to return the notes how would I go about doing it?
They should be returned by default if you're not doing any projection excluding them.
Also, as per the docs(and general practise), the callback for the findOneAndUpdate has a signature of
(error, doc) => { }
instead of
(documents, err) => { }
so you should handle that.

How to delete an object from an array in mongodb?

MongoDB collection/doc :
{
_id:something,
name:something,
todos: [
{key:1234},
{key:5678}
]
}
I want to delete the object with key:5678 using mongoose query. I did something like this but It's not deleting the object at all and returning the User with unchanged todos array.
Node Route:
router.post('/:action', async (req, res) => {
try {
if (req.params.action == "delete") {
const pullTodo = { $pull: { todos: { key: 5678 } } }
const todo = await User.findOneAndUpdate({ _id:req.body.id} },pullTodo)
if (todo) {
res.json({ msg: "Todo Deleted", data: todo });
}
}
} catch (err) {
console.log(err)
}
})
I have allso tried findByIdAndUpdate(),update() methods but none of them deleting the object from the array. Getting User as a result without deleting the object from the array.
It is working, but you forgot give an configuration to the function call of Model.findByIdAndUpdate..
const todo = await User.findOneAndUpdate({ _id:req.body.id} },pullTodo, {new: true});
// if {new: true} is enabled, then it will give the latest & updated document from the
// result of the query. By default it gives the previous document.
Do some, research first. This isn't a type of question that should be asked. It's already been answered several times in stackoverflow.
Try using Model.findOneAndRemove() instead. It also makes only one call to the database.
Example: User.findOneAndRemove({'todos':{'$elemMatch':{key}});
can you please re-visit your JSON like below and see if this design works for you.
> db.test6.find()
{ "_id" : "mallik", "name" : "mallik-name", "todos1" : { "key1" : [ 1234, 5678 ] } }
> db.test6.update({},{$pull:{"todos1.key1":5678}},{multi:true});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.test6.find()
{ "_id" : "mallik", "name" : "mallik-name", "todos1" : { "key1" : [ 1234 ] } }
>
I was adding a key property to every "Todo" using "mongoose.Types.ObjectId()" and I was querying with id string like : "5f439....." which was the problem. So I used:
1st Step: MongoId = require('mongodb').ObjectID;
2nd Step: const key = MongoId (**<actual id here>**);

MongoDB Upserts Variable name as a key (Node.JS)

Node.JS, MONGODB, not using Mongoose.
I have a document I'm saving. When I use UPSERT, it structures the data as so :
{ "_id" : ObjectId("573a123f55e195b227e7a78d"),
"document" :
[ {"name" : "SomeName" } ]
}
When I use INSERT it inserts it all on the root level :
{ "_id" : ObjectId("573a123f55e195b227e7a78d"), "name" : "SomeName" }
This is obviously going to lead to lots of inconsistencies. I have tried various things. I've tried various methods such as findOneAndReplace, Update with Upsert, I've tried Replace, $setOnInsert. It all does the same thing when Upsert is involved it seems.
I have tried to use document[0] to access the first block of the array, but that throws an error... 'Unexpected Token ['
I have tried various methods and dug for hours through the various documentation, and have searched high and low for someone else having this problem, but it doesn't seem to be well documented issue for anyone else.
Anyone have any recommendations to make sure that all the fields are on the ROOT level, not nested under the variable name? Relevant code below.
findReplace: function(db, document, collection) {
var dbCollection = db.collection(collection);
var filter = document.name;
return new Promise(function(resolve, reject) {
dbCollection.updateOne({
"name" : filter
}, {
document
}, {
upsert: true
}, function(err, result) {
if (err) {
console.log('error', err)
reject(err);
}
console.log("Found Document and Upserted replacement Document");
resolve([{'status' : 'success'}]);
});
});
}
When you do this:
{
document
}
You are creating an object containing a document property and the variable's value as its value:
{
document: [ {"name" : "SomeName" } ]
}
This is new functionality from ES6. If you want to access the first item of the document variable, don't create a new object:
return new Promise(function(resolve, reject) {
dbCollection.updateOne({
"name" : filter
}, document[0], { // <========
upsert: true
}, function(err, result) {
if (err) {
console.log('error', err)
reject(err);
}
console.log("Found Document and Upserted replacement Document");
resolve([{'status' : 'success'}]);
});
});

MongoDb Node.js Driver - findOneAndUpdate() behavior without a field given

I noticed a strange behavior with the mongodb node.js driver findOneAndUpate()...
I mistakenly gave it just a objectId string....thinking it would default to searching by _id field of a document.... so, when I used
User.prototype.updatePetArray = function(user, petElement) {
return this.collection.findOneAndUpdate(user,
{ $push: { pets: petElement } },
{ returnOriginal: false,
maxTimeMS: QUERY_TIME});
}
it pulled up and modified this document, which does not have this number at all:
{ "_id" : ObjectId("56d4e2a381c9c28b3056f792"), "username" : "bob123", "location" : "AT", ...}
Why did it modify this document when 56d4d35881c9c28b3056f78a is not in the document?
After I test it following your code with one non-exist ObjectID,
var col = db.collection('users');
col.findOneAndUpdate('56cd129222222', {fname: 'Daved'}, function(err, r) {
if (err)
console.log(err);
else
console.log(r);
db.close();
});
As a result the first document in the collection was changed .
Per this doc
findOneAndUpdate(filter, update, options, callback)
filter: Specify an empty document { } to update the first document returned in the collection.
So I think this non-exist ObjectId is consider to the same behavior with {}
After reviewing the source code of findAndModify, eventually, the
// Execute the command
self.s.db.command(queryObject
is invoked, and the queryObject is
var queryObject = {
'findandmodify': self.s.name
, 'query': query
};
So I test runCommand in mongo shell with non-exist ObjectId as below, as result, the first document is returned.
> db.users.runCommand({findandmodify: 'users', query: '123ddae344', update: true
})
{
"lastErrorObject" : {
"updatedExisting" : true,
"n" : 1
},
"value" : {
"_id" : ObjectId("56c687275d81735019263d1f"),
"fname" : "Daved"
},
"ok" : 1
}
The docs keep saying the filter parameter should be an object.
The wrong behavior is likely to some side effect of mis-interpreting the value, being a string not an object (and maybe a truthy value, non-empty string).

MongoDB node native driver creating duplicate documents

I'm getting a duplicate document when using the mongodb-native-driver to save an update to a document. My first call to save() correctly creates the document and adds a _id with an ObjectID value. A second call creates a new document with a text _id of the original ObjectID. For example I end up with:
> db.people.find()
{ "firstname" : "Fred", "lastname" : "Flintstone", "_id" : ObjectId("52e55737ae49620000fd894e") }
{ "firstname" : "Fred", "lastname" : "Flintstone with a change", "_id" : "52e55737ae49620000fd894e" }
My first call correctly created Fred Flinstone. A second call that added " with a change" to the lastname, created a second document.
I'm using MongoDB 2.4.8 and mongo-native-driver 1.3.23.
Here is my NodeJS/Express endpoint:
app.post("/contacts", function (req, res) {
console.log("POST /contacts, req.body: " + JSON.stringify(req.body));
db.collection("people").save(req.body, function (err, inserted) {
if (err) {
throw err;
} else {
console.dir("Successfully inserted/updated: " + JSON.stringify(inserted));
res.send(inserted);
}
});
});
Here is the runtime log messages:
POST /contacts, req.body: {"firstname":"Fred","lastname":"Flintstone"}
'Successfully inserted/updated: {"firstname":"Fred","lastname":"Flintstone","_id":"52e55737ae49620000fd894e"}'
POST /contacts, req.body: {"firstname":"Fred","lastname":"Flintstone with a change","_id":"52e55737ae49620000fd894e"}
'Successfully inserted/updated: 1'
Why doesn't my second update the existing record? Does the driver not cast the _id value to an ObjectID?
What you are posting back the 2nd time contains a field named "_id", and it's a string. That is the problem.
Look at the document, what the save method does is a "Simple full document replacement function". I don't use this function quit often so here's what I guess. The function use the _id field to find the document and then replace the full document with what you provided. However, what you provided is a string _id. Apparently it doesn't equal to the ObjectId. I think you should wrap it to an ObjectId before passing to the function.
Besides, the save method is not recommended according to the document. you should use update (maybe with upsert option) instead
I don't exactly know why a second document is created, but why don't you use the update function (maybe with the upsert operator)?
An example for the update operation:
var query = { '_id': '52e55737ae49620000fd894e' };
db.collection('people').findOne(query, function (err, doc) {
if (err) throw err;
if (!doc) {
return db.close();
}
doc['lastname'] = 'Flintstone with a change';
db.collection('people').update(query, doc, function (err, updated) {
if (err) throw err;
console.dir('Successfully updated ' + updated + ' document!');
return db.close();
});
});
And now with the upsert operator:
var query = { '_id': '52e55737ae49620000fd894e' };
var operator = { '$set': { 'lastname': 'Flintstone with a change' } };
var options = { 'upsert': true };
db.collection('people').update(query, operator, options, function (err, upserted) {
if (err) throw err;
console.dir('Successfully upserted ' + upserted + ' document!');
return db.close();
});
The difference is that the upsert operator will update the document if it exist, otherwise it will create a new one. When using the upsert operator you should keep in mind that this operation can be underspecified. That means if your query does not contain enough information to identify a single document, a new document will be inserted.

Resources