How to validate array length when using $push? - node.js

I'm trying to limit the amount of elements a user can add to an array field on one of my schemas. I'm currently adding the elements to the array using Schema.findOneAndUpdate(); with the $push operator.
The first thing I tried was the solution given by another answer here on StackOverflow, namely: https://stackoverflow.com/a/29418656/6502807
This solution adds a validate function to the fields in the schema definition. By setting runValidators to true, I did get the function to run with Schema.findOneAndUpdate(). It was at that moment, however, that I stumbled upon the next problem. At the end of the Validation chapter in the Mongoose docs it says:
Also, $push, $addToSet, $pull, and $pullAll validation does not run any validation on the array itself, only individual elements of the array.
So attempting to check for array length did not work when using $pull. It simply supplied the validation function with an empty array every time, regardless of its actual contents in the database.
Next thing I tried was to use a pre hook. This was without any success as well. For some reason it did not execute the hook, even with runValidators set to true. This is how I defined said hook:
Settings.pre('update', async function (next) {
if (this.messages.length > MAX_MESSAGES) {
throw new Error('Too many messages');
} else {
next();
}
});
EDIT: The reason the function did not fire was because I was using findOneAndUpdate instead of update this is fixed and the function now runs. The solution code above, however, does not work.
The schema with the array looks like this:
const Settings = new mongoose.Schema({
// A lot more fields not relevant to this question
messages: {
type: [{
type: String
}]
}
});
Another thing worth mentioning is that these update statements are used in conjunction with other options. I need the update statement to behave like an update or insert so my complete set of options looks like this:
{
runValidators: true,
setDefaultsOnInsert: true,
upsert: true,
new: true
}
When executing queries with the pre hook set like this, the array limit can be exceeded without any validation error being thrown.
At this point I'm wondering if there is any sensible way to do a max length check like this without having to do it myself outside of mongoose's abstraction layer.
I am using Mongoose 5.2.6 running on node v9.11.1 with MongoDB 4.0.0.
Any help is much appreciated!

Well if you are using latest version from mongodb and mongoose then you can use $expr operator
const udpate = await db.collection.update(
{ $expr: { $gt: [{"$size": "$messages" }, MAX_MESSAGES] }},
{ update }
)

You should be able to do that with the pre update hook. The thing is that that hook would not by default give you the update being mage so you can verify etc. You have to take it via this.getUpdate():
Settings.pre('update', async function (next) {
var preUpdate = this.getUpdate()
// now inside of the preUpdate you would have your update being made and should have the array in there on which you can check the length
});
To give you an idea in my test schema I had to do something like this on an update with a $set:
this.getUpdate().$set.books.length // gave me 2 which was correct etc
I also had no issues running and hitting the update hook at all. It looks super simple out of the mongoose docs:
AuthorSchema.pre('update', function(next) {
console.log('UPDATE hook fired!')
console.log(this.getUpdate())
next();
});

Related

Mongoose, Nodejs - replace many documents in one I/O?

I have an array of objects and I want to store them in a collection using only one I/O operation if it's possible. If any document already exists in the collection I want to replace it, or insert it otherwise.
These are the solutions that I found, but doesn't work exactly as I want:
insertMany(): this doesn't replace the document that already exists, but throws exception instead (This is what I found in the Mongodb documentation, but I don't know if it's the same as mongoose).
update() or ‎updateMany() with upsert = true: this doesn't help me as well, because here I have to do the same updates to all the to stored documents.
‎There is no replaceMany() in mongodb or mongoose.
Is there anyone how knows any optimal way to do replaceMany using mongoose and node.js
There is bulkWrite (https://docs.mongodb.com/manual/reference/method/db.collection.bulkWrite/), which makes it possible to execute multiple operations at once. In your case, you can use it to perform multiple replaceOne operations with upsert. The code below shows how you can do it with Mongoose:
// Assuming *data* is an array of documents that you want to insert (or replace)
const bulkData = data.map(item => (
{
replaceOne: {
upsert: true,
filter: {
// Filter specification. You must provide a field that
// identifies *item*
},
replacement: item
}
}
));
db.bulkWrite(bulkData);
You need to query like this:
db.getCollection('hotspot').update({
/Your Condition/
}, {
$set: {
"New Key": "Value"
}
}, {
multi: true,
upsert: true
});
It fulfils your requirements..!!!

Mongoose's findByIdAndUpdate not changing database's state

I have a strange problem. I want to update a document in my MongoDB with the mongoose.findByIdAndUpdate method, but it seems not to be working. The code is:
Device.findByIdAndUpdate(
req.params.id,
{ $set: { power: power } },
{ new: true },
(err, device) => { ... }
I get no error, but the device returned in the callback does not have the updated value. At first I thought maybe it was some sort of problem with the { new: true } option that tells mongoose to return the updated document, but then I checked the database, and the value there also has not been updated.
I also tried replacing findByIdAndUpdate with update function, but the results are the same - the db is not getting updated.
If it changes anything I use mongoose.update() function in other places and it works fine. I also tried the 'classical' way of updating the value here - meaning I used findOne function and then changed returned document's power field value and saved it and it also worked fine.
I will be really gratefull for any advice on fixing this. Thank you!

Run custom validation in mongoose update query

I have been trying to run a custom validator to check if the name entered by the user already exists in the database. Since, mongoDb treats uppercase and lowercase names as different, I created my own validator for it.
function uniqueFieldInsensitive ( modelName, field ){
return function(val, cb){
if( val && val.length ){ // if string not empty/null
var query = mongoose.models[modelName]
.where( field, new RegExp('^'+val+'$', 'i') ); // lookup the collection for somthing that looks like this field
if( !this.isNew ){ // if update, make sure we are not colliding with itself
query = query.where('_id').ne(this._id)
}
query.count(function(err,n){
// false when validation fails
cb( n < 1 )
})
} else { // raise error of unique if empty // may be confusing, but is rightful
cb( false )
}
}
}
Now, the problem is that the validator runs while saving the document in the DB but not while update.
Since, I am using mongoose version 4.x, I also tried using { runValidators: true } in my update query. That doesn't work either as the 'this' keyword in my validator is 'null' while in the case of update whereas it refers to the updated doc in the case of save.
Could you please let me know if there is something i missed or is there any other way by which I can run custom validators in update query.
Finally I found a way out to do this.
According to MongoDB documentation, it says:
First, update validators only check $set and $unset operations. Update validators will not check $push or $inc operations.
The second and most important difference lies in the fact that, in document validators, this refers to the document being updated. In the case of update validators, there is no underlying document, so this will be null in your custom validators.
Refer to : Validators for update()
So, now we are only left with calling save() instead of update() in our queries. Since, save() calls all the custom and inbuilt validators, our validator will also be called. I achieved it like this:
function(req, res, next) {
_.assign(req.libraryStep, req.body);
req.libraryStep.save().then(function(data){
res.json(data);
}).then(null, function (err) {
console.info(err);
var newErr = new errorHandler.error.ProcessingError(errorHandler.getErrorMessage(err));
next(newErr);
});
};
Notice here req.libraryStep is the document that i queried from the database. I have used lodash method assign which takes the updated json and assigns it to the existing database document.
https://lodash.com/docs#assign
I dont think this is the ideal way but as for now till Mongoose doesnt come up with supporting custom validators, we can use this to solve our problem.
This is a fairly old thread, but I wanted to update the answer for those who come across it like I did.
While you're correct about the context of this being empty in an update validator (per the docs), there is a context option you can use to set the context of this. See the docs
However, a plugin also exists that will check the uniqueness of the field you are setting: mongoose-unique-validator. I use this for checking for duplicate emails. This also has an option for case insensitivity, so I would check it out. It also does run correctly using the update command with the runValidators: true option.

Mongodb, incrementing value inside an array. (save() ? update() ?)

var Poll = mongoose.model('Poll', {
title: String,
votes: {
type: Array,
'default' : []
}
});
I have the above schema for my simple poll, and I am uncertain of the best method to change the value of the elements in my votes array.
app.put('/api/polls/:poll_id', function(req, res){
Poll.findById(req.params.poll_id, function(err, poll){
// I see the official website of mongodb use something like
// db.collection.update()
// but that doesn't apply here right? I have direct access to the "poll" object here.
Can I do something like
poll.votes[1] = poll.votes[1] + 1;
poll.save() ?
Helps much appreciated.
});
});
You can to the code as you have above, but of course this involves "retrieving" the document from the server, then making the modification and saving it back.
If you have a lot of concurrent operations doing this, then your results are not going to be consistent, as there is a high potential for "overwriting" the work of another operation that is trying to modify the same content. So your increments can go out of "sync" here.
A better approach is to use the standard .update() type of operations. These will make a single request to the server and modify the document. Even returning the modified document as would be the case with .findByIdAndUpdate():
Poll.findByIdAndUpdate(req.params.poll_id,
{ "$inc": { "votes.1": 1 } },
function(err,doc) {
}
);
So the $inc update operator does the work of modifying the array at the specified position using "dot notation". The operation is atomic, so no other operation can modify at the same time and if there was something issued just before then the result would be correctly incremented by that operation and then also by this one, returning the correct data in the result document.

Override document on update

I am doing a findOneAndUpdate in mongoose:
Item.findOneAndUpdate({_id: 12345}, updateDoc, function (err, updatedItem) {
//....
});
However I want to completely overwrite the document. According to mongoose docs:
All top level keys which are not atomic operation names are treated as
set operations:
Is there anyway that I can override that behavior such that mongoose does not issue a $set operation for top level elements and instead overwrite the document?
An "overwrite" option has recently been added. It replaces the entire document, the way Mongo updates by default. It's used like this:
Item.findOneAndUpdate({_id: 12345}, updateDoc, {overwrite: true}, function(err, updatedItem) {
....
});
I found some history on this feature in thier GitHub Issues area.

Resources