I have a User schema with a activated field of Boolean type. I want queries to only return documents which have activated: true. And I hope there is a more efficient and DRY way of doing so than adding a conditional to every find, findOne, or findById.
What would be the most effective approach?
while there may be some way to do this, it is generally a bad idea to always hide this information.
speaking from experience trying to do this with other languages and database systems, you will, at some point, want / need to load items that are not actived. but if you always and only return activated items, you'll never be able to get the list you need.
for your purposes, i would recommend creating a findActive method on your schema:
someSchema.static("findActive", function(query, cb){
// check if there is a query and callback
if (!cb){
cb = query;
query = {};
}
// set up an empty query, if there isn't one provided
if (!query) { query = {}; }
// make sure you only load activated items
query.activated = true;
// run the query
this.find(query, cb);
});
with this method, you will have a findActive method the same as findOne, but it will always filter for activated items.
MyModel.findActive(function(err, modelList){ ... });
and it optionally supports additional query filters
MyModel.findActive({some: "stuff"}, function(err, modelList){ ... });
You might want to look at Mongoose Query middleware here
Query middleware is supported for the following Model and Query
functions.
count
find
findOne
...
For example:
User.pre('find', function() {
console.log(this instanceof mongoose.Query); // true
this.activated = true;
});
Related
I'm trying to limit the fields a user can post when inserting an object in mongodb. I know ho i can enforce fields to be filled but I can't seem to find how to people from inserting fields that I don't want.
This is the code I have now for inserting an item.
app.post("/obj", function (req, res) {
var newObj = req.body;
//TODO filter fields I don't want ?
if (!(newObj .id || newObj .type)) {
handleError(res, "Invalid input", "Must provide a id and type.", 400);
return;
}
db.collection(OBJ_COLLECTION).insertOne(newObj, function(err, doc) {
if (err) {
handleError(res, err.message, "Failed to create new object.");
} else {
res.status(201).json(doc.ops[0]);
}
});
});
There's likely JS native ways to do this, but I tend to use Lodash as my toolbox for most projects, and in that case what I normally do is setup a whitelist of allowed fields, and then extract only those from the posted values like so:
const _ = require('lodash');
app.post("/obj", function (req, res) {
var newObj = _.pick(req.body, ['id', 'type','allowedField1','allowedField2']);
This is pretty straightforward, and I usually also define the whitelist somewhere else for reuse (e.g. on the model or the like).
As a side note, I avoid using 'id' as a field that someone can post to for new objects, unless I really need to, to avoid confusion with the autogenerated _id field.
Also, you should really look into mongoose rather than using the straight mongodb driver, if you want to have more model-based control of your documents. Among other things, it will strip any fields off the object if they're not defined in the schema. I still use the _.pick() method when there are things that are defined in the schema, but I don't want people to change in a particular controller method.
When using Mongoose (with bluebird in my case, but using callbacks to illustrate), the following codes all return a document from the collection:
model.findOne({}, function(err, document) {
//returns a document
})
model.findOne(null, function(err, document) {
//returns a document
})
model.findOne([], function(err, document) {
//returns a document
})
I would like to know if and how I can disable this kind of behaviour, as it is becoming a liability to my code where I infer queries from data a user feeds into the system. Especially the null query returning a valid document worries me.
As of right now I check the input for being an non-empty, non-array, non-null object, but it's becoming a bit cumbersome at scale.
What would be the best way to exclude this behaviour?
Not sure if it is the best way to go about it, but right now I've settled on using a pre-hook on the model itself which checks for the _conditions property of the 'this' object (which I inferred from printing seems to hold the query object) to not be empty.
Inserting a self-defined object in the next functionality causes the Promise to reject in which the query was originally called from.
( _ is the underscore package)
//model.js
//model is a mongoose.Schema type in the following code
model.pre('findOne', function(next) {
var self = this
if (_.isEmpty(self._conditions)) {
next(mainErrors.malformedRequest)
} else {
next()
}
})
I have been trying to run a custom validator to check if the name entered by the user already exists in the database. Since, mongoDb treats uppercase and lowercase names as different, I created my own validator for it.
function uniqueFieldInsensitive ( modelName, field ){
return function(val, cb){
if( val && val.length ){ // if string not empty/null
var query = mongoose.models[modelName]
.where( field, new RegExp('^'+val+'$', 'i') ); // lookup the collection for somthing that looks like this field
if( !this.isNew ){ // if update, make sure we are not colliding with itself
query = query.where('_id').ne(this._id)
}
query.count(function(err,n){
// false when validation fails
cb( n < 1 )
})
} else { // raise error of unique if empty // may be confusing, but is rightful
cb( false )
}
}
}
Now, the problem is that the validator runs while saving the document in the DB but not while update.
Since, I am using mongoose version 4.x, I also tried using { runValidators: true } in my update query. That doesn't work either as the 'this' keyword in my validator is 'null' while in the case of update whereas it refers to the updated doc in the case of save.
Could you please let me know if there is something i missed or is there any other way by which I can run custom validators in update query.
Finally I found a way out to do this.
According to MongoDB documentation, it says:
First, update validators only check $set and $unset operations. Update validators will not check $push or $inc operations.
The second and most important difference lies in the fact that, in document validators, this refers to the document being updated. In the case of update validators, there is no underlying document, so this will be null in your custom validators.
Refer to : Validators for update()
So, now we are only left with calling save() instead of update() in our queries. Since, save() calls all the custom and inbuilt validators, our validator will also be called. I achieved it like this:
function(req, res, next) {
_.assign(req.libraryStep, req.body);
req.libraryStep.save().then(function(data){
res.json(data);
}).then(null, function (err) {
console.info(err);
var newErr = new errorHandler.error.ProcessingError(errorHandler.getErrorMessage(err));
next(newErr);
});
};
Notice here req.libraryStep is the document that i queried from the database. I have used lodash method assign which takes the updated json and assigns it to the existing database document.
https://lodash.com/docs#assign
I dont think this is the ideal way but as for now till Mongoose doesnt come up with supporting custom validators, we can use this to solve our problem.
This is a fairly old thread, but I wanted to update the answer for those who come across it like I did.
While you're correct about the context of this being empty in an update validator (per the docs), there is a context option you can use to set the context of this. See the docs
However, a plugin also exists that will check the uniqueness of the field you are setting: mongoose-unique-validator. I use this for checking for duplicate emails. This also has an option for case insensitivity, so I would check it out. It also does run correctly using the update command with the runValidators: true option.
I'm working with some old data where some of the schema has a "mixed" type.
Basically sometimes a value will be a referenced ObjectID, but other times it'll be some text (super poor design).
I unable to correctly populate this data because of the times a non-ObjectID appears.
So, for my actual question: Is it possible to create a populate (on a collection) that is conditional; I need to be able to tell the populate to skip those other values.
Yes, you can do that check the middleware function on the Mongoose API reference
http://mongoosejs.com/docs/middleware.html
What you need to do is before you populate those data, you validate the data if is is Object ID or not, if it is Object ID, you call next() to pass the next function, else you just return, this will skip it
Example
xSchema.pre('validate', function(next){
var x = this;
var checkXType = typeof x.id;
if (checkXType === String) {
return;
} else {
next();
}
});
NodeJS + Express, MongoDB + Mongoose
I have a JSON feed where each record has a set of "venue" attributes (things like "venue name" "venue location" "venue phone" etc). I want to create a collection of all venues in the feed -- one instance of each venue, no dupes.
I loop through the JSON and test whether the venue exists in my venue collection. If it doesn't, save it.
jsonObj.events.forEach(function(element, index, array){
Venue.findOne({'name': element.vname}, function(err,doc){
if(doc == null){
var instance = new Venue();
instance.name = element.vname;
instance.location = element.location;
instance.phone = element.vphone;
instance.save();
}
}
}
Desired: A list of all venues (no dupes).
Result: Plenty of dupes in the venue collection.
Basically, the loop created a new Venue record for every record in the JSON feed.
I'm learning Node and its async qualities, so I believe the for loop finishes before even the first save() function finishes -- so the if statement is always checking against an empty collection. Console.logging backs this claim up.
I'm not sure how to rework this so that it performs the desired task. I've tried caolan's async module but I can't get it to help. There's a good chance I'm using incorrectly.
Thanks so much for pointing me in the right direction -- I've searched to no avail. If the async module is the right answer, I'd love your help with how to implement it in this specific case.
Thanks again!
Why not go the other way with it? You didn't say what your persistence layer is, but it looks like mongoose or possibly FastLegS. In either case, you can create a Unique Index on your Name field. Then, you can just try to save anything, and handle the error if it's a unique index violation.
Whatever you do, you must do as #Paul suggests and make a unique index in the database. That's the only way to ensure uniqueness.
But the main problem with your code is that in the instance.save() call, you need a callback that triggers the next iteration, otherwise the database will not have had time to save the new record. It's a race condition. You can solve that problem with caolan's forEachSeries function.
Alternatively, you could get an array of records already in the Venue collection that match an item in your JSON object, then filter the matches out of the object, then iteratively add each item left in the filtered JSON object. This will minimize the number of database operations by not trying to create duplicates in the first place.
Venue.find({'name': { $in: jsonObj.events.map(function(event){ return event.vname; }) }}, function (err, docs){
var existingVnames = docs.map(function(doc){ return doc.name; });
var filteredEvents = jsonObj.events.filter(function(event){
return existingVnames.indexOf(event.vname) === -1;
});
filteredEvents.forEach(function(event){
var venue = new Venue();
venue.name = event.vname;
venue.location = event.location;
venue.phone = event.vphone;
venue.save(function (err){
// Optionally, do some logging here, perhaps.
if (err) return console.error('Something went wrong!');
else return console.log('Successfully created new venue %s', venue.name);
});
});
});