Im developing an app that uses node-->express-->mongodb-->mongoose.
The app has a concept similar to an auction, where there are bids made and it is very likely that most users will try to make their bid at the last seconds of the auction.
Before users bid is accepted, there is a validation process ( dates, times, bid sizes, users available account balance etc) , and once that is done, the bid is accepted and auction document is saved.
The question is, is it possible that users will update the document between
findOne() call, and a save() call made on the resulting document.
In other words,
auction.findOne(_id: "something").exec(function(err, doc) {
if ( !err && doc ) {
//use the found document to do some validation that may or may not
//take some time.
//then document validates,
if ( document is valid ) {
doc.last_bid = "xxx";
//is it possible that someone makes an update to the document
//here that invalidates the document, and i will end up saving
//an invalid document
doc.save();
}
// else -> dont save anything and reject deferred promise.
//Is it possible that after ive done my validation and document validated successfully, that someone updates it, before i get to call save().
}
});
Is it possible to "lock" a document when making findOne/find query ,and reject any updates to it until im done with using the document ?
I think that you can't lock a mongoose document. I propose you a solution, I think that you have a classical problem of "exclussion", so, I will use a mutex to try to access critical section, in your case, "critical section" is udpate mongoose document "doc.last_bid = 'xxx'". Try "locks":
https://github.com/Wizcorp/locks
var rwlock = locks.createReadWriteLock();
rwlock.readLock(function () {
if(document is valid) {
doc.last_bid = 'xxx'
doc.set_is_not_valid_or_used = true; // invalidate document => if condition
}
rwlock.unlock();
});
Related
When using Mongoose (with bluebird in my case, but using callbacks to illustrate), the following codes all return a document from the collection:
model.findOne({}, function(err, document) {
//returns a document
})
model.findOne(null, function(err, document) {
//returns a document
})
model.findOne([], function(err, document) {
//returns a document
})
I would like to know if and how I can disable this kind of behaviour, as it is becoming a liability to my code where I infer queries from data a user feeds into the system. Especially the null query returning a valid document worries me.
As of right now I check the input for being an non-empty, non-array, non-null object, but it's becoming a bit cumbersome at scale.
What would be the best way to exclude this behaviour?
Not sure if it is the best way to go about it, but right now I've settled on using a pre-hook on the model itself which checks for the _conditions property of the 'this' object (which I inferred from printing seems to hold the query object) to not be empty.
Inserting a self-defined object in the next functionality causes the Promise to reject in which the query was originally called from.
( _ is the underscore package)
//model.js
//model is a mongoose.Schema type in the following code
model.pre('findOne', function(next) {
var self = this
if (_.isEmpty(self._conditions)) {
next(mainErrors.malformedRequest)
} else {
next()
}
})
I have been trying to run a custom validator to check if the name entered by the user already exists in the database. Since, mongoDb treats uppercase and lowercase names as different, I created my own validator for it.
function uniqueFieldInsensitive ( modelName, field ){
return function(val, cb){
if( val && val.length ){ // if string not empty/null
var query = mongoose.models[modelName]
.where( field, new RegExp('^'+val+'$', 'i') ); // lookup the collection for somthing that looks like this field
if( !this.isNew ){ // if update, make sure we are not colliding with itself
query = query.where('_id').ne(this._id)
}
query.count(function(err,n){
// false when validation fails
cb( n < 1 )
})
} else { // raise error of unique if empty // may be confusing, but is rightful
cb( false )
}
}
}
Now, the problem is that the validator runs while saving the document in the DB but not while update.
Since, I am using mongoose version 4.x, I also tried using { runValidators: true } in my update query. That doesn't work either as the 'this' keyword in my validator is 'null' while in the case of update whereas it refers to the updated doc in the case of save.
Could you please let me know if there is something i missed or is there any other way by which I can run custom validators in update query.
Finally I found a way out to do this.
According to MongoDB documentation, it says:
First, update validators only check $set and $unset operations. Update validators will not check $push or $inc operations.
The second and most important difference lies in the fact that, in document validators, this refers to the document being updated. In the case of update validators, there is no underlying document, so this will be null in your custom validators.
Refer to : Validators for update()
So, now we are only left with calling save() instead of update() in our queries. Since, save() calls all the custom and inbuilt validators, our validator will also be called. I achieved it like this:
function(req, res, next) {
_.assign(req.libraryStep, req.body);
req.libraryStep.save().then(function(data){
res.json(data);
}).then(null, function (err) {
console.info(err);
var newErr = new errorHandler.error.ProcessingError(errorHandler.getErrorMessage(err));
next(newErr);
});
};
Notice here req.libraryStep is the document that i queried from the database. I have used lodash method assign which takes the updated json and assigns it to the existing database document.
https://lodash.com/docs#assign
I dont think this is the ideal way but as for now till Mongoose doesnt come up with supporting custom validators, we can use this to solve our problem.
This is a fairly old thread, but I wanted to update the answer for those who come across it like I did.
While you're correct about the context of this being empty in an update validator (per the docs), there is a context option you can use to set the context of this. See the docs
However, a plugin also exists that will check the uniqueness of the field you are setting: mongoose-unique-validator. I use this for checking for duplicate emails. This also has an option for case insensitivity, so I would check it out. It also does run correctly using the update command with the runValidators: true option.
Learning in progress - Stuck working out how have to have 2 separate documents in different collections: events & event reviews
I'm able to reference objectId's between the two but what's the best way to reference data from within those documents if I wanted to populate information from the event into the event review as it's created? I've been trying to trying to google the answer for a few hours now but have so far been unsuccessful with each method I've found. Not looking for a specific answer just somebody with a bit more knowledge to point me in the right direction.
You need to explicitly state that a field in your event reviews schema references the ObjectID of a document in your event collection. Like so...
var mongoose = require('mongoose'), Schema = mongoose.Schema
var EventReviewSchema = new Schema({
someField:{
type: Schema.Types.ObjectId,
ref: 'Event' // Note this is the name of the event model, not schema
}
})
You mentioned that you want to do this "as it's created."
Remember that in MongoDB, write operations are only atomic at the single document level. So there is no "built in" way to guarantee that every event document you create will have a corresponding event review document (I assume that this is what you're trying to do.)
You will need to save an event document, grab that document's ObjectId and use it to populate the appropriate field in your event review document. But remember, there is chance that the second write operation will fail if some error occurs between the time of the first and second write operation.
My advice is to try to write your application in a way that can use a mongoose FindOrCreate method (with a plugin) when querying for event review documents. It's a quick and dirty way to solve this problem.
EDIT:
var Review = mongoose.model('Review', EventReviewSchema)
module.exports.reviewCreate = function(req, res, next) {
var eventid = req.params.eventid
if (eventid) {
var review = new Review() // Review model
review.someField = eventId
review.save(function(err){
if(err) return next(err)
// save successful
})
}
}
I want to make a nice and elegant call to db, but constantly suffer from lack of mongoose experience.
Hope i'm not too annoying, i'm trying to adress Stack Overflow only when my docs-google-Stack-digging skills fails.
What I want to do:
-find a doc in DB
-populate array inside this doc (this array in Schema is an array of Objectids from UserMeta Schema)
-send it throug sockets to some better places
-update found doc with another _id reffering to doc in UserMeta
-save this updates to db & as future reference to var currentroom
Problem occures in last 2 steps, as i can't 'unpopulate' doc, that i already got as response and update-save it further.
For the moment that is how i'm doing this without any population:
Room.findOne({ Roomid: roomid }, function (err, oldRoom) {
client.emit('others', oldRoom);
oldRoom.UsersMeta.push(currentUser._id);
oldRoom.save(function (err, newRoom) {
currentroom = newRoom;
});
})
I can just brute-force-ish copy needed docs through toJSON from parrent UserMeta to this Room doc and just manually maintain both of them. But if there is a way to do this automagically via handy mongoose tools, I would like to take this way. And in the sake of curiosity, of course.
It's a continuation of my previous question Saving reference to a mongoose document, after findOneAndUpdate -
just a remark, you dont really need to go there
Upd: Thing is that I need to run populate() In query with findOne, therefore in response I got oldRoom already with populated _ids
Room.findOne({ Roomid: roomid }).populate('UsersMeta').exec(function (err, oldRoom) {
if (err) {
console.log(err);
}
else if (oldRoom) {
console.log(oldRoom.UsersMeta)
client.emit('others', oldRoom.UsersMeta);
oldRoom.UsersMeta.push(currentUser._id);
oldRoom.save(function (err, newRoom) {
currentroom = newRoom;
console.log(newRoom);
});
}
else { console.log('nothing found') };
})
upd2: so i figured out, that if i push new _id in already populated oldroom and save it, in db it will automagically appear as set of just _id's as it should be. Yet I now confused if i will continue to work with this currentroom reference, as it was already pupulated, how can i safely remove something from populated array without removing populated entry from db completely.
upd3: Ah i just made a mess in my head. For some weird reason i thought that reference to doc saved in variable for each socket client will be always pointing to up-to-date doc in db, and that i will be able to work with this doc through it elluminating need to using find db tools more than once to get this reference... I need to rethink my db logic.
SO
There is a question then. If user connected to Rooms which is a doc from RoomSchema, and a user is a socket user i.e he has a personal scope in which i can store his personal session details. Can i somehow store direct link to this particular Room doc to elluminate need of searching for this room through whole db if user, for example, changes room's name. If i NEED to searh - it seems that it a better practice to save an id of room in which user is, and then just look up in db for room by this id and change it's name, am I right?
Here is the query to get UserMeta with ids only
Room.findOne({ Roomid: roomid },function (err, oldRoom) {
});
NodeJS + Express, MongoDB + Mongoose
I have a JSON feed where each record has a set of "venue" attributes (things like "venue name" "venue location" "venue phone" etc). I want to create a collection of all venues in the feed -- one instance of each venue, no dupes.
I loop through the JSON and test whether the venue exists in my venue collection. If it doesn't, save it.
jsonObj.events.forEach(function(element, index, array){
Venue.findOne({'name': element.vname}, function(err,doc){
if(doc == null){
var instance = new Venue();
instance.name = element.vname;
instance.location = element.location;
instance.phone = element.vphone;
instance.save();
}
}
}
Desired: A list of all venues (no dupes).
Result: Plenty of dupes in the venue collection.
Basically, the loop created a new Venue record for every record in the JSON feed.
I'm learning Node and its async qualities, so I believe the for loop finishes before even the first save() function finishes -- so the if statement is always checking against an empty collection. Console.logging backs this claim up.
I'm not sure how to rework this so that it performs the desired task. I've tried caolan's async module but I can't get it to help. There's a good chance I'm using incorrectly.
Thanks so much for pointing me in the right direction -- I've searched to no avail. If the async module is the right answer, I'd love your help with how to implement it in this specific case.
Thanks again!
Why not go the other way with it? You didn't say what your persistence layer is, but it looks like mongoose or possibly FastLegS. In either case, you can create a Unique Index on your Name field. Then, you can just try to save anything, and handle the error if it's a unique index violation.
Whatever you do, you must do as #Paul suggests and make a unique index in the database. That's the only way to ensure uniqueness.
But the main problem with your code is that in the instance.save() call, you need a callback that triggers the next iteration, otherwise the database will not have had time to save the new record. It's a race condition. You can solve that problem with caolan's forEachSeries function.
Alternatively, you could get an array of records already in the Venue collection that match an item in your JSON object, then filter the matches out of the object, then iteratively add each item left in the filtered JSON object. This will minimize the number of database operations by not trying to create duplicates in the first place.
Venue.find({'name': { $in: jsonObj.events.map(function(event){ return event.vname; }) }}, function (err, docs){
var existingVnames = docs.map(function(doc){ return doc.name; });
var filteredEvents = jsonObj.events.filter(function(event){
return existingVnames.indexOf(event.vname) === -1;
});
filteredEvents.forEach(function(event){
var venue = new Venue();
venue.name = event.vname;
venue.location = event.location;
venue.phone = event.vphone;
venue.save(function (err){
// Optionally, do some logging here, perhaps.
if (err) return console.error('Something went wrong!');
else return console.log('Successfully created new venue %s', venue.name);
});
});
});