Mongoose unique validation error type - node.js

I'm using this schema with mongoose 3.0.3 from npm:
var schema = new Schema({
_id: Schema.ObjectId,
email: {type: String, required: true, unique: true}
});
If I try to save a email that is already in db, I expect to get a ValidationError like if a required field is omitted. However this is not the case, I get a MongoError: E11000 duplicate key error index.
Which is not a validation error (happens even if I remove the unique:true).
Any idea why?

I prefer putting it in path validation mechanisms, like
UserSchema.path('email').validate(function(value, done) {
this.model('User').count({ email: value }, function(err, count) {
if (err) {
return done(err);
}
// If `count` is greater than zero, "invalidate"
done(!count);
});
}, 'Email already exists');
Then it'll just get wrapped into ValidationError and will return as first argument when you call validate or save .

I had some issues with the approved answer. Namely:
this.model('User') didn't work for me.
the callback done wasn't working properly.
I resolved those issues by:
UserSchema.path('email').validate(async (value) => {
const emailCount = await mongoose.models.User.countDocuments({email: value });
return !emailCount;
}, 'Email already exists');
I use async/await which is a personal preference because it is much neater: https://javascript.info/async-await.
Let me know if I got something wrong.

This is expected behavior
The unique: true is equivalent to setting an index in mongodb like this:
db.myCollection.ensureIndex( { "email": 1 }, { unique: true } )
To do this type of validation using Mongoose (Mongoose calls this complex validation- ie- you are not just asserting the value is a number for example), you will need to wire in to the pre-save event:
mySchema.pre("save",function(next, done) {
var self = this;
mongoose.models["User"].findOne({email : self.email},function(err, results) {
if(err) {
done(err);
} else if(results) { //there was a result found, so the email address exists
self.invalidate("email","email must be unique");
done(new Error("email must be unique"));
} else {
done();
}
});
next();
});

Simply response to json
try {
let end_study_year = new EndStudyYear(req.body);
await end_study_year.save();
res.json({
status: true,
message: 'បានរក្សាទុក!'
})
}catch (e) {
res.json({
status: false,
message: e.message.toString().includes('duplicate') ? 'ទិន្នន័យមានរួចហើយ' : e.message.split(':')[0] // check if duplicate message exist
})
}

Sorry for answering an old question. After testing I feel good to have find these answers, so I will give my experience. Both top answers are great and right, just remember that:
if your document is new, you can just validate if count is higher than 0, thats the common situation;
if your document is NOT new and has modified the unique field, you need to validate with 0 too;
if your document is NOT new and has NOT being modified, just go ahead;
Here is what I made in my code:
UserSchema.path('email').validate(async function validateDuplicatedEmail(value) {
if (!this.isNew && !this.isModified('email')) return true;
try {
const User = mongoose.model("User");
const count = await User.countDocuments({ email: value });
if (count > 0) return false;
return true;
}
catch (error) {
return false;
}
}, "Email already exists");

Related

mongoose-unique-validator detects all inputs as already existed

I am trying to create a unique email value in MongoDB. I used mongoose-unique-validator to implement that but resulted in error claiming that my unique emails that I just inputted are already existed.
This is the error message I received from trying to input unique email.
"message": "User validation failed: email: Error, expected email to be
unique., _id: Error, expected _id to be unique."
They said the Email and _id are not unique. But.., _id is an auto-generated value to be unique in MongoDB meaning that it should not be detected as a duplicated value.
I am not sure if that was caused by my own implementation so I would look forward to see any assumption or ways to debug to the root cause of this too. I tried restarting from fresh Mongo DB and manually inputting uniquely via Postman but still no hope.
These are a part of the codes that might be related to the data creation on MongoDB
UserModel.js
var uniqueValidator = require('mongoose-unique-validator');
const { Schema } = mongoose;
const UsersSchema = new Schema({
email: { type: String, unique: true, required: true},
hashedPassword: String,
salt: String,
});
UsersSchema.plugin(uniqueValidator, { message: 'Error, expected {PATH} to be unique.' });
UsersSchema.set('autoIndex', false);
Server.js ---- /api/users/register
const finalUser = new Users(user);
finalUser.setPassword(user.password);
finalUser.save((err, data) => {
if (err) {
return res.status(400).json(err)
}
return res.json({ user: finalUser.toAuthJSON() })
})
Additional Information
I tried this solution from Sritam to detect another email with the same value but it still claims that the inputted email is already existed.
UsersSchema.path('email').validate(async (value) => {
const emailCount = await mongoose.models.User.countDocuments({email: value });
return !emailCount;
}, 'Email already exists');
"message": "User validation failed: email: Email already exists"
You can use validateModifiedOnly option in Document.save(). _id and email fields will never be validated unless they are modified. The code should look like this:
finalUser.save({ validateModifiedOnly: true }, (err, data) => {
if (err) {
return res.status(400).json(err)
}
return res.json({ user: finalUser.toAuthJSON() })
})
Found the issue and solution!
They are acting weird like in the question because the model was not initialized.
I must perform Schema.init() before performing any model validation.
The solution is to add UsersSchema.init().then(() => {...your operation})
Now Server.js ---- /api/users/register should look like this.
Users.init().then(() => { // where Users is my UsersModel
finalUser.save((err, data) => {
if (err) {
return res.status(400).json(err)
}
return res.json({ user: finalUser.toAuthJSON() })
})
})
Hope this helps other developers who experience similarly odd error!

Custom Validation on a field that checks if field already exists and is active

I have a mongodb Collection "Users" having "Name", "Email", "Active" fields.
I want to add a validation that for every document email should be unique. However if a document is invalid i.e Active is false then the email can be accepted.
Here is the model
email: { type: String, validate: {
validator: function(v) {
return new Promise((resolve, reject)=> {
console.log("validating email")
const UserModel = mongoose.model('User');
UserModel.find({email : v, active: true}, function (err, docs)
{
if (!docs.length){
resolve();
}else{
console.log('user exists: ',v);
reject(new Error("User exists!"));
}
});
})
},
message: '{VALUE} already exists!'
}
},
name: {
type: String,
required: true
},
active: {
type: Boolean,
default: true
}
Problem is whenever i do any updation on this model then this validation is called.
So if i update the name then also this validation is called and it gives the error that email already exists.
How do I add a validation on email field so if someone adds a new entry to database or updates email it checks in database if existing user has same email id and is active?
I would first call Mongoose findOne function if the User is already registered the Mongo DB, for example;
let foundUser = await User.findOne({email});
if (!foundUser) {
// update user, create user etc.
...
}
I think it is better to not use logic inside the Mongoose document object. Maybe there is a way to achieve it but I prefer to do these validations in the code, not in the document, it is just my preference.
Also you can try making email unique as follows:
email: {
type: String,
unique: true
}
I'd use unique compound index, instead of having one more additional query to your db. Your code would look like this:
const schema = = new Schema(...);
schema.index({email: 1, active: 1}, {unique: true});
Mongo itself will reject your documents and you can catch it in your code like this:
const {MongoError} = require('mongodb'); // native driver
try {
await model.updateOne(...).exec(); // or .save()
} catch (err) {
//11000 is error code for unique constraints
if (err instanceof MongoError && err.code === 11000)
console.error('Duplicate email/active pair');
}

Mongodb/mongoose omit a field in response [duplicate]

I have a NodeJS application with Mongoose ODM(Mongoose 3.3.1). I want to retrieve all fields except 1 from my collection.For Example: I have a collection Product Which have 6 fields,I want to select all except a field "Image" . I used "exclude" method, but got error..
This was my code.
var Query = models.Product.find();
Query.exclude('title Image');
if (req.params.id) {
Query.where('_id', req.params.id);
}
Query.exec(function (err, product) {
if (!err) {
return res.send({ 'statusCode': 200, 'statusText': 'OK', 'data': product });
} else {
return res.send(500);
}
});
But this returns error
Express
500 TypeError: Object #<Query> has no method 'exclude'.........
Also I tried, var Query = models.Product.find().exclude('title','Image'); and var Query = models.Product.find({}).exclude('title','Image'); But getting the same error. How to exclude one/(two) particular fields from a collection in Mongoose.
Use query.select for field selection in the current (3.x) Mongoose builds.
Prefix a field name you want to exclude with a -; so in your case:
Query.select('-Image');
Quick aside: in JavaScript, variables starting with a capital letter should be reserved for constructor functions. So consider renaming Query as query in your code.
I don't know where you read about that .exclude function, because I can't find it in any documentation.
But you can exclude fields by using the second parameter of the find method.
Here is an example from the official documentation:
db.inventory.find( { type: 'food' }, { type:0 } )
This operation returns all documents where the value of the type field is food, but does not include the type field in the output.
Model.findOne({ _id: Your Id}, { password: 0, name: 0 }, function(err, user){
// put your code
});
this code worked in my project. Thanks!! have a nice day.
You could do this
const products = await Product.find().select(['-image'])
I am use this with async await
async (req, res) => {
try {
await User.findById(req.user,'name email',(err, user) => {
if(err || !user){
return res.status(404)
} else {
return res.status(200).json({
user,
});
}
});
} catch (error) {
console.log(error);
}
In the updated version of Mongoose you can use it in this way as below to get selected fields.
user.findById({_id: req.body.id}, 'username phno address').then(response => {
res.status(200).json({
result: true,
details: response
});
}).catch(err => {
res.status(500).json({ result: false });
});
I'm working on a feature. I store a userId array name "collectedUser" than who is collected the project. And I just want to return a field "isCollected" instead of "collectedUsers". So select is not what I want. But I got this solution.
This is after I get projects from database, I add "isCollected".
for (const item of projects) {
item.set("isCollected", item.collectedUsers.includes(userId), {
strict: false,
})
}
And this is in Decorator #Schema
#Schema({
timestamps: true,
toObject: {
virtuals: true,
versionKey: false,
transform: (doc, ret, options): Partial<Project> => {
return {
...ret,
projectManagers: undefined,
projectMembers: undefined,
collectedUsers: undefined
}
}
}
})
Finally in my controller
projects = projects.map(i => i.toObject())
It's a strange tricks that set undefined, but it really work.
Btw I'm using nestjs.
You can do it like this
const products = await Product.find().select({
"image": 0
});
For anyone looking for a way to always omit a field - more like a global option rather than doing so in the query e.g. a password field, using a getter that returns undefined also works
{
password: {
type: String,
required: true,
get: () => undefined,
},
}
NB: Getters must be enabled with option { toObject: { getters:true } }
you can exclude the field from the schema definition
by adding the attribute
excludedField : {
...
select: false,
...
}
whenever you want to add it to your result,
add this to your find()
find().select('+excludedFiled')

Bcrypt-NodeJS compare() returns false whatever the password

I know that question has already been asked a few times (like here, here or there, or even on Github, but none of the answers actually worked for me...
I am trying to develop authentication for a NodeJS app using Mongoose and Passport, and using Bcrypt-NodeJS to hash the users' passwords.
Everything was working without any problem before I decided to refactor the User schema and to use the async methods of bcrypt. The hashing still works while creating a new user but I am now unable to verify a password against its hash stored in MongoDB.
What do I know?
bcrypt.compare() always returns false whatever the password is correct or not, and whatever the password (I tried several strings).
The password is only hashed once (so the hash is not re-hashed) on user's creation.
The password and the hash given to the compare method are the right ones, in the right order.
The password and the hash are of type "String".
The hash isn't truncated when stored in the database (60 characters long string).
The hash fetched in the database is the same as the one stored on user's creation.
Code
User schema
Some fields have been stripped to keep it clear, but I kept the relevant parts.
var userSchema = mongoose.Schema({
// Local authentication
password: {
hash: {
type: String,
select: false
},
modified: {
type: Date,
default: Date.now
}
},
// User data
profile: {
email: {
type: String,
required: true,
unique: true
}
},
// Dates
lastSignedIn: {
type: Date,
default: Date.now
}
});
Password hashing
userSchema.statics.hashPassword = function(password, callback) {
bcrypt.hash(password, bcrypt.genSaltSync(12), null, function(err, hash) {
if (err) return callback(err);
callback(null, hash);
});
}
Password comparison
userSchema.methods.comparePassword = function(password, callback) {
// Here, `password` is the string entered in the login form
// and `this.password.hash` is the hash stored in the database
// No problem so far
bcrypt.compare(password, this.password.hash, function(err, match) {
// Here, `err == null` and `match == false` whatever the password
if (err) return callback(err);
callback(null, match);
});
}
User authentication
userSchema.statics.authenticate = function(email, password, callback) {
this.findOne({ 'profile.email': email })
.select('+password.hash')
.exec(function(err, user) {
if (err) return callback(err);
if (!user) return callback(null, false);
user.comparePassword(password, function(err, match) {
// Here, `err == null` and `match == false`
if (err) return callback(err);
if (!match) return callback(null, false);
// Update the user
user.lastSignedIn = Date.now();
user.save(function(err) {
if (err) return callback(err);
user.password.hash = undefined;
callback(null, user);
});
});
});
}
It may be a "simple" mistake I made but I wasn't able to find anything wrong in a few hours... May you have any idea to make that method work, I would be glad to read it.
Thank you guys.
Edit:
When running this bit of code, match is actually equal to true. So I know my methods are correct. I suspect this has something to do with the storage of the hash in the database, but I really have no idea of what can cause this error to occur.
var pwd = 'TestingPwd01!';
mongoose.model('User').hashPassword(pwd, function(err, hash) {
console.log('Password: ' + pwd);
console.log('Hash: ' + hash);
user.password.hash = hash;
user.comparePassword(pwd, function(err, match) {
console.log('Match: ' + match);
});
});
Edit 2 (and solution) :
I put it there in case it could be helpful to someone one day...
I found the error in my code, which was occurring during the user's registration (and actually the only piece of code I didn't post here). I was hashing the user.password object instead of user.password.plaintext...
It's only by changing my dependencies from "brcypt-nodejs" to "bcryptjs" that I was able to find the error because bcryptjs throws an error when asked to hash an object, while brcypt-nodejs just hashes the object as if it were a string.
I know the solution has been found but just in case you are landing here out of google search and have the same issue, especially if you are using a schema.pre("save") function, sometimes there is a tendency of saving the same model several times, hence re-hashing the password each time. This is especially true if you are using references in mongoDB to create schema relationship. Here is what my registration function looked like:
Signup Code
User.create(newUser, (err, user) => {
if (err || !user) {
console.warn("Error at stage 1");
return res.json(transformedApiRes(err, "Signup error", false)).status(400);
}
let personData: PersonInterface = <PersonInterface>{};
personData.firstName = req.body.first_name;
personData.lastName = req.body.last_name;
personData.user = user._id;
Person.create(personData, function (err1: Error, person: any): any {
if (err1 || !person) {
return res.json(transformedApiRes(err1, "Error while saving to Persons", false));
}
/* One-to-One relationship */
user.person = person;
user.save(function (err, user) {
if (err || !user) {
return res.json({error: err}, "Error while linking user and person models", false);
}
emitter.emit("userRegistered", user);
return res.json(transformedApiRes(user, `Signup Successful`, true));
});
});
});
As you can see there is a nested save on User because I had to link the User model with Person model (one-to-one). As a result, I had the mismatch error because I was using a pre-save function and every time I triggered User.create or User.save, the function would be called and it would re-hash the existing password. A console statement inside pre-save gave me the following, showing that indeed that password was re-hashed:
Console debug after a single signup call
{ plain: 'passwd',
hash: '$2b$10$S2g9jIcmjGxE0aT1ASd6lujHqT87kijqXTss1XtUHJCIkAlk0Vi0S' }
{ plain: '$2b$10$S2g9jIcmjGxE0aT1ASd6lujHqT87kijqXTss1XtUHJCIkAlk0Vi0S',
hash: '$2b$10$KRkVY3M8a8KX9FcZRX.l8.oTSupI/Fg0xij9lezgOxN8Lld7RCHXm' }
The Fix, The Solution
To fix this, you have to modify your pre("save") code to ensure the password is only hashed if it is the first time it is being saved to the db or if it has been modified. To do this, surround your pre-save code in these blocks:
if (user.isModified("password") || user.isNew) {
//Perform password hashing here
} else {
return next();
}
Here is how the whole of my pre-save function looks like
UsersSchema.pre("save", function (next: NextFunction): any {
let user: any = this;
if (user.isModified("password") || user.isNew) {
bcrypt.genSalt(10, function (err: Error, salt: string): any {
if (err) {
return next(err);
}
bcrypt.hash(user.password, salt, function (err: Error, hash: string) {
if (err) {
console.log(err);
return next(err);
}
console.warn({plain: user.password, hash: hash});
user.password = hash;
next();
});
});
} else {
return next();
}
});
Hopefully this helps someone.
I am dropping this here because it might help someone someday.
In my own case, the reason why I was having bcrypt.compare as false even when I supplied the right authentication details was because of the constraints on the datatype in the model. So each time the hash was saved in the DB, it was truncated in order to fit into the 50 characters constraints.
I had
'password': {
type: DataTypes.STRING(50),
allowNull: false,
comment: "null"
},
The string could only contain 50 characters but the result of bcrypt.hash was more than that.
FIX
I modified the model thus DataTypes.STRING(255)
bcrypt.hash() has 3 arguments... you have 4 for some reason.
Instead of
bcrypt.hash(password, bcrypt.genSaltSync(12), null, function(err, hash) {
it should be
bcrypt.hash(password, bcrypt.genSaltSync(12), function(err, hash) {
Since you were hashing only during user creation, then you might not have been hashing properly. You may need to re-create the users.
Tip: If you are switching
then().then()
Block always check return value.
You can always check the max length for the password field in the database. Make sure it is large. In my case, I have set it to 500. And then the code worked flawlessly!
TS version
const { phone, password } = loginDto;
const user = await this.usersService.findUserByPhone(phone);
const match = await compare(password, user.password);
if (user && match){
return user
}else{
throw new UnauthorizedException();
}
JS version
const { phone, password } = loginDto;
const user = await this.usersService.findUserByPhone(phone);
const match = await bcrypt.compare(password, user.password);
if (user && match){
return user
}else{
throw new UnauthorizedException();
}

Mongoose helper method has no findOne method?

How can I add helper methods to find and save objects in Mongoose. A friend told me to use helper methods but I cannot get them to work after a day. I always receive errors saying that either findOne() or save() does not exist OR that next callback is undefined (when node compiles ... before I execute it):
I've tried _schema.methods, _schema.statics... nothing works...
var email = require('email-addresses'),
mongoose = require('mongoose'),
strings = require('../../utilities/common/strings'),
uuid = require('node-uuid'),
validator = require('validator');
var _schema = new mongoose.Schema({
_id: {
type: String,
trim: true,
lowercase: true,
default: uuid.v4
},
n: { // Name
type: String,
required: true,
trim: true,
lowercase: true,
unique: true,
index: true
}
});
//_schema.index({
// d: 1,
// n: 1
//}, { unique: true });
_schema.pre('save', function (next) {
if (!this.n || strings.isNullOrWhitespace(this.n)){
self.invalidate('n', 'Domain name required but not supplied');
return next(new Error('Domain name required but not supplied'));
}
var a = email.parseOneAddress('test#' + this.n);
if (!a || !a.local || !a.domain){
self.invalidate('n', 'Name is not valid domain name');
return next(new Error('Name is not valid domain name'));
}
next();
});
_schema.statics.validateForSave = function (next) {
if (!this.n || strings.isNullOrWhitespace(this.n)){
return next(new Error('Domain name required but not supplied'));
}
var a = email.parseOneAddress('test#' + this.n);
if (!a || !a.local || !a.domain){
return next(new Error('Name is not valid domain name'));
}
next();
}
_schema.statics.findUnique = function (next) {
this.validateForSave(function(err){
if (err){ return next(err); }
mongoose.model('Domain').findOne({ n: this.n }, next);
//this.findOne({ n: this.n }, next);
});
}
_schema.statics.init = function (next) {
this.findUnique(function(err){
if (err){ return next(err); }
this.save(next);
});
}
var _model = mongoose.model('Domain', _schema);
module.exports = _model;
I believe you are running into issues because of your usage with this. Every time you enter a new function this's context is changing. You can read more about this at mdn.
Additionally your callbacks aren't allowing anything to be passed into the mongoose method. For example if I was to create the most basic "save" method I I would do the following:
_schema.statics.basicCreate = function(newDocData, next) {
new _model(newDocData).save(next);
}
Now if I wanted to search the Domain collection for a unique document I would use the following:
_schema.statics.basicSearch = function(uniqueName, next) {
var query = {n: uniqueName};
_model.findOne(query, function(err, myUniqueDoc) {
if (err) return next(err);
if (!myUniqueDoc) return next(new Error("No Domain with " + uniqueName + " found"));
next(null, myNewDoc);
});
}
Mongoose has built in validations for what you are doing:
_schema.path("n").validate(function(name) {
return name.length;
}, "Domain name is required");
_schema.path("n").validate(function(name) {
var a = email.parseOneAddress("test#" + name);
if (!a || !a.local || !a.domain) {
return false;
}
return true;
}, "Name is not a valid domain name");
It returns a boolean. If false, it passes an error to the .save() callback with the stated message. For validating uniqueness:
_schema.path("n").validate(function(name, next) {
var self = this;
this.model("Domain").findOne({n: name}, function(err, domain) {
if (err) return next(err);
if (domain) {
if (self._id === domain._id) {
return next(true);
}
return next(false);
}
return next(true);
});
}, "This domain is already taken");
You're using self = this here so that you can access the document inside the findOne() callback. false is being passed to the callback if the name exists and the result that is found isn't the document itself.
I've tried _schema.methods, _schema.statics
To clarify, .statics operate on the Model, .methods operate on the document. Zane gave a good example of statics, so here is an example of methods:
_schema.methods.isDotCom = function() {
return (/.com/).test(this.n);
}
var org = new Domain({n: "stuff.org"});
var com = new Domain({n: "things.com"});
org.isDotCom(); // false
com.isDotCom(); // true
Opinion: It's neat to have mongoose do validations, but it's very easy to forget it's happening. You also may want to have some validation in one area of your app while using different validations elsewhere. I'd avoid using most of it unless you definitively know you will have to do the same thing every time and will never NOT have to do it.
Methods/statics are a different story. It's very convenient to call isDotCom() instead of writing out a regex test every time you need to check. It performs a single and simple task that saves you some typing and makes your code more readable. Using methods for boolean checks can add a ton of readability. Defining statics like findByName (Zane's basicSearch) is great when you know you're going to do a simple query like that repeatedly.
Treat Mongoose as a utility, not as core functionality.

Resources