I was wondering if there is way to force a unique collection entry but only if entry is not null.
e
Sample schema:
var UsersSchema = new Schema({
name : {type: String, trim: true, index: true, required: true},
email : {type: String, trim: true, index: true, unique: true}
});
'email' in this case is not required but if 'email' is saved I want to make sure that this entry is unique (on a database level).
Empty entries seem to get the value 'null' so every entry wih no email crashes with the 'unique' option (if there is a different user with no email).
Right now I'm solving it on an application level, but would love to save that db query.
thx
As of MongoDB v1.8+ you can get the desired behavior of ensuring unique values but allowing multiple docs without the field by setting the sparse option to true when defining the index. As in:
email : {type: String, trim: true, index: true, unique: true, sparse: true}
Or in the shell:
db.users.ensureIndex({email: 1}, {unique: true, sparse: true});
Note that a unique, sparse index still does not allow multiple docs with an email field with a value of null, only multiple docs without an email field.
See http://docs.mongodb.org/manual/core/index-sparse/
tl;dr
Yes, it is possible to have multiple documents with a field set to null or not defined, while enforcing unique "actual" values.
requirements:
MongoDB v3.2+.
Knowing your concrete value type(s) in advance (e.g, always a string or object when not null).
If you're not interested in the details, feel free to skip to the implementation section.
longer version
To supplement #Nolan's answer, starting with MongoDB v3.2 you can use a partial unique index with a filter expression.
The partial filter expression has limitations. It can only include the following:
equality expressions (i.e. field: value or using the $eq operator),
$exists: true expression,
$gt, $gte, $lt, $lte expressions,
$type expressions,
$and operator at the top-level only
This means that the trivial expression {"yourField"{$ne: null}} cannot be used.
However, assuming that your field always uses the same type, you can use a $type expression.
{ field: { $type: <BSON type number> | <String alias> } }
MongoDB v3.6 added support for specifying multiple possible types, which can be passed as an array:
{ field: { $type: [ <BSON type1> , <BSON type2>, ... ] } }
which means that it allows the value to be of any of a number of multiple types when not null.
Therefore, if we want to allow the email field in the example below to accept either string or, say, binary data values, an appropriate $type expression would be:
{email: {$type: ["string", "binData"]}}
implementation
mongoose
You can specify it in a mongoose schema:
const UsersSchema = new Schema({
name: {type: String, trim: true, index: true, required: true},
email: {
type: String, trim: true, index: {
unique: true,
partialFilterExpression: {email: {$type: "string"}}
}
}
});
or directly add it to the collection (which uses the native node.js driver):
User.collection.createIndex("email", {
unique: true,
partialFilterExpression: {
"email": {
$type: "string"
}
}
});
native mongodb driver
using collection.createIndex
db.collection('users').createIndex({
"email": 1
}, {
unique: true,
partialFilterExpression: {
"email": {
$type: "string"
}
}
},
function (err, results) {
// ...
}
);
mongodb shell
using db.collection.createIndex:
db.users.createIndex({
"email": 1
}, {
unique: true,
partialFilterExpression: {
"email": {$type: "string"}
}
})
This will allow inserting multiple records with a null email, or without an email field at all, but not with the same email string.
Just a quick update to those researching this topic.
The selected answer will work, but you might want to consider using partial indexes instead.
Changed in version 3.2: Starting in MongoDB 3.2, MongoDB provides the
option to create partial indexes. Partial indexes offer a superset of
the functionality of sparse indexes. If you are using MongoDB 3.2 or
later, partial indexes should be preferred over sparse indexes.
More doco on partial indexes: https://docs.mongodb.com/manual/core/index-partial/
Actually, only first document where "email" as field does not exist will get save successfully. Subsequent saves where "email" is not present will fail while giving error ( see code snippet below). For the reason look at MongoDB official documentation with respect to Unique Indexes and Missing Keys here at http://www.mongodb.org/display/DOCS/Indexes#Indexes-UniqueIndexes.
// NOTE: Code to executed in mongo console.
db.things.ensureIndex({firstname: 1}, {unique: true});
db.things.save({lastname: "Smith"});
// Next operation will fail because of the unique index on firstname.
db.things.save({lastname: "Jones"});
By definition unique index can only allow one value to be stored only once. If you consider null as one such value it can only be inserted once! You are correct in your approach by ensuring and validating it at application level. That is how it can be done.
You may also like to read this http://www.mongodb.org/display/DOCS/Querying+and+nulls
Related
I have the following postSchema and would like to fetch datas depending on updatedAt field. When people make comment I increase numberofreply by one and its updatedAt is updated. How should I fetch datas for infinite scroll and should I use indexing for this operation ?
const postScheme = mongoose.Schema(
{
post: {
type: String,
trim: true,
},
numberOfReply: {
type: Number,
default: 0
},
owner: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
hasImage: {
type: Boolean,
},
image: {
type: String,
trim: true
},
},
{timestamps: true}
)
this is what I use to fetch first page
Post.Post.find({}).sort({'updatedAt': -1}).limit(10).populate('owner').populate('coin').exec(function (err, posts) {
res.send(posts)
})
this is for infinite scroll
Post.Post.find({isCoin: true, updatedAt: {$lt: req.body.last}}).sort({'updatedAt': -1}).populate('owner').limit(
10).exec(function (err, posts) {
res.send(posts)
})
The limit, skip syntax is Mongo's way of paginating through data so you got that worked out, from a code perspective you can't really change anything to work better.
should I use indexing for this operation
Most definitely yes, indexes are the way to make this operation be efficient. otherwise Mongo will do a collection scan for each pagination which is very inefficient.
So what kind of index you should built? Well you want to build a compound index that will allow the query to both satisfy the query and the sort conditions, and in your case that is on the isCoin and updateAt fields, like so:
db.collection.createIndex( { isCoin: 1, updateAt: -1 } )
A few improvements you can make to make the index a bit more efficient (for this specific query) are:
Consider creating the index as a sparse index, this will only index documents with both fields in them, obviously if the data doesn't include this options you can ignore it.
This one has a few caveats in it, but partial indexes are designed for this case, to improve query performance by indexing a smaller subset of the data. and in your case you can add this option
{ partialFilterExpression: { isCoin: true } }
with that said this will limit your index usage for other queries so it might not be the ultimate choice for you.
I am using the mongoose framework and trying to insert data. This is my schema:
var personSchema = new mongoose.Schema({
"gender": {
type: String,
enum: ["male", "female", "other", "unknown"],
lowercase: false,
},
...
});
I only want to allow case-sensitive values of the name. So the dataset {"gender":"Male"} should create an error. But in fact, it doesn't matter if I use lowercase: true or false, it creates the Object and uses the lowercase value of my enum. Only when I delete the lowercase attribute, then my inserted dataset isn't accepted.
Is there a fix for this problem?
Ok, this is apperently a bug from mongoose. I created a bug report on GitHub: https://github.com/Automattic/mongoose/issues/4622
I have a mongoose model: (With a field that has a default)
var MySchema= new mongoose.Schema({
name: {
type: String,
required: true
},
isClever: {
type: Boolean,
default: false
}
});
I can save a model of this type by just saving a name and in mongoDB, only name can be seen in the document (and not isClever field). That's fine because defaults happen at the mongoose level. (?)
The problem I am having then is, when trying to retrieve only people called john and isClever = false:
MySchema.find({
'name' : 'john',
'isClever': false
}).exec( function(err, person) {
// person is always null
});
It always returns null. Is this something related to how defaults work with mongoose? We can't match on a defaulted value?
According to Mongoose docs, default values are applied when the document skeleton is constructed.
When you execute a find query, it is passed to Mongo when no document is constructed yet. Mongo is not aware about defaults, so since there are no documents where isClever is explicitly true, that results in empty output.
To get your example working, it should be:
MySchema.find({
'name' : 'john',
'isClever': {
$ne: true
}
})
For my project, I want to keep a mongoose document for groups of organizations, like this:
var groupSchema = Schema({
name : { type : String },
org : { type : Schema.Types.ObjectId, ref : 'Organization' },
...
users : [{
uid : { type : Schema.Types.ObjectId, ref : 'User' },
...
}]
});
I want to prevent the same user from being in the same group twice. To do this, I need to force users.uid to be unique in the users array. I tried stating 'unique : true' for uid, but that didn't work. Is there a way to do this with mongoose or mongoDB without extra queries or splitting the schema?
Edit:
I changed the previous value of uid to
uid : { type : Schema.Types.ObjectId, ref : 'User', index: {unique: true, dropDups: true} }
But this still doesn't seem to work.
Edit: Assuming there is no simple way to achieve this, I added an extra query checking if the user is already in the group. This seems to me the simplest way.
A unique index on an array field enforces that the same value cannot appear in the arrays of more than one document in the collection, but doesn't prevent the same value from appearing more than once in a single document's array. So you need to ensure uniqueness as you add elements to the array instead.
Use the $addToSet operator to add a value to an array only if the value is not already present.
Group.updateOne({name: 'admin'}, {$addToSet: {users: userOid}}, ...
However, if the users array contains objects with multiple properties and you want to ensure uniqueness over just one of them (uid in this case), then you need to take another approach:
var user = { uid: userOid, ... };
Group.updateOne(
{name: 'admin', 'users.uid': {$ne: user.uid}},
{$push: {users: user}},
function(err, numAffected) { ... });
What that does is qualify the $push update to only occur if user.uid doesn't already exist in the uid field of any of the elements of users. So it mimics $addToSet behavior, but for just uid.
Well this might be old question but for mongoose > v4.1, you can use $addToSet operator.
The $addToSet operator adds a value to an array unless the value is already present, in which case $addToSet does nothing to that array.
example:
MyModal.update(
{ _id: 1 },
{ $addToSet: {letters: [ "c", "d" ] } }
)
I'm using Sails.js (0.9.8) and MongoDB (via the sails-mongo adaptor) to create a collection of pages that can be positioned in a tree-view. I would like to store the path of a page in an array of UUIDs
My model:
module.exports = {
schema: true,
attributes: {
uuid: {
type: 'string',
unique: true,
required: true,
uuidv4: true
},
name: {
type: 'string',
required: true,
empty: false
},
path: {
type: 'array',
required: true,
array: true
}
}
}
It works well when I save a 'root' page (the 'path' property has just one item because it's a root page. Here is what it was saved in MongoDB:
{
_id: ObjectId("52f853e9609fb6c0341bdfcc"),
createdAt: ISODate("2014-02-10T04:22:01.828Z"),
name: "Home Page",
path: [
"a2b23e1f-954b-49a3-91f1-4d62d209a093"
],
updatedAt: ISODate("2014-02-10T04:22:01.833Z"),
uuid: "a2b23e1f-954b-49a3-91f1-4d62d209a093"
}
But when I want to create a 'subpage' below my previous created page (Home Page/Products), I get this error:
MongoError: E11000 duplicate key error index: cms-project.item.$path_1
dup key: { : "a2b23e1f-954b-49a3-91f1-4d62d209a093" }
Here is the data I sent:
{ name: 'Products',
uuid: 'a004ee54-7e42-49bf-976c-9bb93c118038',
path:
[ 'a2b23e1f-954b-49a3-91f1-4d62d209a093',
'a004ee54-7e42-49bf-976c-9bb93c118038' ] }
I probably missed something but I don't know what.
If I store the path in a string instead of an array, it work well, but I find it much less elegant and handy.
Not sure of all the Sails / Waterline parts myself as I've never played with it. But by the error the problem is there is a unique index on your array field.
When you are inserting your second document, you already have one of the values (the parent) in your path field in another document. The unique constraint is not going to allow this. Most certainly for what you are modelling, you do not want this and the index cannot be unique.
I hope that you set this up yourself under the assumption that it meant unique within the array contained in the document. If you did then you know where to look and what to change now. If this is being automatically deployed somehow, then I'm not the one to help.
Change the index to not be unique. You can confirm this through the mongo shell:
use cms-project
db.item.getIndices()
Good luck