Using sparse: true still getting MongoError: E11000 duplicate key error - node.js

Schema (../models/add.js)
var addSchema = new Schema({
name: {type: String, unique: true, sparse: true},
phone: Number,
email: String,
country: Number
});
module.exports = mongoose.model('Contact', addSchema);
add-manager.js
var Add = require('../models/add.js');
var AM = {};
var mongoose = require('mongoose');
module.exports = AM;
AM.notOwned = function(country, callback)
{
Add.update({country: country}, {country: country}, {upsert: true}, function(err, res){
if (err) callback (err);
else callback(null, res);
})
}
news.js
// if country # is not in the database
AM.notOwned(country, function(error, resp){
if (error) console.log("error: "+error);
else
{
// do stuff
}
})
error:
MongoError: E11000 duplicate key error index: bot.contacts.$name_1 dup key: { : null }
After seeing the error message, I googled around and learned that when the document is created, since name isn't set, its treated as null. See Mongoose Google Group Thread The first time AM.notOwned is called it will work as there isn't any documents in the collection without a name key. AM.notOwned will then insert a document with an ID field, and a country field.
Subsequent AM.notOwned calls fails because there is already a document with no name field, so its treated as name: null, and the second AM.notOwned is called fails as the field "name" is not set and is treated as null as well; thus it is not unique.
So, following the advice of the Mongoose thread and reading the mongo docs I looked at using sparse: true. However, its still throwing the same error. Further looking into it, I thought it may be the same issue as: this, but setting schema to name: {type: String, index: {unique: true, sparse: true}} doesn't fix it either.
This S.O. question/answer leads me to believe it could be caused by the index not being correct, but I'm not quite sure how to read the the db.collection.getIndexes() from Mongo console.
db.contacts.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "bot.contacts",
"name" : "_id_"
},
{
"v" : 1,
"key" : {
"name" : 1
},
"unique" : true,
"ns" : "bot.contacts",
"name" : "name_1",
"background" : true,
"safe" : null
}
]
What can I do to resolve this error?

You need to drop the old, non-sparse index in the shell so that Mongoose can recreate it with sparse: true the next time your app runs.
> db.contacts.dropIndex('name_1')

Related

Use arrayFilters in order to update a property of an object in nested array of objects, with Mongoose

I have a collection in MongoDb that has documents "flights" which contain a field array of objects. I want to update one property of one object at a time. In order to do so, I have to use two filters: One in order to select the document that I want to update, and a second one to select the object in the array.
I am using arrayFilters with Mongoose as follows:
This is my Flight shema
const mongoose = require('mongoose')
const Schema = mongoose.Schema
const flightSchema = new Schema({
flightName :{ type : String, required :true},
sits : {type : Array, required : true}, //[{n:1, d:f, s:f}]
origin : {type: String, required : true},
destination : {type : String, required: true},
departure : {type : Date, required : true},
arrival : {type : Date, required : true}
})
module.exports = mongoose.model('Flight', flightSchema)
// Models/Flight.js
{
flightName: a164651,
origin: Monterrey,
detination: Cancun,
sits: [{
sitNumber: 1,
isAvailable: true,
isSuspended: false
}, {
sitNumber: 2,
isAvailable: true,
isSuspended: false
}]
}
Lets imagine that I want to update the property IsSuspended from false to true in the object with sitNumber : 2.
//Controllers dashboard.js
blockSit : async (req, res) => {
try {
const flight = req.body.flightName
const sit = req.body.sitToBlock //sit es 2
const updateSit = await Flight.updateOne(
{ "flightName": flight},
{ "$set" : {"sits.$[si].isSuspended": true} },
{ "arrayFilters": [{ "si.sitNumber": sit} ]}
)
console.log(updateSit)
} catch (error) {
console.log(error)
}
}
As far as I can see my sintaxis is correct. However I keep receiving the following error message:
Error: Could not find path "sits.0.sitNumber" in schema
I do not have to use arrayfilters necesarily. I am open to try any other solution that allows me to update a property in a nested array of objects with mongoose.
It looks like your sits field is array of sub-documents but there is not a schema to describe the fields. Try defining the schema.
const sitSchema = new Schema({
sitNumber: Number,
isAvailable: Boolean,
isSuspended: Boolean,
// etc
});
// in the flight schema
sits: [sitSchema],
This is how I solved:
First I needed to use the method findOneandUpdate in Mongoose.
Also, I added to arrayFilters the property new and set it to true.
blockSit : async (req, res) =>{
try {
const flight = req.body.flightName
console.log(flight)
const sit = req.body.sit
console.log(sit)
const updateSit = await Flight.findOneAndUpdate(
{"flightName" : flight},
{ "$set" : {"sits.$[si].isSuspended" : true, "sits.$[si].isAvailable": false} },
{"arrayFilters" :[ { "si.sitNumber" :sit} ], new : true}
)
res.send(updateSit)
}
catch (error) {
console.log(error)
}
},

Why is MongoDB creating indexes for fields that don't exist within that schema?

TL;DR: I am having trouble with my mongo database seemingly creating indexes for my boards collection from fields from my users collection which is causing E11000 errors when I try to create a new board.
I am building a kanban board (like Jira) and have board, task and user collections (entity relationship diagram : https://imgur.com/a/Nu6Eg91). All of the collections work fine following a .dropIndexes(). However, when I have been working with tasks via the UI on one board, when I try to create another board I get this E11000 error:
{ MongoError: insertDocument :: caused by :: 11000 E11000 duplicate key error index: kanban.boards.$username_1 dup key: { : null }
at Function.create (/home/ubuntu/workspace/kanban v1.0/node_modules/mongodb-core/lib/error.js:43:12)
at toError (/home/ubuntu/workspace/kanban v1.0/node_modules/mongodb/lib/utils.js:149:22)
at coll.s.topology.insert (/home/ubuntu/workspace/kanban v1.0/node_modules/mongodb/lib/operations/collection_ops.js:859:39)
at /home/ubuntu/workspace/kanban v1.0/node_modules/mongodb-core/lib/connection/pool.js:532:18
at _combinedTickCallback (internal/process/next_tick.js:73:7)
at process._tickCallback (internal/process/next_tick.js:104:9)
driver: true,
name: 'MongoError',
index: 0,
code: 11000,
errmsg: 'insertDocument :: caused by :: 11000 E11000 duplicate key error index: kanban.boards.$username_1 dup key: { : null }' }
This appears to be because of a null value in the username field in the board schema but there is no such field in this model as can be seen below:
// Board Schema
// Config
var mongoose = require("mongoose"),
passportLocalMongoose = require("passport-local-mongoose");
// Create Board Schema
var BoardSchema = new mongoose.Schema({
user: {type: mongoose.Schema.Types.ObjectId, ref: "User", unique: true, sparse: true},
todo: [{type: mongoose.Schema.Types.ObjectId, ref: "Task"}],
inProgress: [{type: mongoose.Schema.Types.ObjectId, ref: "Task"}],
testing: [{type: mongoose.Schema.Types.ObjectId, ref: "Task"}],
completed: [{type: mongoose.Schema.Types.ObjectId, ref: "Task"}]
});
// Validation
BoardSchema.plugin(passportLocalMongoose);
// Export Model
module.exports = mongoose.model("Board", BoardSchema);
Here is an example board in the db:
{ "_id" : ObjectId("5c65767c8977670a2366ffe5"),
"todo" : [ ObjectId("5c657a1e451bc80ac315ef35"), ObjectId("5c657a4a451bc80ac315ef37"), ObjectId("5c657a76451bc80ac315ef39") ],
"inProgress" : [ ObjectId("5c6579e1451bc80ac315ef31"), ObjectId("5c657a03451bc80ac315ef33"), ObjectId("5c657a3a451bc80ac315ef36"), ObjectId("5c657a8a451bc80ac315ef3a") ],
"testing" : [ ObjectId("5c657a9a451bc80ac315ef3b") ],
"completed" : [ ObjectId("5c657a60451bc80ac315ef38"), ObjectId("5c65bb8f6f5b731aec27b4a7"), ObjectId("5c65bba06f5b731aec27b4a8"), ObjectId("5c657a11451bc80ac315ef34") ],
"user" : ObjectId("5c65767b8977670a2366ffe4"),
"__v" : 51 }
For some reason, I think a username index has been created by the database for the boards schema. The indexes for the boards collection are below:
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "kanban.boards"
},
{
"v" : 1,
"key" : {
"user" : 1
},
"name" : "user_1",
"ns" : "kanban.boards",
"background" : true
},
{
"v" : 1,
"unique" : true,
"key" : {
"username" : 1
},
"name" : "username_1",
"ns" : "kanban.boards",
"background" : true
}
]
For completeness, here is the user schema:
// User Schema
// Config
var mongoose = require("mongoose"),
uniqueValidator = require("mongoose-unique-validator"),
passportLocalMongoose = require("passport-local-mongoose");
// Create User Schema
var UserSchema = new mongoose.Schema({
username: {type: String, required: true, unique: true},
password: String
});
// Validation and Hash/Salt PW
UserSchema.plugin(uniqueValidator, {message: "A user is already registered with that {PATH}."});
UserSchema.plugin(passportLocalMongoose);
// Export Model
module.exports = mongoose.model("User", UserSchema);
The code causing the error is in the signup route, part of which is shown below:
// create new board for user
var newBoard = {
user: user._id,
todo: [],
inProgress: [],
testing: [],
review: [],
completed: [],
};
Board.create(newBoard, function(err, board) {
if(err) {
console.log(err);
req.flash("error", "Uh oh! Something went wrong.");
return res.redirect("/");
}
// authenticate user
passport.authenticate("local")(req, res, function() {
req.flash("success", "Welcome to your Kanban board, " + user.username + ".");
return res.redirect("/board");
});
});
I don't understand why this index is being created (or indeed if this is what is causing the E11000 errors but I'm pretty sure it is). Apologies for the spam of code, I am quite inexperienced with using mongo and so don't know what is relevant and what is not. I have built mongo databases with multiple collections before but not with a link collection as I have here so I can't work out what is going wrong. Please let me know if I have missed anything that is useful or important. Thanks.
I'm not 100% sure if this is the issue, but you should remove the unique: true and sparse: true from the BoardSchema.user parameter. Since you declare those in the UserSchema, you shouldn't also need them in the BoardSchema.
EDIT: You might also have to manually delete the index after completing this first step.
The problem was that I was plugging passport-local-mongoose into the board schema when that should only be on the user schema.

Mongoose unable to perform $near query on array of locations

I've tried looking for solutions and I've read parts of the documentation for the $near and $geoNear queries but I have not been successful.
var userSchema = new Schema({
zipcode : {
formatted : {type : String, required : false, default : ""},
geo : { type: [Number], default: [0,0]} // long, lat
}
});
userSchema.index({"zipcode.geo" : '2d'});
ON A DIFFERENT FILE
var locationSchema = new Schema({
formatted : {type : String, unique : false, required : true},
geo : { type: [Number], default: [0,0]}
});
locationSchema.index({geo: '2d'});
var storeSchema = new Schema({
locations : [locationSchema],
});
On my controller
let nearQuery = {
$near : [ user.zipcode.geo[0], user.zipcode.geo[1]],
spherical: true,
$maxDistance: 7000
}
Stores.find(locations : nearQuery).exec(function(error, doc) {
});
But i end up getting the error - "Error: Can't use spherical with Array."
When I remove
spherical : true
I get the error - " planner returned error: unable to find index for $geoNear query"
I know i am doing something wrong, I just cant figure out how to fix it. How can I fix this? or What is the best way of doing this?
Thank you in advance.
Turns out I was using the wrong syntax for the query and was trying to run the query on the locations field, while I should have done it on the locations.geo field.
https://docs.mongodb.com/manual/tutorial/query-a-2d-index/

Mongoose duplicate key error with upsert

I have problem with duplicate key.
Long time can`t find answer. Please help me solve this problem or explain why i get duplicate key error.
Trace: { [MongoError: E11000 duplicate key error collection: project.monitor index: _id_ dup key: { : 24392490 }]
name: 'MongoError',
message: 'E11000 duplicate key error collection: project.monitor index: _id_ dup key: { : 24392490 }',
driver: true,
index: 0,
code: 11000,
errmsg: 'E11000 duplicate key error collection: project.monitor index: _id_ dup key: { : 24392490 }' }
at /home/project/app/lib/monitor.js:67:12
at callback (/home/project/app/node_modules/mongoose/lib/query.js:2029:9)
at Immediate.<anonymous> (/home/project/app/node_modules/kareem/index.js:160:11)
at Immediate._onImmediate (/home/project/app/node_modules/mquery/lib/utils.js:137:16)
at processImmediate [as _immediateCallback] (timers.js:368:17)
but in monitor i use upsert, so why i get duplicate error??
monitor.js:62-70
monitor schema
var monitorSchema = db.Schema({
_id : {type: Number, default: utils.minute},
maxTicks : {type: Number, default: 0},
ticks : {type: Number, default: 0},
memory : {type: Number, default: 0},
cpu : {type: Number, default: 0},
reboot : {type: Number, default: 0},
streams : db.Schema.Types.Mixed
}, {
collection: 'monitor',
strict: false
});
index
monitorSchema.index({_id: -1});
Monitor = db.model('Monitor', monitorSchema);
and increase by property
exports.increase = function (property, incr) {
var update = {};
update[property] = utils.parseRound(incr) || 1;
Monitor.update({_id: utils.minute()}, {$inc: update}, {upsert: true}, function (err) {
if (err) {
console.trace(err);
}
});
};
utils.js
exports.minute = function () {
return Math.round(Date.now() / 60000);
};
exports.parseRound = function (num, round) {
if (isNaN(num)) return 0;
return Number(parseFloat(Number(num)).toFixed(round));
};
An upsert that results in a document insert is not a fully atomic operation. Think of the upsert as performing the following discrete steps:
Query for the identified document to upsert.
If the document exists, atomically update the existing document.
Else (the document doesn't exist), atomically insert a new document that incorporates the query fields and the update.
So steps 2 and 3 are each atomic, but another upsert could occur after step 1 so your code needs to check for the duplicate key error and then retry the upsert if that occurs. At that point you know the document with that _id exists so it will always succeed.
For example:
var minute = utils.minute();
Monitor.update({ _id: minute }, { $inc: update }, { upsert: true }, function(err) {
if (err) {
if (err.code === 11000) {
// Another upsert occurred during the upsert, try again. You could omit the
// upsert option here if you don't ever delete docs while this is running.
Monitor.update({ _id: minute }, { $inc: update }, { upsert: true },
function(err) {
if (err) {
console.trace(err);
}
});
}
else {
console.trace(err);
}
}
});
See here for the related documentation.
You may still be wondering why this can happen if the insert is atomic, but what that means is that no updates will occur on the inserted document until it is completely written, not that no other insert of a doc with the same _id can occur.
Also, you don't need to manually create an index on _id as all MongoDB collections have a unique index on _id regardless. So you can remove this line:
monitorSchema.index({_id: -1}); // Not needed

How to find a record using dot notation & update the value in a schema using mongoose

I am using mongoose to perform CRUD operation on my db. This is how my model looks.
var EmployeeSchema = new Schema({
name: String,
description: {
type: String,
default: 'No description'
},
department: [],
lastUpdated: {
type: Date,
default: Date.now
}
});
The department can contains array of object like this.
[
{
"id" : "55ba28f680dec4383eeebf97",
"text" : "Sales",
"topParentId" : "55ba28f680dec4383eeebf8b",
"topParentText" : "XYZ"
},
{
"id" : "55ba28f680dec4383eeebf98",
"text" : "IT",
"topParentId" : "55ba28f680dec4383eeebf8b",
"topParentText" : "XYZ"
},
{
"id" : "55ba28f680dec4383eeebf94",
"text" : "Marketing",
"topParentId" : "55ba28f680dec4383eeebccc",
"topParentText" : "ABC"
}
]
Now I need to find all the employee where department.id = '55ba28f680dec4383eeebf94' and then I need to update the text of the object.
Employee.find({'department.id': '55ba28f680dec4383eeebf94'}, function(err, Employees) {
_.each(Employees, function (emp) {
_.each(emp.department, function (dept) {
if(dept.id === '55ba28f680dec4383eeebf94'){
dept.text = 'XXXXX'; // How to update the employee to save the updated text
}
});
});
});
What is the right way to save the employee with updated text for that department?
Iterating is code is not a "sharp" way to do this. It is better to use the MongoDB update operators, especially since there is no schema defined for the array items here, so no rules to worry about:
Employee.update(
{'department.id': '55ba28f680dec4383eeebf94'},
{ "$set": { "department.$.text": "XXXXX" },
function(err,numAffected) {
// handling in here
}
);
The $set is the important part, otherwise you overwrite the whole object. As is the positional $ operator in the statement, so only the matched ( queried item in the array ) index is updated.
Also see .find**AndUpdate() variants for a way to return the modified object.
I think you can use the update model:
Employee.update({department.id: '55ba28f680dec4383eeebf94'}, {department.text: 'XXXXX'}, {multi: true},
function(err, num) {
console.log("updated "+num);
}
);
First object is the query, what to find: {department.id: '55ba28f680dec4383eeebf94'}, the second one is the update, what to update: {department.text: 'XXXXX'} and the third one is the options to pass to the update, multi means update every records you find: {multi: true}

Resources