I have a mongoose model defined like this.
const custSchema = new mongoose.Schema({
name: {
type: String,
es_indexed: true,
es_type: text
},
phoneNumber: {
type: String,
es_indexed: true,
es_type: String
},
email: String
})
custSchema.plugin(mongoosastic);
const Cust = module.exports = mongoose.model('Cust', custSchema);
Cust.createMapping(function(err, mapping) {
if(err) {
console.log(err)
} else {
console.log(mapping);
}
});
let count = 0;
const stream = Cust.synchronize();
stream.on('data', () => {
count = count + 1;
})
stream.on('close', () => {
console.log("Total " + count + " documents indexed");
})
stream.on('error', (err) => {
console.log(err)
});
When I add new Collection to Cust, new document does not get added to elasticsearch unless I restart the server.
How can I solve this issue?
Struggled with this for a while until I discovered the issue was related to the ElasticSearch upgrade from v5 to v6, where "type" became a candidate for deprecation (removed entirely in v7) and no longer allows for more than one type. The library most likely forces its own "type" ("_doc" in addition to "customer" likely in your case), but we need to manually set the one type in mongoosastic as "_doc" (which is necessary for ElasticSearch v6).
First, delete your original index where the index and type may be causing the conflict.
Then, in your model, change
custSchema.plugin(mongoosastic);
to
custSchema.plugin(mongoosastic, {
type: '_doc'
});
And create a new index after changing this option in the plugin section
stream.on("data", function(err, doc) {
client.indices.create({
index: 'customers',
body: {
doc
}
}, function (error, response) {
console.log(response);
});
Was a pain to resolve, hopefully this will help a few others avoid the headache as well.
Related
I have a schema nested inside another(main) schema. I'd like to increase a Number field in the first schema, however it is an array. So I'd need to access that specific object in that array, and increase a field inside it which is a number. I think what I am looking for is the $inc operator, however I couldn't seem to get it to work.
My schema's:
const chainSchema = new mongoose.Schema({
chainName: String,
streak: Number,
});
const userSchema = new mongoose.Schema({
email: String,
password: String,
googleId: String,
pomodoroStreak: Number,
chains: [chainSchema],
});
Post route:
app.post("/chainDisplay", function (req, res) {
const clickedChain = req.body.secret;
const clickedButton = req.body.submit;
if (clickedButton === "increase") {
Chain.findOneAndUpdate(
{ chainName: clickedChain },
{ $inc: { streak: 1 } },
function (err, foundChain) {
if (!err) {
res.redirect("/chain");
}
}
);
} else if (clickedButton === "decrease") {
Chain.findOneAndUpdate(
{ chainName: clickedChain },
{ $inc: { streak: -1 } },
function (err, foundChain) {
if (!err) {
res.redirect("/chain");
}
}
);
}
});
I obtain the chainName and trying to use that as a parameter to find that specific object, and increase or decrease the streak by 1. Thank you for your help in advance.
I've tried using the $inc operator alongside mongoose's findOneandUpdate method. I am wondering if I should be updating the User, rather than the Chain itself. I was expecting to increase or decrease the "streak" key by 1 when the relevant button is clicked.
So my knowledge of NodeJS and MongoDD are non-existent (just need to do a small code update for a friend) and I'm stuck.
Need to update a single document inside a collection via a unique id but can't seem to do it.
Here's the Model (I've trimmed it down and cut out all unnecessary data). I'm trying to update the field notes inside a transaction.
In short each entry in the given (an Agent) table will have a collection of multiple Transactions & Documents. I need to update a specific Transaction with the unique _id that is auto generated.
import { Schema, model } from 'mongoose';
interface Transaction {
first_name: string;
last_name: string;
type: string;
notes: string;
}
interface Agent {
org_id: number;
transactions: Array<Transaction>;
documents: Array<string>;
}
const transactionSchema = new Schema<Transaction>({
first_name: { type: String },
last_name: { type: String },
type: { type: String },
notes: String,
});
const transactionsSchema = new Schema<Agent>({
org_id: { type: Number },
transactions: [transactionSchema],
documents: [documentTypesSchema],
});
const AgentTransaction = model<Agent>(
'agent_transaction_table',
transactionsSchema
);
export default AgentTransaction;
Here's what I tried but didn't work (obviously), again I've trimmed out all unnecessary data. Just to clarify, the endpoint itself works, but the DB update does not.
import AgentTransaction from '../models/transaction'; // the above model
transaction.put('/notes', async (req, res) => {
const { org_id, transaction_id, notes } = req.body;
try {
const notesResult = await AgentTransaction.updateOne({
'transactions._id': transaction_id,
}, {
$set: {
'notes': notes
},
});
res
.status(200)
.json({ message: 'Updated', success: true, notesResult });
} catch (error) {
res.status(400).send(error);
}
});
So I figured it out. Maybe it'll help someone else as well.
const notesResult = await AgentTransaction.updateOne({
'transactions._id': { $in: [trunc2] },
}, {
$set: {
'transactions.$.notes': notes
},
});
The main issue was that the payload object needed to target the collection folder + the wildcard + the field, not just only the field.
NOTE: there's an edit at the bottom of the question:
Can I check the database for uniqueness using either a custom validator or a pre hook in a Mongoose.js model file. I am aware that I can check it in the controller, but I'd rather put it in the model file with the rest of the validators just for consistency.
I am also aware there is an npm package called mongoose-unique-validator that does this but I'm no fan of installing a library to do what should be one to five lines of code tops.
Mongoose also has a "unique" property that will throw an error if the item is not unique. But their documents clearly state this is not a validator. And the error it throws does not get routed the same as the validation errors.
Here is the relevant parts of the model file. This will check the db and if there is no dup then it creates the article but if there is a dup it throws an error but not a validation error which is what I want. If I simply return false if there is a dup it just ignores the validation and creates the duplicate article. This is no doubt related to Promises/Async. Here are the relevant Mongoose docs https://mongoosejs.com/docs/validation.html#async-custom-validators. And they talk about how the unique property is not a validator https://mongoosejs.com/docs/faq.html.
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const articleSchema = new Schema({
title: {
type: String,
required: [true, "Title is required"],
// unique: true,
// isAsync: true,
validate: {
validator: function(value) {
this.constructor.findOne({title: value}, (err, article) => {
if (err || !article) {
return true;
} else {
// return false;
throw new Error('Duplicate');
}
});
},
message: (props) => `Title "${props.value}" is already in use.`
},
},
content: { type: String, required: true }
});
EDIT: I figured this out, but it only works when creating a new article, not on updates. So the question is still open but the focus is on how to get it to work on updates. On update Mongoose does not treat "this" as the document object like it does on create. Instead "this" is the request object, and "this.constructor.findOne()" throws the error "this.constructor.findOne is not a function". Here's the revised validator:
title: {
type: String,
required: [true, "Title is required"],
isAsync: true,
validate: {
validator: async function(value) {
const article = await this.constructor.findOne({title: value});
if (article) {
throw new Error(`${value} is already in use.`);
}
}
}
}
Your validator function will only run the script and it not pass any callback or promise to mongoose, so mongoose assume that the validator return true and continue the process.
According to the document, you should return promise or use callback.
Promise:
validator: function(value) {
var here = this;
return new Promise(function(resolve, reject) {
here.constructor.findOne({title: value}, (err, article) => {
if (err || !article) {
resolve(true);
} else {
resolve(false);
}
});
})
}
Callback: (need to set isAsync: true)
validator: function(value, cb) {
this.constructor.findOne({title: value}, (err, article) => {
if (err || !article) {
cb(true);
} else {
cb(false, "Content is used");
}
});
}
I have been working with node.js and mongoose for sometime and I am hitting a wall. I have a database with 20,000 documents and when i search the database from the cli it works fine.
db.Tickets.find({ "Customers.Customer.CustomerID" : '123123123' })
This returns 256 results
Schema
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
// Define collection and schema for Ticket
var Ticket = new Schema({
UserName: {
type: String
},
Status: {
type: String
},
TicketNumber: {
type: Number
},
Name: {
type: String
},
Description: {
type: String
},
TicketTypeName: {
type: String
},
DueDate: {
type: Date
},
MapCollectDate : {
type: Date
},
NumberofUsersAffected : {
type: Number
},
DNNumber : {
type : String
},
RevisionDate : {
type : Date
},
CommercialImpact : {
type: String
},
Customers :[{
Customer: [{
CustomerID: Number,
CustomerName: String
}]
}],
Although if I test this in node.js using mongoose. I can't get it to return anything
I have a generic search that works
Ticket.find(function (err, tickets){
But can't get the specific search to work.
I am Connecting to Mongo
const config = require('./db');
//const Course = require('./models/Course');
//const CourseRoute = require('./routes/CourseRoute');
const Ticket = require('./models/Ticket');
const TicketRoute = require('./routes/TicketRoute');
const PORT = 4000;
mongoose.connect(config.DB).then(
() => {console.log('Connected to MongoDB') },
err => { console.log('Error connecting to MongoDB' +err)
});
Output of the log
Your node js server is running on PORT: 4000
Connected to MongoDB
Connected to MySQL
My Route End point
router.route('/').get(function (req, res) {
Ticket.find({ "Customers.Customer.CustomerID" : global.auth_username }, function(err, ticket) {
if(err){
console.log(err);
}
else {
res.json(tickets);
}
});
});
Also tried without the variable
router.route('/').get(function (req, res) {
Ticket.find({ "Customers.Customer.CustomerID" : "123123123" }, function(err, ticket) {
if(err){
console.log(err);
}
else {
res.json(tickets);
}
});
});
I had the same issue when I forgot to connect to Mongoose before running query
mongoose.connect(MONGO_URL, mongoOptions)
.then(() => {
// do your thing here
})
You had over a year to figured this out, and I am sure that you did so, but either way it seems that you have a typo in your code. The callback parameter is named ticket - function(err, ticket) {, whereas you are logging tickets - res.json(tickets);. In the generic test you correctly wrote tickets - Ticket.find(function (err, tickets){, which is probably why it worked.
The takeaway lesson here is - use debugging tools instead of logging, makes it easier to catch such problems.
Also, it would be appropriate to answer your own question once you've figured it out. But given that this is probably completely useless, you might as well delete it. Cheers!
I am new to NodeJs and MongoDB, i want to insert row with auto increment primary key 'id'. also defined a function called getNextSequence on mongo server.
this is working perfect on Mongodb server
> db.user.insert({
"id" : getNextSequence('user_id'),
"username" : "test",
"email" : "test#test.com",
"password" : "test123"
})
now i want to insert from NodeJs.I have tried this but not working
db.collection('user').insertOne({
id : "getNextSequence('user_id')",
username : query.name,
email: query.email,
password: query.pass
}, function(err, result) {
assert.equal(err, null);
console.log("row insterted ");
callback();
});
Assuming that getNextSequence is a server-script function (i.e. a method you defined and saved via db.system.js.save), it is not callable outside of the server. One way to go is to use eval, which forces the server to evaluate a string as a js code, even though it is not a good practice. Here is an example:
db.eval('getNextSequence(\'user_id\')', function(err, result) {
db.collection('users').insert({
"id" : result,
"username" : "test",
"email" : "test#test.com",
"password" : "test123"
});
});
Another way is to follow the mongo tutorial and to implement the getNextSequence directly in NodeJS. The syntax is pretty much the same:
function getNextSequence(db, name, callback) {
db.collection("counters").findAndModify( { _id: name }, null, { $inc: { seq: 1 } }, function(err, result){
if(err) callback(err, result);
callback(err, result.value.seq);
} );
}
You then use it in your nodeJS code like:
getNextSequence(db, "user_id", function(err, result){
if(!err){
db.collection('users').insert({
"_id": result,
// ...
});
}
});
Note: of course, you need to have set the counters collection as explained in the docs.
You can also use "mongoose-auto-increment".
The code has just 4 lines
var mongoose = require('mongoose');
var autoIncrement = require('mongoose-auto-increment');
autoIncrement.initialize(mongoose.connection);
userSchema.plugin(autoIncrement.plugin, 'user');
example :
npm i mongoose-auto-increment
connections.js :
const mongoose = require('mongoose');
require("dotenv").config;
const uri = process.env.MONGOURL;
mongoose.connect(uri, { useNewUrlParser: true }, (err) => {
if (!err) { console.log('MongoDB Connection Succeeded.') }
else { console.log('Error in DB connection : ' + err) }
});
require('../schema/userSchema');
userSchema.js :
var mongoose = require('mongoose'); // 1. require mongoose
var autoIncrement = require('mongoose-auto-increment'); // 2. require mongoose-auto-increment
var userSchema = new mongoose.Schema({
name: { type: String },
password: { type: String },
email: { type: String, unique: true, required: 'This field is required.' },
});
autoIncrement.initialize(mongoose.connection); // 3. initialize autoIncrement
userSchema.plugin(autoIncrement.plugin, 'user'); // 4. use autoIncrement
mongoose.model('user', userSchema);
To accomplish this, we will create a function that will keep trying to save the document untill it will have been saved with incremented _id
async function retryUntilSave(db, task) {
try {
const index = await db.collection('tasks').find().count() + 1;
const result = await db.collection('tasks').insertOne(Object.assign(task, { _id: index }))
} catch (error) {
if (error.message.includes("_id_ dup key")) {
console.log("ID already exists!")
console.log("Retrying...");
retryUntilSave(db, task)
} else {
console.log(error.message);
}
}
}
We can use task._id: index instead of Object.assign()
finally you can test this by making some concurrent requests
for (let index = 0; index < 20; index++) {
setTimeout(async () => {
await retryUntilSave(db, { title: "Some Task" })
}, 1000);
}
This function will handle easily if two or more tasks submitted at the same time because mogod throws error when we try to insert a document with duplicate _id, then we will retry saving the document again with incremented _id and this process will run until we save the document successfully !
You can also use "mongodb-autoincrement" module of node js. For example:
var autoIncrement = require("mongodb-autoincrement");
exports.yourMethod = function(newData, callback) {
autoIncrement.getNextSequence(db, your-collection-name, function (err, autoIndex) {
newData.id = autoIndex;
//save your code with this autogenerated id
});
}
You can use the below package on a model schema to auto-increment your collection field.
mongoose-auto-increment //you can download it from npm
Here I am not focusing on how to connect MongoDB. I just focus on how you can integrate auto increment in your model/collection/table.
const mongoose = require("mongoose"); //
const autoIncrement = require("mongoose-auto-increment");
const post_schema = new mongoose.Schema({
title: {
type: String,
required: true,
min: 3,
max: 225,
},
slug: {
type: String,
required: true,
},
});
autoIncrement.initialize(mongoose.connection);
post_schema.plugin(autoIncrement.plugin, {
model: "post", // collection or table name in which you want to apply auto increment
field: "_id", // field of model which you want to auto increment
startAt: 1, // start your auto increment value from 1
incrementBy: 1, // incremented by 1
});
module.exports = mongoose.model("post", post_schema);