I want to make a query on mongo collection and when the collection changes I want to run the query again. But to optimize it, I don't want to run the query on every collection change, only when documents matching the query changes. I have following code:
const query = { author: someUserID };
const fetch = async () => await collection.find(query).toArray();
const watcher = collection
.watch([{ $match: { fullDocument: query } }])
.on("change", () => fetch().then(sendData)); // This does not work
fetch().then(sendData); // This works
On first run it fetches documents and executes sendData but when new document is inserted, the event is not triggered. When I wun collection.watch() without arguments, it works.
Where is the problem?
Thanks.
EDIT: I want to be able to reuse the query for .find() and for .watch().
The $match stage in that example is essentially
{$match: { fullDocument: { author: someUserId }}}
That will match only if the fullDocument is exactly { author: someUserId } with no other fields or values.
In order to match the author while permitting other fields in the document, use dotted notation, like
const query = { "fullDocument.author": someUserId };
and match like:
{$match: query }
Related
I want to search for records using the below-mentioned Field and code I have used, returning nothing. I'm sure I'm querying wrongly, and can anyone help me build the correct query code?
Code I have used:
const { id } = req.params;
const requests = await Request.find({
itemsList: { item: id },
});
Return:
console.log(requests);
requests: []
const requests = await Request.find({
"itemsList.item": id,
});
will return the entire array if there is a match.
Playground
And make sure your data on the query part is matching the type.
In the image, you have a string. Use ObjectId if it is ObjectId.
Please try this
const requests = await Request.find({
itemsList: { $elemMatch: { item: mongoose.Types.ObjectId(id) }}},
});
I'm using Mongoose in NodeJS to control a MongoDB database.
I'm creating an API and for obvious security reasons, I want to prevent the auto generated document _id field from getting replaced by a manually generated one in the API request.
Schema:
{ name: String }
Creating a document:
const record = {
_id: '5e35517cc894c90327a34baf'
name: 'bob'
}
const insertRecords = async () => {
await Quiz.create(record);
};
insertRecords();
Results in the following document:
{
_id: '5e35517cc894c90327a34baf'
name: 'bob'
}
As can be seen, the _id supplied in the query, as long as it's a valid ObjectID, would replace the _id that was supposed to be auto generated by mongo.
Is there a way to check if this _id field is in the query so that I can reject the API request? The .create method triggers the pre save middleware hook which would always have the _id of the final document so I cannot depend on it to know whether the _id was in the query or it's the auto generated one.
The only option I found is to disable the _id field altogether but this does not make sense.
Solution #1 - Use .create() method with an explicit object.
It's actually easier than you think. This is self-explanatory - we only define what we want to allow. Mongoose will ignore anything that's not in the object.
const record = {
_id: '5e35517cc894c90327a34baf'
name: 'bob'
}
const insertRecords = async () => {
await Quiz.create({
name: record.name // only allow names.
});
};
insertRecords();
Solution #2 - Define a function to clear unwanted objects.
You can define a helper function to clear out unwanted fields.
const filterObj = (obj, ...allowedFields) => {
const newObject = {};
// If the current field is one of the allowed fields, keep them in the new object.
Object.keys(obj).forEach((el) => {
if (allowedFields.includes(el)) {
newObject[el] = obj[el];
}
});
return newObject;
};
How to use:
const filteredRecord = filterObj(record, 'name'); // arbitrary list of allowed fields. In this case, we'll only allow 'name'.
await Quiz.create(filteredRecord);
This below is an imaginary syntax i wish it was availabe so i could fetch my sessions with .
i have a Course collection that every single course has a sessions array as sub-document
i want to fetch courses based on my users query params when they search, so that if somebody asked for sessions from a specific teacher i use that in my $match stage in aggregate if not just find all sessions from all teacher , the same applies for subject and status if they get defined use them if not ignore them and find all possible.
there is another procedure if there was a keyword in mongo or aggregate that i could use and by that it knew that it find all something like * or regex syntax
i also come across to a $cond keyword but did not match my expectations in here .
app.get('/sessions',(req, res)=>{
const teacher = req.query.teacherId
const subject = req.query.subjectId
const status = req.query.statusId
Course.aggregate([
if(teacher) {$match:{teacher:teacher}},
if(subject) {$match:{subject:subject}},
if(status) {$match:{status:status}}
]).then(sessions=>res.json(sessions))
.catch(err=>res.json(err))
})
not sure if this helps but this is my Course Schema
const CourseSchema = new mongoose.Schema({
name:{type:String,required:true},
status:{type:String,required:true},
subject:{
type:mongoose.Schema.Types.ObjectId,
ref :'subjects'
},
teacher:{
type:mongoose.Schema.Types.ObjectId,
ref :'users'
},
sessions:[
{
Date:{type:Date,required:true},
timeStart:{type:String,required:true},
timeEnd :{type:String,required:true},
students :[{
type:mongoose.Schema.Types.ObjectId,
ref :'users'
}]
}
]
})
Thank You in advance .
I believe you are trying to achieve something like this:
app.get('/sessions', (req, res) => {
const teacher = req.query.teacherId
const subject = req.query.subjectId
const status = req.query.statusId
const query = [];
if (teacher) query.push({ $match: { teacher: teacher } });
if (subject) query.push({ $match: { subject: subject } });
if (status) query.push({ $match: { status: status } });
Course.aggregate(query).then(sessions => res.json(sessions))
.catch(err => res.json(err));
})
REMEMBER, I did not check if the query is okay.
Using: node/express/mongodb/mongoose
With the setup listed above, I have created my schema and model and can query as needed. What I'm wondering how to do though is, pass the express request.query object to Model.find() in mongoose to match and query the _id of a nested document. In this instance, the query may look something like:
http://domain.com/api/object._id=57902aeec07ffa2290f179fe
Where object is a nested object that exists elsewhere in the database. I can easily query other fields. _id is the only one giving an issue. It returns an empty array of matches.
Can this be done?
This is an example and not the ACTUAL schema but this gets the point across..
let Category = mongoose.Schema({
name: String
})
let Product = mongoose.Schema({
name: String,
description:String,
category:Category
})
// sample category..
{
_id:ObjectId("1234567890"),
name: 'Sample Category'
}
// sample product
{
_id:ObjectId("0987654321"),
name:'Sample Product',
description:'Sample Product Description',
category: {
_id:ObjectId("1234567890"),
name: 'Sample Category'
}
}
So, what I'm looking for is... if I have the following in express..
app.get('/products',function(req,res,next){
let query = req.query
ProductModel.find(query).exec(function(err,docs){
res.json(docs)
})
})
This would allow me to specify anything I want in the query parameters as a query. So I could..
http://domain.com/api/products?name=String
http://domain.com/api/products?description=String
http://domain.com/api/products?category.name=String
I can query by category.name like this, but I can't do:
http://domain.com/api/products?category._id=1234567890
This returns an empty array
Change your query to http://domain.com/api/object/57902aeec07ffa2290f179fe and try
app.get('/api/object/:_id', function(req, res) {
// req._id is Mongo Document Id
// change MyModel to your model name
MyModel.findOne( {'_id' : req._id }, function(err, doc){
// do smth with this document
console.log(doc);
});
});
or try this one
http://domain.com/api/object?id=57902aeec07ffa2290f179fe
app.get('/api/object', function(req, res) {
var id = req.param('id');
MyModel.findOne( {'_id' : id }, function(err, doc){
console.log(doc);
});
})
First of all increase your skills in getting URL and POST Parameters by this article.
Read official Express 4.x API Documentation
Never mind I feel ridiculous. It works just as I posted above.. after I fixed an error in my schema.
Due to a management decision, we are using userId for the users collection, postId for the posts collection, and topicId for the topics collection, instead of '_id' for each collection as the unique identifier.
This causes a few problems getting started - one of the problems I have encountered is with upserts -
Using Mongoose, we have a schema that restricts userId to be a unique value - but when doing an update on a user model, with upsert set to true, MongoDB appears to only look at the ObjectIds of a collection to see if the same one exists - it doesn't check to see if a model already exists with the same userId - therefore Mongo does an insert instead of an update.
let me illustrate this with some data:
let's say the user's collection has one document:
{
_id:'561b0fad638e99481ab6d84a'
userId:3,
name:'foo'
}
we then run:
User.update({userId:3},{"$set":{name:'bar'},{upsert:true},function(err,resp){
if(err){
// "errMessage": "insertDocument :: caused by :: 11000 E11000 duplicate key error index: app42153482.users.$userId_1 dup key: { : 3 }",
}
});
one would think that MongoDB would find the existing document with userId:3 and udpate it, so there must be something I am doing wrong since it's giving me the duplicate key error?
Typically the default value ObjectId is more ideal for the _id. Here, in this situation you can either override the default _id or you can have your own field for id(like userId in your case).
Use a separate counters collection to track the last number sequence used. The _id field contains the sequence name and the seq field contains the last value of the sequence.
Insert into the counters collection, the initial value for the userid:
db.counters.insert( {
_id: "userid",
seq: 0 } )
Create a getNextSequence function that accepts a name of the sequence. The function uses the findAndModify() method to atomically increment the seq value and return this new value:
function getNextSequence(name) {
var ret = db.counters.findAndModify(
{
query: { _id: name },
update: { $inc: { seq: 1 } },
new: true
}
);
return ret.seq;
}
Use this getNextSequence() function during insert().
db.users.insert(
{
_id: getNextSequence("userid"),
name: "Sarah C."
}
)
db.users.insert(
{
_id: getNextSequence("userid"),
name: "Bob D."
}
)
This way you can maintain as many sequences as you want in the same counter collection. For the upsert issue, check out the Optimistic Loop block in this link Create an auto-increment sequence field.
The second approach is to use a mongoose middleware like mongodb-autoincrement.
Hope it helps.
I don't know which versions of MongoDB and Mongoose you are using, but I couldn't reproduce your problem with MongoDB 3.0 and Mongoose 4.1.10.
I made a sample for you which will create and save a new user, update (using upsert) it, and create another one through an upsert. Try running this code:
"use strict";
var mongoose=require("mongoose");
var Schema = require('mongoose').Schema;
var ObjectId = mongoose.Schema.Types.ObjectId;
// Connect to test
mongoose.connect("mongodb://localhost:27017/test");
// Lets create your schema
var userSchema = new Schema({
_id: ObjectId,
userId: {type: Number, unique: true },
name: String
});
var User = mongoose.model("User", userSchema, "Users");
User.remove() // Let's prune our collection to start clean
.then( function() {
// Create our sample record
var myUser = new User({
_id:'561b0fad638e99481ab6d84a',
userId:3,
name:'foo'
});
return myUser.save();
})
.then( function() {
// Now its time to update (upsert userId 3)
return User.update({userId:3},{"$set":{name:'bar'}},{upsert:true});
})
.then( function() {
// Now its time to insert (upsert userId 4)
return User.update({userId:4},{"$set":{name:'bee'}},{upsert:true});
})
.then( function() {
// Lets show what we have inserted
return User.find().then(function(data) {console.log(data)});
})
.catch( function(err) {
// Show errors if anything goes wrong
console.error("ERROR", err);
})
.then( function() {
mongoose.disconnect();
});
Following the documentation (of MongoDB 3.0) upsert:true will only not insert a non-existing document if your query conditions match on the _id field.
See: https://docs.mongodb.org/manual/reference/method/db.collection.update/#mongodb30-upsert-id
Why are you not using the user_name for a user as unique id?
Because auto-incrementing fields as ids are a bad practice to use in a mongodb environment, especially if you want to use sharding
=> all your inserts will occur on the latest shard
=> the mongodb cluster will have to rebalance often / redistribute the data around.
(Currently this will not occur on your system as you still use the generated _id field)
You can off course also create a unique index on the user_id field:
https://docs.mongodb.org/manual/core/index-unique/#index-type-unique