Issue
I have a pouchdb-express server I am using for some tests for a CouchDB database.
I have the following database with an item like so:
{
_id: "12345",
email: 'u0#email.com',
companies: [{ id: 'company', uid: 'u0' }]
}
I then run the following command that works on my API hitting the CouchDB database but does not work when I try it on the PouchDB Express server.
.find({selector: { "companies": { "$elemMatch": {id: "company", uid: "u0"} } }})
I get the following error:
{ error: 'bad_request',
reason: 'unknown operator "0" - should be one of $eq, $lte, $lt, $gt, $gte, $exists, $ne, $in, $nin, $size, $mod, $regex, $elemMatch, $type, $allMatch or $all',
name: 'bad_request',
status: 400,
message: 'unknown operator "0" - should be one of $eq, $lte, $lt, $gt, $gte, $exists, $ne, $in, $nin, $size, $mod, $regex, $elemMatch, $type, $allMatch or $all' }
I also get the same exact error during the following query:
.find({
limit:9999999,
selector:{
$or: [
{$and: [{type:"something"},{uid:"u0"}] },
{$and: [{type:"document"}]}
]
}
})
I've also tried doing $eq almost exactly like the test suite does with still no dice.
Does anyone know why this would happen?
Info
Environment: Node.JS
Server: PouchDB Server
Here is my package.json
"pouchdb-find": "^6.4.3", // tried 7.0.0 and still have the issue
"pouchdb-node": "^6.4.3",
"pouchdb-server": "^4.1.0",
I have tinkered with this and discovered that for records like:
{
"borough": "Brooklyn",
"cuisine": "American ",
"marks": [50, 60, 45, 43],
"grades": [
{
"grade": "A",
"score": 5
},
{
"grade": "A",
"score": 7
},
{
"grade": "A",
"score": 12
},
{
"grade": "A",
"score": 12
}
],
"name": "Riviera Caterer"
},
This simple selector will return the correct results :
{ selector: { '$elemMatch': { '$gte': 0, '$lt': 30 } } }
and, while this "composite (?)" selector, ignores mismatches, and returns all rows!! ...
{ selector: 'grades': { '$elemMatch': { 'score': 14 } } }
... this one bombs with the error you indicate :
{ selector: 'grades': { '$elemMatch': { "grade": "B" } } }
I suspect the "pouchdb-find" version of $elemMatch can only handle simple arrays, not arrays of objects.
Seems like a PR is required :-(
Related
This query is returning the first object but it should not return. Because it has the BU but in different domain. Its doing fine in single objects in collaborators. When there is multiple Its not behaving as expected. How can we do this any suggestions?
My criteria is In the collaborator array
Only BU name or
Only Domain or
Both BU and Domain it should return.
In below situation first one has same domain <{"domain": "xyz.com"}> but still its not returning. Why?
[
{
name: "1",
collaborators: [
{
"domain": "xyz.com"
},
{
"buName": "Vignesh B"
},
{
"domain": "yz.com"
},
{
"domain": "xyz.com",
"buName": "Vignesh B"
}
]
},
{
name: "2",
collaborators: [
{
"domain": "xyz.com",
"buName": "Vignesh BU"
}
]
},
{
name: "3",
collaborators: [
{
"domain": "xyz.com"
}
]
},
{
name: "4",
collaborators: [
{
"buName": "Vignesh BU"
},
{
"domain": "xyz.com"
},
{
"domain": "xyz.com",
"buName": "Vignesh BU"
}
]
}
]
db.collection.find({
$or: [
{
"collaborators.domain": "xyz.com",
"collaborators.buName": {
"$exists": false
}
},
{
"collaborators.buName": "Vignesh BU",
"collaborators.domain": {
"$exists": false
}
},
{
"collaborators.buName": "Vignesh BU",
"collaborators.domain": "xyz.com"
}
]
})
It is not returning the first document because the buName values in this document are "Vignesh B" and not "Vignesh BU". Only add an U in Vignesh B and it works.
Link to mongodb playground
I think there was a comment at wone point that said that the name: "1" document was expected to return (as it matches the second "Only Domain" criteria) but it is not currently. This is because you will need to use the $elemMatch operator since you are querying an array with multiple conditions.
The query should look as follows, as demonstrated in this playground example (note that I've changed the name: 3 document so that it would not match):
db.collection.find({
$or: [
{
"collaborators": {
$elemMatch: {
"domain": "xyz.com",
"buName": {
"$exists": false
}
}
}
},
{
"collaborators": {
$elemMatch: {
"buName": "Vignesh BU",
"domain": {
"$exists": false
}
}
}
},
{
"collaborators": {
$elemMatch: {
"buName": "Vignesh BU",
"domain": "xyz.com"
}
}
}
]
})
Why is this change needed? It is because of the semantics of how querying an array works in MongoDB. When querying on multiple nested conditions without using $elemMatch you are telling the database that different entries in the array can each individually satisfy the requirements. As shown in this playground example, that means that when you run this query:
db.collection.find({
"arr.str": "abc",
"arr.int": 123
})
The following document will match:
{
_id: 1,
arr: [
{
str: "abc"
},
{
int: 123
}
]
}
This is because the first entry in the array satisfies one of the query predicates while the other entry in the array satisfies the second predicate. Changing the query to use $elemMatch changes the semantics to specify that a single entry in the array must successfully satisfy all query predicate conditions which prevents the document above from matching.
In your specific situation the same thing was happening with your first set of conditions of:
{
"collaborators.domain": "xyz.com",
"collaborators.buName": {
"$exists": false
}
}
The first array item in the name: "1" document was matching the collaborators.domain condition. The problem was the second condition. While that same first array entry did not have a buName field, two of the other entries in the array did. Since there is no $elemMatch present, the database checked those other entries, found that the buName existed there, and that caused the query predicates to fail to match and for the document to not get returned. Adding the $elemMatch forces both of those checks to happen against the single entry in the array hence resolving the issue.
Below is my code to display review array data which is part of the restaurant collection object:
async get(reviewId) {
const restaurantsCollection = await restaurants();
reviewId = ObjectId(reviewId)
const r = await restaurantsCollection.findOne(
{ reviews: { $elemMatch: { _id : reviewId } } },
{"projection" : { "reviews.$": true }}
)
return r
}
My object looks like:
{
_id: '6176e58679a981181d94dfaf',
name: 'The Blue Hotel',
location: 'Noon city, New York',
phoneNumber: '122-536-7890',
website: 'http://www.bluehotel.com',
priceRange: '$$$',
cuisines: [ 'Mexican', 'Italian' ],
overallRating: 0,
serviceOptions: { dineIn: true, takeOut: true, delivery: true },
reviews: []
}
My output looks like:
{
"_id": "6174cfb953edbe9dc5054f99", // restaurant Id
"reviews": [
{
"_id": "6176df77d4639898b0c155f0", // review Id
"title": "This place was great!",
"reviewer": "scaredycat",
"rating": 5,
"dateOfReview": "10/13/2021",
"review": "This place was great! the staff is top notch and the food was delicious! They really know how to treat their customers"
}
]
}
What I want as output:
{
"_id": "6176df77d4639898b0c155f0",
"title": "This place was great!",
"reviewer": "scaredycat",
"rating": 5,
"dateOfReview": "10/13/2021",
"review": "This place was great! the staff is top notch and the food was delicious! They really know how to treat their customers"
}
How can I get the output as only the review without getting the restaurant ID or the whole object?
So the query operators, find and findOne do not allow "advanced" restructure of data.
So you have 2 alternatives:
The more common approach will be to do this in code, usually people either use some thing mongoose post trigger or have some kind of "shared" function that handles all of these transformations, this is how you avoid code duplication.
Use the aggregation framework, like so:
const r = await restaurantsCollection.aggregate([
{
$match: { reviews: { $elemMatch: { _id : reviewId } } },
},
{
$replaceRoot: {
newRoot: {
$arrayElemAt: [
{
$filter: {
input: "$reviews",
as: "review",
cond: {$eq: ["$$review._id", reviewId]}
}
},
0
]
}
}
}
])
return r[0]
I am trying to query an embedded subdocument and then only return an array in that subdocument via projection. After a query you can select fields that you want returned via projection. I want to use the native functionality because it is possible and the most clean way. The problem is it returns arrays in two documents.
I tried different query and projection options, but no result.
User model
// Define station schema
const stationSchema = new mongoose.Schema({
mac: String,
stationName: String,
syncReadings: Boolean,
temperature: Array,
humidity: Array,
measures: [{
date: Date,
temperature: Number,
humidity: Number
}],
lastUpdated: Date
});
// Define user schema
var userSchema = mongoose.Schema({
apiKey: String,
stations : [stationSchema]
}, {
usePushEach: true
}
);
api call
app.get('/api/stations/:stationName/measures',function(req, res, next) {
var user = {
apiKey: req.user.apiKey
}
const query = {
apiKey: user.apiKey,
'stations.stationName': req.params.stationName
}
const options = {
'stations.$.measures': 1,
}
User.findOne(query, options)
.exec()
.then(stations => {
res.status(200).send(stations)
})
.catch(err => {
console.log(err);
res.status(400).send(err);
})
});
Expected result
{
"_id": "5c39c99356bbf002fb092ce9",
"stations": [
{
"stationName": "livingroom",
"measures": [
{
"humidity": 60,
"temperature": 20,
"date": "2019-01-12T22:49:45.468Z",
"_id": "5c3a6f09fd357611f8d078a0"
},
{
"humidity": 60,
"temperature": 20,
"date": "2019-01-12T22:49:46.500Z",
"_id": "5c3a6f0afd357611f8d078a1"
},
{
"humidity": 60,
"temperature": 20,
"date": "2019-01-12T22:49:47.041Z",
"_id": "5c3a6f0bfd357611f8d078a2"
}
]
}
]
}
Actual result
{
"_id": "5c39c99356bbf002fb092ce9",
"stations": [
{
"stationName": "livingroom",
"measures": [
{
"humidity": 60,
"temperature": 20,
"date": "2019-01-12T22:49:45.468Z",
"_id": "5c3a6f09fd357611f8d078a0"
},
{
"humidity": 60,
"temperature": 20,
"date": "2019-01-12T22:49:46.500Z",
"_id": "5c3a6f0afd357611f8d078a1"
},
{
"humidity": 60,
"temperature": 20,
"date": "2019-01-12T22:49:47.041Z",
"_id": "5c3a6f0bfd357611f8d078a2"
}
]
},
******************************************************
// this whole object should not be returned
{
"stationName": "office",
"measures": []
}
******************************************************
]
}
edit
The answer below with aggregation works, but I still find it odd that I would need so much code. If after my normal query I get the same result with ".stations[0].measures", instead of the whole aggregation pipeline:
.then(stations => {
res.status(200).send(stations.stations[0].measures)
})
The way I read the code, the above does exactly the same as:
const options = {'stations.$.measures': 1}
Where the dollar sign puts in the index 0 as that was the index of the station that matches the query part: stationName: "livingroom"
Can someone explain?
This is not described in terms of mongoose but this will find a particular station name in an array of stations in 1 or more docs and return only the measures array:
db.foo.aggregate([
// First, find the docs we are looking for:
{$match: {"stations.stationName": "livingroom"}}
// Got the doc; now need to fish out ONLY the desired station. The filter will
// will return an array so use arrayElemAt 0 to extract the object at offset 0.
// Call this intermediate qqq:
,{$project: { qqq:
{$arrayElemAt: [
{ $filter: {
input: "$stations",
as: "z",
cond: { $eq: [ "$$z.stationName", "livingroom" ] }
}}, 0]
}
}}
// Lastly, just project measures and not _id from this object:
,{$project: { _id:0, measures: "$qqq.measures" }}
]);
$elemMatch operator limits the contents of an array field from the query results to contain only the first element matching the $elemMatch condition.
Try $elemMatch in Select Query as below :
const query = {
apiKey: user.apiKey,
'stations.stationName': req.params.stationName
}
const options = {
'stations' : {$elemMatch: { 'stationName' : req.params.stationName }}
}
I have documents that looks something like that, with a unique index on bars.name:
{ name: 'foo', bars: [ { name: 'qux', somefield: 1 } ] }
. I want to either update the sub-document where { name: 'foo', 'bars.name': 'qux' } and $set: { 'bars.$.somefield': 2 }, or create a new sub-document with { name: 'qux', somefield: 2 } under { name: 'foo' }.
Is it possible to do this using a single query with upsert, or will I have to issue two separate ones?
Related: 'upsert' in an embedded document (suggests to change the schema to have the sub-document identifier as the key, but this is from two years ago and I'm wondering if there are better solutions now.)
No there isn't really a better solution to this, so perhaps with an explanation.
Suppose you have a document in place that has the structure as you show:
{
"name": "foo",
"bars": [{
"name": "qux",
"somefield": 1
}]
}
If you do an update like this
db.foo.update(
{ "name": "foo", "bars.name": "qux" },
{ "$set": { "bars.$.somefield": 2 } },
{ "upsert": true }
)
Then all is fine because matching document was found. But if you change the value of "bars.name":
db.foo.update(
{ "name": "foo", "bars.name": "xyz" },
{ "$set": { "bars.$.somefield": 2 } },
{ "upsert": true }
)
Then you will get a failure. The only thing that has really changed here is that in MongoDB 2.6 and above the error is a little more succinct:
WriteResult({
"nMatched" : 0,
"nUpserted" : 0,
"nModified" : 0,
"writeError" : {
"code" : 16836,
"errmsg" : "The positional operator did not find the match needed from the query. Unexpanded update: bars.$.somefield"
}
})
That is better in some ways, but you really do not want to "upsert" anyway. What you want to do is add the element to the array where the "name" does not currently exist.
So what you really want is the "result" from the update attempt without the "upsert" flag to see if any documents were affected:
db.foo.update(
{ "name": "foo", "bars.name": "xyz" },
{ "$set": { "bars.$.somefield": 2 } }
)
Yielding in response:
WriteResult({ "nMatched" : 0, "nUpserted" : 0, "nModified" : 0 })
So when the modified documents are 0 then you know you want to issue the following update:
db.foo.update(
{ "name": "foo" },
{ "$push": { "bars": {
"name": "xyz",
"somefield": 2
}}
)
There really is no other way to do exactly what you want. As the additions to the array are not strictly a "set" type of operation, you cannot use $addToSet combined with the "bulk update" functionality there, so that you can "cascade" your update requests.
In this case it seems like you need to check the result, or otherwise accept reading the whole document and checking whether to update or insert a new array element in code.
if you dont mind changing the schema a bit and having a structure like so:
{ "name": "foo", "bars": { "qux": { "somefield": 1 },
"xyz": { "somefield": 2 },
}
}
You can perform your operations in one go.
Reiterating 'upsert' in an embedded document for completeness
I was digging for the same feature, and found that in version 4.2 or above, MongoDB provides a new feature called Update with aggregation pipeline.
This feature, if used with some other techniques, makes possible to achieve an upsert subdocument operation with a single query.
It's a very verbose query, but I believe if you know that you won't have too many records on the subCollection, it's viable. Here's an example on how to achieve this:
const documentQuery = { _id: '123' }
const subDocumentToUpsert = { name: 'xyz', id: '1' }
collection.update(documentQuery, [
{
$set: {
sub_documents: {
$cond: {
if: { $not: ['$sub_documents'] },
then: [subDocumentToUpsert],
else: {
$cond: {
if: { $in: [subDocumentToUpsert.id, '$sub_documents.id'] },
then: {
$map: {
input: '$sub_documents',
as: 'sub_document',
in: {
$cond: {
if: { $eq: ['$$sub_document.id', subDocumentToUpsert.id] },
then: subDocumentToUpsert,
else: '$$sub_document',
},
},
},
},
else: { $concatArrays: ['$sub_documents', [subDocumentToUpsert]] },
},
},
},
},
},
},
])
There's a way to do it in two queries - but it will still work in a bulkWrite.
This is relevant because in my case not being able to batch it is the biggest hangup. With this solution, you don't need to collect the result of the first query, which allows you to do bulk operations if you need to.
Here are the two successive queries to run for your example:
// Update subdocument if existing
collection.updateMany({
name: 'foo', 'bars.name': 'qux'
}, {
$set: {
'bars.$.somefield': 2
}
})
// Insert subdocument otherwise
collection.updateMany({
name: 'foo', $not: {'bars.name': 'qux' }
}, {
$push: {
bars: {
somefield: 2, name: 'qux'
}
}
})
This also has the added benefit of not having corrupted data / race conditions if multiple applications are writing to the database concurrently. You won't risk ending up with two bars: {somefield: 2, name: 'qux'} subdocuments in your document if two applications run the same queries at the same time.
I am using Node.js and MongoDB and I'm trying to setup a DELETE route. In the function responsible for handling the delete I am using Mongo's "$pull" operator. I've looked at a couple of examples now and I don't know what I am doing wrong.
Here's a sample of how the database documents are setup
{
"_id": {
"$oid": "123abc"
},
"sleepData": [
{
"date": "03/28/2016",
"hour": "11",
"minute": "11",
"meridiem": "PM",
"feeling": "7"
},
{
"date": "03/29/2016",
"hour": "3",
"minute": "41",
"meridiem": "PM",
"feeling": "1"
},
{
"date": "03/30/2016",
"hour": "1",
"minute": "29",
"meridiem": "AM",
"feeling": "5"
},
{
"date": "03/30/2016",
"hour": "1",
"minute": "38",
"meridiem": "AM",
"feeling": "4"
},
]
}
*Note the near-duplicate data, thus the reason why my $pull query is so specific.
Here is my function for the route
module.exports.DELETE = function(req, res) {
var sleepDataToDelete = {
date: req.query.date,
hour: req.query.hour,
minute: req.query.minute,
meridiem: req.query.meridiem,
feeling: req.query.feeling
};
// next code block is what this console prints out
console.log("Deleting req.query:\n", sleepDataToDelete);
var sleepObjectId = req.query.sleepObjectId;
var sleepDataCollection = db.get().collection('sleepData');
sleepDataCollection.update(
{
_id: sleepObjectId
},
{
$pull: {
sleepData: {
date: sleepDataToDelete.date,
hour: sleepDataToDelete.hour,
minute: sleepDataToDelete.minute,
meridiem: sleepDataToDelete.meridiem,
feeling: sleepDataToDelete.feeling
}
}
},
function(err, result) {
if(err) {
console.log("err", err);
return res.status(400).end();
} else {
console.log("Count: ", result.result.n);
console.log("Deleted! :) ");
return res.status(200).end();
}
}
);
};
This is what the console.log("Deleting req.query:\n", sleepDataToDelete); prints out, which also matches the third index in the sleepData array.
Deleting req.query:
{
date: '03/30/2016',
hour: '1',
minute: '29',
meridiem: 'AM',
feeling: '5'
}
I have even tried putting the json field names in double/single quotes, but that didn't work either. The number of objects modified is 0. I have also tried reducing the "$pull {...}" query to just "date" instead of having "date", "hour", "minute", "meridiem", and "feeling." This still results in 0 modified items from the print statement.
As #BlakesSeven pointed out, I was not passing in an ObjectId in my query. So, credit goes to him. Needless to say, this solved my issue.