I have problems putting two $OR operations in one query statement. It ignores the earlier $OR operators and only takes into consideration the last $OR operator in the query. (There is no issue when only one $OR operator is in the query.) Am wondering if I am doing something wrong, if this is possible to achieve using CouchDB, or if there is a way around it. Thank you!
I am running a blockchain on Hyperledger Fabric using CouchDB as the state database. (https://hyperledger-fabric.readthedocs.io/en/release-1.4/couchdb_tutorial.html) I am definitely not seasoned in the workings of CouchDB so I may be slightly ignorant in how it's supposed to behave.
It's sort of complicated but my objects basically "belong" to two owners and companies. I want to perform queries such that when I search for a owner or a company, it searches in two different columns to see the the owner/company exists.
e.g. When I search for Company A, it should search for Company A in both company1_id and company2_id.
Query Statement:
{"selector":{"docType":"object", "owner_id": {"$in": ["owner_id"]},
"$or": [{"company1_id": { "$in": ["company_id_1", "company_id_2"]}},
{"company2_id": { "$in": ["company_id_1", "company_id_2"]}}], "$or":
[{"owner1_id": { "$in": ["owner_id_1", "owner_id_2"]}}, {"owner2_id":
{ "$in": ["owner_id_1", "owner_id_2"]}}],
"object_id":{"$lt":"99999999999"}}, "sort": [{"object_id": "desc"}]}
Expected: Get results that corresponds to the above query
What happened: I get results which ignored the first query, so it returns results which corresponds to the following query:
{"selector":{"docType":"object", "owner_id": {"$in": ["owner_id"]},
"$or": [{"owner1_id": { "$in": ["owner_id_1", "owner_id_2"]}},
{"owner2_id": { "$in": ["owner_id_1", "owner_id_2"]}}],
"object_id":{"$lt":"99999999999"}}, "sort": [{"object_id": "desc"}]}
Yes, it ignores it because it's a standard JSON object, so each key must be unique. You have the same situation with a simpler example:
{
"foo": 123,
"foo": 345,
}
Depending on whether your JSON parser accepts the first or last value, your resulting object will be either { "foo": 123 } or { "foo": 345 }.
The solution is to use another logical layer. In your case, you probably want an $and wrapping your $or conditions:
"$and": [
{ "$or": [ ... ] },
{ "$or": [ ... ] },
]
Related
I have a mongoose schema that is structured like this:
Schema E = {
_id,
... some fields,
details: [
{
...somefields,
people: [ObjectIds]
}
]
}
First, I have an aggregate query where I am using $geoNear then $match, and then $facet.
After the operations the document that I get is as follows:
estates: [
{
_id,
... some fields,
details: [
{
...somefields,
people: [ObjectIds]
}
],
... other fields
},
... more estate objects
]
],
page: [...some objects]
I have an array called approved which has some object Ids.
I want to filter the page array inside events.details while keeping the rest of the fields intact.
The result I want is as follows:
NOTE: *The field filteredPeople is the array I want after filtering people with approved.
estate: [
{
_id,
... some fields,
details: [
{
...somefields,
filteredPeople: [ObjectIds],
numberOfPeople: Size of people array
}
],
... other fields
},
... more estate objectes
],
page: [...some objects]
This is what I tried doing:
{
"estates": {
"$map": {
"input": "$estates",
"as": "estate",
"in": {
"details": {
"$map": {
"input": "$$estate.details",
"as": "detail",
"in": {
"filteredPeople": {
"$filter": {
"input": "$$detail.people",
"as": "people",
"cond": { "$in": ["$$people", approved] }
}
}
}
}
}
}
}
},
}
But this erases the other fields. The other way is to create a separate field called estatePeople where the result of the $addFields will be stored.
I could then try to merge the two arrays. But I don't have any field to match them as the second estatePoeple array will not have anything but the filteredPeople. So I will then somehow have to merge the two arrays just by the index of the array and where they appear.
Can someone please help me out on how to get the desired document with relatively good performance?
For anyone who has the same problem:
In the end, I was unable to find any way to execute the query that I wanted with reasonable performance.
This schema design is not the optimal way to execute such complicated queries. What I ended up doing was making the details array an object and have separate documents for separate details. And then I made a parent schema that kept reference of the details for the same estate.
You can use reverse referencing or referencing according to the queries that you want to execute.
Is there a way to specify a heterogeneous array as a schema property where it can contain both ObjectIds and strings? I'd like to have something like the following:
var GameSchema = new mongoose.schema({
players: {
type: [<UserModel reference|IP address/socket ID/what have you>]
}
Is the only option a Mixed type that I manage myself? I've run across discriminators, which look somewhat promising, but it looks like it only works for subdocuments and not references to other schemas. Of course, I could just have a UserModel reference and create a UserModel that just stores the IP address or whatever I'm using to identify them, but that seems like it could quickly get hugely out of control in terms of space (having a model for every IP I come across sounds bad).
EDIT:
Example:
A game has one logged in user, three anonymous users, the document should look something like this:
{ players: [ ObjectId("5fd88ea85...."), "192.0.0.1", "192.1.1.1", "192.2.2.1"] }
Ideally this would be populated to:
{ players: [ UserModel(id: ..., name: ...), "192.0.0.1", "192.1.1.1", "192.2.2.1"] }
EDIT:
I've decided to go a different route: instead of mixing types, I'm differentiating with different properties. Something like this:
players: [
{
user: <object reference>,
sessionID: <string>,
color: {
type: String
},
...other properties...
}
]
I have a validator that ensures only one of user or sessionID are populated for a given entry. In some ways this is more complex, but it does obviate the need to do this kind of conditional populating and figuring out what type each entry is when iterating over them. I haven't tried any of the answers, but they look promising.
If you are content to go with using Mixed or at least some scheme that will not work with .populate() then you can shift the "join" responsibility to the "server" instead using the $lookup functionality of MongoDB and a little fancy matching.
For me if I have a "games" collection document like this:
{
"_id" : ObjectId("5933723c886d193061b99459"),
"players" : [
ObjectId("5933723c886d193061b99458"),
"10.1.1.1",
"10.1.1.2"
],
"__v" : 0
}
Then I send the statement to the server to "join" with the "users" collection data where an ObjectId is present like this:
Game.aggregate([
{ "$addFields": {
"users": {
"$filter": {
"input": "$players",
"as": "p",
"cond": { "$gt": [ "$$p", {} ] }
}
}
}},
{ "$lookup": {
"from": "users",
"localField": "users",
"foreignField": "_id",
"as": "users"
}},
{ "$project": {
"players": {
"$map": {
"input": "$players",
"as": "p",
"in": {
"$cond": {
"if": { "$gt": [ "$$p", {} ] },
"then": {
"$arrayElemAt": [
{ "$filter": {
"input": "$users",
"as": "u",
"cond": { "$eq": [ "$$u._id", "$$p" ] }
}},
0
]
},
"else": "$$p"
}
}
}
}
}}
])
Which gives the result when joined to the users object as:
{
"_id" : ObjectId("5933723c886d193061b99459"),
"players" : [
{
"_id" : ObjectId("5933723c886d193061b99458"),
"name" : "Bill",
"__v" : 0
},
"10.1.1.1",
"10.1.1.2"
]
}
So the "fancy" part really relies on this logical statement when considering the entries in the "players" array:
"$filter": {
"input": "$players",
"as": "p",
"cond": { "$gt": [ "$$p", {} ] }
}
How this works is that to MongoDB, an ObjectId and actually all BSON types have a specific sort precedence. In this case where the data is "Mixed" between ObjectId and String then the "string" values are considered "less than" the value of a "BSON Object", and the ObjectId values are "greater than".
This allows you to separate the ObjectId values from the source array into their own list. Given that list, you $lookup to perform the "join" at get the objects from the other collection.
In order to put them back, I'm using $map to "transpose" each element of the original "players" where the matched ObjectId was found with the related object. An alternate approach would be to "split" the two types, do the $lookup and $concatArrays between the Users and the "strings". But that would not maintain the original array order, so $map may be a better fit.
I will add of note that the same basic process can be applied in a "client" operation by similarly filtering the content of the "players" array to contain just the ObjectId values and then calling the "model" form of .populate() from "inside" the response of the initial query. The documentation shows an example of that form of usage, as do some answers on this site before it was possible to do a "nested populate" with mongoose.
The other point of mind here is that .populate() itself existed as a mongoose method long before the $lookup aggregation pipeline operator came about, and was a solution for a time when MongoDB itself was incapable of performing a "join" of any sort. So the operations are indeed "client" side as an emulation and really only perform additional queries that you do not need to be aware of in issuing the statements yourself.
Therefore it should generally be desirable in a modern scenario to use the "server" features, and avoid the overhead involved with multiple queries in order to get the result.
so in my MongoDB database I have a collection holding user posts.
Within that collection I have a set called "likes", which holds an array of the ids of the users that have liked that post. When querying I would like to pass a user id to my query and have a boolean in the result telling me whether the id exists in the array to see whether the user has already liked the post. I understand this would be easy to do with two queries, one to get the post and one to check if the user has liked it, but I would like to find the most efficient way to do this.
For example, one of my documents looks like this
{
_id: 24jef247jos991,
post: "Test Post",
likes: ["userid1", "userid2"]
}
When I query from "userid1" I would like the return
{
_id: 24jef247jos991,
post: "Test Post",
likes: ["userid1", "userid2"],
userLiked: true
}
But when I query from let's say "userid3" I would like
{
_id: 24jef247jos991,
post: "Test Post",
likes: ["userid1", "userid2"],
userLiked: false
}
You can add the $addFields stage checking each of the document likes arrays against the input user.
db.collection.aggregate( [
{
$addFields: {
"userLiked":{ $in: [ "userid1", "$likes" ] }
}
}
] )
Starting from MongoDB 3.4 you can use the $in aggregation operator to check if an array contains a given element. You can use the $addFields operator aggregation operator to add the newly computed value to your document without explicitly including other fields.
db.collection.aggregate( [
{ "$addFields": { "userLiked": { "$in": [ "userid1", "$likes" ] } } }
])
In MongoDB 3.2, you can use the $setIsSubset operator and the square bracket [] operator to do this. The downside of this approach is that you need to manually $project all the field in your document. Also the $setIsSubset operator with de-duplicate your array which may not be what you want.
db.collection.aggregate([
{ "$project": {
"post": 1, "likes": 1,
"userLiked": { "$setIsSubset": [ [ "userid3" ], "$likes" ] }
}}
])
Finally if your mongod version is 3.0 or older you need to use the $literal operator instead of the [] operator.
mongodb native for node.js (driver version is 2.2.4 and MongoDB shell version: 3.2.9)
My collection has objects like this:
{x:[{v:0.002},{v:0.00002}],t:0.00202} //<this one has the full total in its values
{x:[{v:0.002},{v:0.002}],t:0.00202}
{x:[{v:0.002},{v:0.002}],t:0.3}
(shown here without their object ids)
I am unsure how to add up all the x.v to return only objects where the total of x.v is greater or equal to the objects t
aggregate({"t":{"$gte":{"$add":["x.v"]}}})
returns every object, I don't have any other idea on the order of syntax from reading the docs.
Can mongodb even do this in a query?
With MongoDB 3.2, a couple of approaches you can take here. You can query with the $where operator:
db.collection.find({
"$where": function() {
return (this.x.reduce(function (a, b) {
return a + b.v;
}, 0) > this.t);
}
})
Sample Output
/* 1 */
{
"_id" : ObjectId("587107b3cbe62793a0f14e74"),
"x" : [
{
"v" : 0.002
},
{
"v" : 0.002
}
],
"t" : 0.00202
}
But note this is bound to be a not very efficient solution since a query operation with the $where operator calls the JavaScript engine to evaluate JavaScript code on every document and checks the condition for each.
This is very slow as MongoDB evaluates non-$where query operations before $where expressions and non-$where query statements may use an index.
It is advisable to combine with indexed queries if you can so that the query may be faster. However, it's strongly recommended to use JavaScript expressions and the $where operator as a last resort when you can't structure the data in any other way, or when you are dealing with a small subset of data.
A better approach would be to use the aggregation framework where you can use the $unwind operator to flatten the array x, calculate the sums for x.v within a $group pipeline and subsequently filtering the documents using the $redact pipeline stage. This allows you to proccess the logical condition with the $cond operator and uses the special operations $$KEEP to "keep" the document where the logical condition is true or $$PRUNE to "remove" the document where the condition is false.
This operation is similar to having a $project pipeline that selects the fields in the collection and creates a new field that holds the result from the logical condition query and then a subsequent $match, except that $redact uses a single pipeline stage which is more efficient.
db.collection.aggregate([
{ "$unwind": "$x" },
{
"$group": {
"_id": "$_id",
"x": { "$push": "$x" },
"t": { "$first": "$t" },
"y": { "$sum": "$x.v" }
}
},
{
"$redact": {
"$cond": [
{ "$gt": [ "$y", "$t" ] },
"$$KEEP",
"$$PRUNE"
]
}
}
])
Sample Output
/* 1 */
{
"_id" : ObjectId("587107b3cbe62793a0f14e74"),
"x" : [
{
"v" : 0.002
},
{
"v" : 0.002
}
],
"t" : 0.00202,
"y" : 0.004
}
However, as much as this solution is better than the previous solution that uses $where, bear in mind that the use of $unwind operator can also limit performance with larger datasets since it produces a cartesian product of the documents i.e. a copy of each document per array entry, which uses more memory (possible memory cap on aggregation pipelines of 10% total memory) and therefore takes time to produce as well processing the documents during the flattening process.
Also, this solution requires knowledge of the document fields since this is needed in the $group pipeline where you retain the fields in the grouping process by using the accumulators like $first or $last. That can be a huge limitation if your query needs to be dynamic.
For the most efficient solution, I would suggest bumping your MongoDB server to 3.4, and use the combination of the $redact pipeline stage and the new $reduce array operator to filter the documents in a seamless manner.
The $reduce is for calculating the sum of the x.v fields in the array by applying an expression to each element in an array and combining them into a single value.
You can then use this an an expression with the $redact pipeline's evaluation to get the desired result:
db.collection.aggregate([
{
"$redact": {
"$cond": [
{
"$gt": [
{
"$reduce": {
"input": "$x",
"initialValue": 0,
"in": { "$add": ["$$value", "$$this.v"] }
}
},
"$t"
]
},
"$$KEEP",
"$$PRUNE"
]
}
}
])
Sample Output
/* 1 */
{
"_id" : ObjectId("587107b3cbe62793a0f14e74"),
"x" : [
{
"v" : 0.002
},
{
"v" : 0.002
}
],
"t" : 0.00202
}
I try to make a query and i don't know the right way to do this.
The mongo collection structure contains multiples user ID (uid) and i want to make a query that get all datas ("Albums") where the User ID match one of the uid.
I do not know if the structure of the collection is good for that and I would like to know if I should do otherwise.
{
"_id": ObjectId("55814a9799677ba44e7826d1"),
"album": "album1",
"pictures": [
"1434536659272.jpg",
"1434552570177.jpg",
"1434552756857.jpg",
"1434552795100.jpg"
],
"uid": [
"12814a8546677ba44e745d85",
"e745d677ba4412814e745d7b",
"28114a85466e745d677d85qs"
],
"__v": 0
}
I just searched on internet and found this documentation http://docs.mongodb.org/manual/reference/operator/query/in/ but I'm not certain that this is the right way.
In short, I need to know: if I use the right method for the stucture of the collection and the operator "$in" is the right solution (knowing that it may have a lot of "User ID": between 2 and 2000 maximum).
You don't need $in unless you are matching for more than one possible value in a field, and that field does not have to be an array. $in is in fact shorthand for $or.
You just need a simple query here:
Model.find({ "uid": "12814a8546677ba44e745d85" },function(err,results) {
})
If you want "multiple" user id's then you can use $in:
Model.find(
{ "uid": { "$in": [
"12814a8546677ba44e745d85",
"e745d677ba4412814e745d7b",
] } },
function(err,results) {
}
)
Which is short for $or in this way:
Model.find(
{
"$or": [
{ "uid": "12814a8546677ba44e745d85" },
{ "uid": "e745d677ba4412814e745d7b" }
]
},
function(err,results) {
}
)
Just to answer your question, you can use the below query to get the desired result.
db.mycollection.find( {uid : {$in : ["28114a85466e745d677d85qs"] } } )
However, you need to revisit your data structure, looks like its a Many-to-Many problem and you might need to think about introducing a mid collection for that.