This is my query:
{
"selector": {
"_id": {
"$regex": "^rati" //need to find all documents in ratings partition
}
},
"fields": [
"MovieID",
"UserId",
"Rating"
],
"limit": 10,
"sort": [
{
"MovieID": "asc"
}
]
}
When I run this query I have that error: Error running query. Reason: (no_usable_index) No global index exists for this sort, try indexing by the sort fields.
If I remove
"sort": [
{
"MovieID": "asc"
}
]
everything works good. Honestly I'm going crazy, I can't understand where I'm wrong.
I've tried this query:
{
"selector": {
"_id": {
"$regex": "^rati"
},
"MovieID": {
"$gte": 0
}
},
"fields": [
"_id",
"MovieID",
"UserId",
"Rating"
],
"limit": 10,
"sort": [
{
"_id": "asc"
}
]
}
but is the same.
You need to create an Index for the field MovieID
//Create via POST /db/_index HTTP/1.1 Content-Type: application/json
{
"index": {
"fields": ["MovieID"]
},
"name" : "MovieID-index",
"type" : "json"
}
Afterward include the field MovieID as part of the Selector.
Try this here:
{
"selector": {
"_id": {
"$regex": "^rati" //need to find all documents in ratings partition
},
"MovieID": {"$gte": 0} // include MovieID, If the ID is non-numeric change the selecor type.
},
"fields": [
"MovieID",
"UserId",
"Rating"
],
"limit": 10,
"sort": [
{
"MovieID": "asc"
}
]
}
Related
I am using CouchDB 3.1.1 to perform Mango queries against a database containing a large number of documents. A very common requirement in my application is to perform queries on a very specific and dynamic set of documents. From what I understand at this moment, these are the only choices I have on how to confront my problem:
Make multiple requests to /db/_find each with a distinct "_id"
Make a single call to /db/_find
Of the ways I can accomplish the second choice:
Use an "$or" array on all the "_id": value pairs
Use an "$or" array on all the values of the "_id" key
The second choice is what I would prefer to use since making multiple POST requests would incur overhead. Unfortunately using "$or" seems to get in the way of the query engine making use of the "_id" index.
Thus, choice #1 returns with a speedy 2 ms per transaction but the results are not sorted (requiring my application to do the sorting). Choice #2, given an array of 2 _ids, regardless of the $or syntax, takes over 3 seconds to render.
What is the most efficient way to use a CouchDB Mango query index against a specific set of documents?
Fast Example: Results using a single _id
{
"selector": {
"_id": "184094"
},
"fields": [
"_id"
]
}
documents examined: 26,312
results returned: 1
execution time: 2 ms
Slow Example: Results using $or of key / value pairs
{
"selector": {
"$or": [
{
"_id": "184094"
},
{
"_id": "157533"
}
]
},
"fields": [
"_id"
]
}
documents examined: 26,312
results returned: 2
execution time: 2,454 ms
Slow Example: Results using $or array of values
{
"selector": {
"_id": {
"$or": [
"184094",
"157533"
]
}
},
"fields": [
"_id"
]
}
documents examined: 26,312
results returned: 2
execution time: 2,522 ms
Slow Example: Results using $in (which is illegal but still returns results)
{
"selector": {
"_id": {
"$in": [
"184094",
"157533"
]
}
},
"fields": [
"_id"
]
}
documents examined: 26,312
results returned: 2
execution time: 2,618 ms
Index: The registered index for _id
{
"_id": "_design/508b5b51e6085c2f96444b82aced1e5dfec986b2",
"_rev": "1-f951eb482f9a521752adfdb6718a6a59",
"language": "query",
"views": {
"foo-index": {
"map": {
"fields": {
"_id": "asc"
},
"partial_filter_selector": {}
},
"reduce": "_count",
"options": {
"def": {
"fields": [
"_id"
]
}}}}}
Explain: An 'explain' summary done to one of the slow queries. Note that the registered index was used.
{
"dbname": "dnp_person_comment",
"index": {
"ddoc": "_design/508b5b51e6085c2f96444b82aced1e5dfec986b2",
"name": "foo-index",
"type": "json",
"partitioned": false,
"def": {
"fields": [
{
"_id": "asc"
}
]
}
},
"partitioned": false,
"selector": {
"$or": [
{
"_id": {
"$eq": "184094"
}
},
{
"_id": {
"$eq": "157533"
}
}
]
},
"opts": {
"use_index": [],
"bookmark": "nil",
"limit": 25,
"skip": 0,
"sort": {},
"fields": [
"_id"
],
"partition": "",
"r": [
49
],
"conflicts": false,
"stale": false,
"update": true,
"stable": false,
"execution_stats": false
},
"limit": 25,
"skip": 0,
"fields": [
"_id"
],
"mrargs": {
"include_docs": true,
"view_type": "map",
"reduce": false,
"partition": null,
"start_key": [],
"end_key": [
"<MAX>"
],
"direction": "fwd",
"stable": false,
"update": true,
"conflicts": "undefined"
}
}
I was stuck on mongodb aggregation to get count of rating. i have already trying to make myself pipeline. looks above
data (products collection)
{
"status": 200,
"message": null,
"data": {
"_id": "5cc570257631a313d489ba4a",
"media": [
"httpsdssd",
"dfdfd"
],
"comment": [
"5cc57f1053273c05cc60e707",
"5cc585bf6ff7a812e0e7d9d9",
"5cc5c654bc73b408787ffadc",
"5cc5c6e3bc73b408787ffadd"
],
"store": "5cc2c9710bc5d615781fcf8a",
"meta": {
"title": "Traveling Sumbar",
"price": "150000",
"max": 5,
"duration": 6,
"description": "fdf fdnf jdnf dfnkdknfkkd",
"location": {
"province": "Sumbar",
"city": "Padang"
}
},
"option": {
"is_promo": false,
"auto_delete": null
},
"created_at": "2019-04-28T09:19:33.233Z",
"updated_at": "2019-04-28T15:29:39.921Z",
"__v": 0
}
}
comment data on (products_comment)
{
"helped": [],
"deleted_at": null,
"_id": "5cc3276e32940613506c3848",
"user": "5cc2c7fb0bc5d615781fcf86",
"rating": "4",
"body": "fdfdlfdlfkdlfkdlfkd",
"created_at": "2019-04-26T15:44:46.224Z",
"updated_at": "2019-04-28T16:00:48.400Z",
"__v": 0
},
{
"helped": [],
"deleted_at": null,
"_id": "5cc3276e32940613506c3848",
"user": "5cc2c7fb0bc5d615781fcf86",
"rating": "4",
"body": "fdfdlfdlfkdlfkdlfkd",
"created_at": "2019-04-26T15:44:46.224Z",
"updated_at": "2019-04-28T16:00:48.400Z",
"__v": 0
},
{
"helped": [],
"deleted_at": null,
"_id": "5cc3276e32940613506c3848",
"user": "5cc2c7fb0bc5d615781fcf86",
"rating": "3",
"body": "fdfdlfdlfkdlfkdlfkd",
"created_at": "2019-04-26T15:44:46.224Z",
"updated_at": "2019-04-28T16:00:48.400Z",
"__v": 0
},
I have already try make aggregation pipeline like this
{
$lookup: {
from: "stores",
localField: "store",
foreignField: "_id",
as: "store"
}
},
{
$lookup: {
from: "products_comment",
localField: "comment",
foreignField: "_id",
as: "comment"
}
},
{ $unwind: "$comment" },
{
$project: {
media: 1,
"store.type": 1,
"store.profile.address.city": 1,
"meta.title": 1,
"meta.price": 1,
"comment.rating": 1
}
}
but result unlike expectation, i want result like this
{
"_id": "5cc570257631a313d489ba4a",
"media": [
"httpsdssd",
"dfdfd"
],
"comment": {
1_rating: 0, <value of rating: count of value>
2_rating: 3,
3_rating: 5,
....,
},
"store": [
{
"type": "craft",
"profile": {
"address": {
city: "Padang
}
}
}
],
"meta": {
"title": "Traveling Sumbar",
"price": "150000"
}
}
how i do to solve my problem ?
Below Query will give you exactly expected Output :
var query = [
{
$lookup: {
from: "comments",
localField: "comment",
foreignField: "_id",
as: "comments"
}
},
{ $unwind: "$comments" },
{ $group : {
_id: {
_id: '$_id',
rating: '$comments.rating',
media : '$media',
meta : '$meta',
store : '$store'
},
totalRating: {$sum: 1}
}
},
{
$group : {
_id : {
_id : '$_id._id',
media : '$_id.media',
meta : '$_id.meta',
store : '$_id.store'
},
comments : {
$push : {
rating : '$_id.rating',
totalRating : '$totalRating'
}
}
}
},
{
$lookup: {
from: "stores",
localField: "store",
foreignField: "_id",
as: "store"
}
},
{
$project: {
'_id' : '$_id._id',
media : '$_id.media',
store : '$store',
meta : {
title: '$_id.meta.title',
price : '$_id.meta.price'
},
comments : { "$arrayToObject": {
"$map": {
"input": "$comments",
"as": "el",
"in": {
"k": "$$el.rating",
"v": "$$el.totalRating"
}
}
}
}
}
}
];
Output :
{
"_id" : ObjectId("5cc718715290f4ed550f5305"),
"media" : [
"httpsdssd",
"dfdfd"
],
"store" : [ ],
"meta" : {
"title" : "Traveling Sumbar",
"price" : "150000"
},
"comments" : {
"3" : 1,
"4" : 2
}
}
{
"_id" : ObjectId("5cc88d99d486568c5745e4b7"),
"media" : [
"maha",
"sagar"
],
"store" : [ ],
"meta" : {
"title" : "Sagar Sumbar",
"price" : "15000"
},
"comments" : {
"3" : 2,
"5" : 1,
"1" : 1
}
}
NOTE: Store data will be fetched from stores collection by $lookup. I don't have a model/data so, not in the output.
Please don't tell me to "googleit"!
I have been poring over the Apache pages and the IBM pages for days trying to find the full allowed syntax for a Design Doc.
From the above readings:
the 'map' property is always a Javascript function
the 'options' property may be one/both of local_seq or include_design.
When I use Fauxton to edit a Mango Query, however, I see that the reality is much broader.
I defined a query ...
{
"selector": {
"data.type": {
"$eq": "invoice"
},
"data.idib": {
"$gt": 0,
"$lt": 99999
}
},
"sort": [
{
"data.type": "desc"
},
{
"data.idib": "desc"
}
]
}
... with an accompanying index ...
{
"index": {
"fields": [
"foo"
]
},
"name": "foo-json-index",
"type": "json"
}
... and then looked at the design doc produced ...
{
"_id": "_design/5b1cf1be5a6b7013019ba4afac2b712fc06ea82f",
"_rev": "1-1e6c5b7bc622d9b3c9b5f14cb0fcb672",
"language": "query",
"views": {
"invoice_code": {
"map": {
"fields": {
"data.type": "desc",
"data.idib": "desc"
},
"partial_filter_selector": {}
},
"reduce": "_count",
"options": {
"def": {
"fields": [
{
"data.type": "desc"
},
{
"data.idib": "desc"
}
]
}
}
}
}
}
Both of the published syntax rules are broken!
map is not a function
options defines the fields of the index
Where can I find a full description of all the allowed properties of a Design Document?
Given collection(#name: users) Structure:
{
"_id" : ObjectId("57653dcc533304a40ac504fc"),
"username" : "XYZ",
"followers" : [
{
"count" : 31,
"ts" : ISODate("2016-06-17T18:30:00.996Z")
},
{
"count" : 31,
"ts" : ISODate("2016-06-18T18:30:00.288Z")
}
]
}
I want to query this collection based on username field, and ts to be returned in 'yyyy-mm-dd' format.
Expected Output:
{
"_id" : ObjectId("57653dcc533304a40ac504fc"),
"username" : "XYZ",
"followers" : [
{
"count" : 31,
"date" : "2016-06-17"
},
{
"count" : 31,
"date" : "2016-06-18"
}
]
}
I have tried something like this:
db.users.aggregate([
{$match:{"username":"xyz"}},
{$project:{ "followers":{"count":1,
"date":"$followers.ts.toISOString().slice(0,10).replace(/-/g,'-')"
}}
}
])
But it doesn't seems to be working. Can anyone please help?
Thanks much.
Consider running an aggregation pipeline that will allow you to flatten the data list first, project the new field using the $dateToString operator, then regroup the flattened docs to get your desired result.
The above can be shown in three distinct pipelines:
db.users.aggregate([
{ "$match": { "username": "xyz" } },
{ "$unwind": "$followers" },
{
"$project": {
"username": 1,
"count": "$followers.count",
"date": { "$dateToString": { "format": "%Y-%m-%d", "date": "$followers.ts" } }
}
},
{
"$group": {
"_id": "$_id",
"username": { "$first": "$username" },
"followers": { "$push": {
"count": "$count",
"date": "$date"
}}
}
}
])
With MongoDB 3.4 and newer, you can use the new $addFields pipeline step together with $map to create the array field without the need to unwind and group:
db.users.aggregate([
{ "$match": { "username": "xyz" } },
{
"$addFields": {
"followers": {
"$map": {
"input": "$followers",
"as": "follower",
"in": {
"count": "$$follower.count",
"date": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$$follower.ts"
}
}
}
}
}
}
}
])
The best and easiest way to do this is to transform each element in the array with the $map operator. Of course in the "in" expression, you need to use the $dateToString to convert you "date" to string using a format specifiers.
db.coll.aggregate(
[
{ "$match": { "username": "XYZ" } },
{ "$project": {
"username": 1,
"followers": {
"$map": {
"input": "$followers",
"as": "f",
"in": {
"count": "$$f.count",
"date": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$$f.ts"
}
}
}
}
}
}}
]
)
which produces:
{
"_id" : ObjectId("57653dcc533304a40ac504fc"),
"username" : "XYZ",
"followers" : [
{
"count" : 31,
"date" : "2016-06-17"
},
{
"count" : 31,
"date" : "2016-06-18"
}
]
}
Cloudant is returning error message:
{"error":"invalid_key","reason":"Invalid key use-index for this request."}
whenever I try to query against an index with the combination operator, "$or".
A sample of what my documents look like is:
{
"_id": "28f240f1bcc2fbd9e1e5174af6905349",
"_rev": "1-fb9a9150acbecd105f1616aff88c26a8",
"type": "Feature",
"properties": {
"PageName": "A8",
"PageNumber": 1,
"Lat": 43.051523,
"Long": -71.498852
},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-71.49978935969642,
43.0508382914137
],
[
-71.49978564033566,
43.052210148524
],
[
-71.49791499857444,
43.05220740550381
],
[
-71.49791875962663,
43.05083554852429
],
[
-71.49978935969642,
43.0508382914137
]
]
]
}
}
The index that I created is for field "properties.PageName", which works fine when I'm just querying for one document, but as soon as I try for multiple ones, I would receive the error response as quoted in the beginning.
If it helps any, here is the call:
POST https://xyz.cloudant.com/db/_find
request body:
{
"selector": {
"$or": [
{ "properties.PageName": "A8" },
{ "properties.PageName": "M30" },
{ "properties.PageName": "AH30" }
]
},
"use-index": "pagename-index"
}
In order to perform an $or query you need to create a text (full text) index, rather than a json index. For example, I just created the following index:
{
"index": {
"fields": [
{"name": "properties.PageName", "type": "string"}
]
},
"type": "text"
}
I was then be able to perform the following query:
{
"selector": {
"$or": [
{ "properties.PageName": "A8" },
{ "properties.PageName": "M30" },
{ "properties.PageName": "AH30" }
]
}
}