cloudant searching index by multiple values - couchdb

Cloudant is returning error message:
{"error":"invalid_key","reason":"Invalid key use-index for this request."}
whenever I try to query against an index with the combination operator, "$or".
A sample of what my documents look like is:
{
"_id": "28f240f1bcc2fbd9e1e5174af6905349",
"_rev": "1-fb9a9150acbecd105f1616aff88c26a8",
"type": "Feature",
"properties": {
"PageName": "A8",
"PageNumber": 1,
"Lat": 43.051523,
"Long": -71.498852
},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-71.49978935969642,
43.0508382914137
],
[
-71.49978564033566,
43.052210148524
],
[
-71.49791499857444,
43.05220740550381
],
[
-71.49791875962663,
43.05083554852429
],
[
-71.49978935969642,
43.0508382914137
]
]
]
}
}
The index that I created is for field "properties.PageName", which works fine when I'm just querying for one document, but as soon as I try for multiple ones, I would receive the error response as quoted in the beginning.
If it helps any, here is the call:
POST https://xyz.cloudant.com/db/_find
request body:
{
"selector": {
"$or": [
{ "properties.PageName": "A8" },
{ "properties.PageName": "M30" },
{ "properties.PageName": "AH30" }
]
},
"use-index": "pagename-index"
}

In order to perform an $or query you need to create a text (full text) index, rather than a json index. For example, I just created the following index:
{
"index": {
"fields": [
{"name": "properties.PageName", "type": "string"}
]
},
"type": "text"
}
I was then be able to perform the following query:
{
"selector": {
"$or": [
{ "properties.PageName": "A8" },
{ "properties.PageName": "M30" },
{ "properties.PageName": "AH30" }
]
}
}

Related

How can I update one document at nested array

{
"_id": "5e28b029a0c8263a8a56980a",
"name": "Recruiter",
"data": [
{
"_id": "5e28b0980f89ba3c0782828f",
"targetLink": "https://www.linkedin.com/in/dan-kelsall-7aa0926b/",
"name": "Dan Kelsall",
"headline": "Content Marketing & Copywriting",
"actions": [
{
"result": 1,
"name": "VISIT"
},
{
"result": 1,
"name": "FOLLOW"
}
]
},
{
"_id": "5e28b0980f89ba3c078283426f",
"targetLink": "https://www.linkedin.com/in/56wergwer/",
"name": "56wergwer",
"headline": "asdgawehethre",
"actions": [
{
"result": 1,
"name": "VISIT"
}
]
}
]
}
Here is one of my mongodb document. I'd like to update data->actions->result
So this is what I've done
Campaign.updateOne({
'data.targetLink': "https://www.linkedin.com/in/dan-kelsall-7aa0926b/",
'data.actions.name': "Follow"
}, {$set: {'data.$.actions.result': 0}})
But it seems not updating anything and even it can't find the document by this 'data.actions.name'
You need the positional filtered operator since the regular positional operator ($) can only be used for one level of nested arrays:
Campaign.updateOne(
{ "_id": "5e28b029a0c8263a8a56980a", "data.targetLink": "https://www.linkedin.com/in/dan-kelsall-7aa0926b/" },
{ $set: { "data.$.actions.$[action].result": 0 } },
{ arrayFilters: [ { "action.name": "Follow" } ] }
)

How to query CouchDB view by specific element in key array?

I have the following view in CouchDB that is reduced via _count:
function (doc) {
if (doc.type === "signature") {
emit([doc.worksite_id, doc.uid, doc.timestamp], doc._id);
}
}
There are cases where rather than using group_level=2 in my query to get my count values sorted by doc.worksite_id and doc.uid pairs (as shown below)...
{
"rows": [
{
"key": [
"worksite-1",
"id-1"
],
"value": 2
},
{
"key": [
"worksite-2",
"id-1"
],
"value": 1
},
{
"key": [
"worksite-2",
"id-2"
],
"value": 26
}
]
}
...I, instead, need to get count values sorted strictly by doc.uid, with an example of something similar to the following:
{
"rows": [
{
"key": [
"id-1"
],
"value": 3
},
{
"key": [
"id-2"
],
"value": 26
}
]
}
Is there an efficient way to do this based on the current view I'm querying from? And if not, what is the most efficient way to do this?

Speeding up Cloudant query for type text index

We have a table with this type of structure:
{_id:15_0, createdAt: 1/1/1, task_id:[16_0, 17_0, 18_0], table:”details”, a:b, c: d, more}
We created indexes using
{
"index": {},
"name": "paginationQueryIndex",
"type": "text"
}
It auto created
{
"ddoc": "_design/28e8db44a5a0862xxx",
"name": "paginationQueryIndex",
"type": "text",
"def": {
"default_analyzer": "keyword",
"default_field": {
},
"selector": {
},
"fields": [
],
"index_array_lengths": true
}
}
We are using the following query
{
"selector": {
"createdAt": { "$gt": 0 },
"task_id": { "$in": [ "18_0" ] },
"table": "details"
},
"sort": [ { "createdAt": "desc" } ],
"limit”: 20
}
It takes 700-800 ms for first time, after that it decreases to 500-600 ms
Why does it take longer the first time?
Any way to speed up the query?
Any way to add indexes to specific fields if type is “text”? (instead of indexing all the fields in these records)
You could try creating the index more explicitly, defining the type of each field you wish to index e.g.:
{
"index": {
"fields": [
{
"name": "createdAt",
"type": "string"
},
{
"name": "task_id",
"type": "string"
},
{
"name": "table",
"type": "string"
}
]
},
"name": "myindex",
"type": "text"
}
Then your query becomes:
{
"selector": {
"createdAt": { "$gt": "1970/01/01" },
"task_id": { "$in": [ "18_0" ] },
"table": "details"
},
"sort": [ { "createdAt": "desc" } ],
"limit": 20
}
Notice that I used strings where the data type is a string.
If you're interested in performance, try removing clauses from your query one at-a-time to see if one is causing the performance problem. You can also look at the explanation of your query to see if it using your index correctly.
Documentation on creating an explicit text query index is here

Keeping nested arrays but pulling out all it's doubly nested arrays in mongodb [duplicate]

This question already has answers here:
How to Update Multiple Array Elements in mongodb
(16 answers)
Updating a Nested Array with MongoDB
(2 answers)
Closed 5 years ago.
Building a Nodejs app, I'm trying to pull all doubly nested records from a Mongo Database. Attempts that I've made only removed one doubly nested record or all nested records. As in the example data below I've been trying to remove all tickets that has the same keyId. I've reduced the example but tickets as an array there might be other elements with the same structure with different "keyIds" that shouldn't be removed. I've looked this question but it only refrains to removing one record of a doubly nested array, not all of them at once.
[
{
"_id": "59fe54098448d822f89a7e62",
"ownerId": "59b23449b20b7c1838eee1a3",
"name": "Home",
"keys": [
{
"id": "6d7435625564594f4a563554796c6a77",
"name": "Front Door"
}
],
"grants": [
{
"id": "307658775634774a556b677650347072",
"userId": "59b23449b20b7c1838eee1a3",
"tickets": [
{
"keyId": "6d7435625564594f4a563554796c6a77",
"iv": "b7090268bdaf9ab55270e133b5629e28"
}
]
},
{
"id": "37703369365765485763484a4358774d",
"userId": "59b23449b20b7c1838eee1a3",
"tickets": [
{
"keyId": "6d7435625564594f4a563554796c6a77",
"iv": "d2e2de0f9387c5d9b16424e8ac66a3c1"
}
]
},
{
"id": "3451483977564d755278397939593633",
"userId": "59b23449b20b7c1838eee1a3",
"tickets": [
{
"keyId": "6d7435625564594f4a563554796c6a77",
"iv": "582ff50ac3d337c62eb53094470e3161"
}
]
},
{
"id": "7059684f4e42754d55456e726b35664e",
"userId": "59b23449b20b7c1838eee1a3",
"tickets": [
{
"keyId": "6d7435625564594f4a563554796c6a77",
"iv": "b110ee5cb5da8941cc8ad6e1c3fe501c"
}
]
}
]
}
]
After removing all tickets with keyId=6d7435625564594f4a563554796c6a77 the intended data should look like this:
[
{
"_id": "59fe54098448d822f89a7e62",
"ownerId": "59b23449b20b7c1838eee1a3",
"name": "Home",
"keys": [
{
"id": "6d7435625564594f4a563554796c6a77",
"name": "Front Door"
}
],
"grants": [
{
"id": "307658775634774a556b677650347072",
"userId": "59b23449b20b7c1838eee1a3",
"tickets": []
},
{
"id": "37703369365765485763484a4358774d",
"userId": "59b23449b20b7c1838eee1a3",
"tickets": []
},
{
"id": "3451483977564d755278397939593633",
"userId": "59b23449b20b7c1838eee1a3",
"tickets": []
},
{
"id": "7059684f4e42754d55456e726b35664e",
"userId": "59b23449b20b7c1838eee1a3",
"tickets": []
}
]
}
]
This code remove all grants at once:
db.places.update({}, {
$pull: {
"grants": {
"tickets": {
$elemMatch: { "keyId": keyID }
}
}
}
}, { multi: true });
This pull out just the first ticket and with "$pullAll" doesn't do anything:
db.places.findAndModify(
{
ownerId: ownerID, "grants.tickets.keyId": keyID
},
[ ],
{ $pull: { "grants.$.tickets": { keyId: keyID } } },
{ multi: true },
next
);
And this throws me an error saying: cannot use the part (grants of grants.tickets.$*.keyId) to traverse the element
db.places.update({ "grants.tickets.keyId": keyID }, {
$pull: {
"grants.tickets.$*.keyId": keyID
}
}, { multi: true });

Elasticsearch match with stemming

How do I do a search for a stemmed match?
I.e. at the moment I have many documents that contain the word "skateboard" in the item_title field, but only 3 documents that contain the word "skateboards". Because of this, when I do the following search:
POST /my_index/my_type/_search
{
"size": 100,
"query" : {
"multi_match": {
"query": "skateboards",
"fields": [ "item_title^3" ]
}
}
}
I only get 3 results. However, I would like also documents with the word "skateboard" to be returned.
From what I understand from Elasticsearch I would expect that this is done by specifying a mapping on the item_title field that contains an analyser which indexes the stemmed version of each word, but I can't seem to find the documentation on how to do this, which suggests that it's done in a different way.
Suggestions?
Here's one example:
PUT /stem
{
"settings": {
"analysis": {
"filter": {
"filter_stemmer": {
"type": "stemmer",
"language": "english"
}
},
"analyzer": {
"tags_analyzer": {
"type": "custom",
"filter": [
"standard",
"lowercase",
"filter_stemmer"
],
"tokenizer": "standard"
}
}
}
},
"mappings": {
"test": {
"properties": {
"item_title": {
"analyzer": "tags_analyzer",
"type": "text"
}
}
}
}
}
Index some sample docs:
POST /stem/test/1
{
"item_title": "skateboards"
}
POST /stem/test/2
{
"item_title": "skateboard"
}
POST /stem/test/3
{
"item_title": "skate"
}
Perform the query:
GET /stem/test/_search
{
"query": {
"multi_match": {
"query": "skateboards",
"fields": [
"item_title^3"
]
}
},
"fielddata_fields": [
"item_title"
]
}
And see the results:
"hits": [
{
"_index": "stem",
"_type": "test",
"_id": "1",
"_score": 1,
"_source": {
"item_title": "skateboards"
},
"fields": {
"item_title": [
"skateboard"
]
}
},
{
"_index": "stem",
"_type": "test",
"_id": "2",
"_score": 1,
"_source": {
"item_title": "skateboard"
},
"fields": {
"item_title": [
"skateboard"
]
}
}
]
I have added, also, the fielddata_fields element so that you can see how the content of the field has been indexed. As you can see, in both cases, the indexed term is skateboard.

Resources