I'm starting to test ArangoSearch to use in an upcoming feature. I have a nested JSON field that I want to do a token search on a nested field.
{ip:{dpdm:{description:<text>}}}
The view is:
{
"writebufferIdle": 64,
"type": "arangosearch",
"writebufferSizeMax": 33554432,
"consolidationPolicy": {
"type": "tier",
"segmentsBytesFloor": 2097152,
"segmentsBytesMax": 5368709120,
"segmentsMax": 10,
"segmentsMin": 1,
"minScore": 0
},
"primarySort": [],
"globallyUniqueId": "h7437427BB0F0/26371461",
"id": "26371461",
"storedValues": [],
"writebufferActive": 0,
"consolidationIntervalMsec": 1000,
"cleanupIntervalStep": 2,
"commitIntervalMsec": 1000,
"links": {
"pmconfig": {
"analyzers": [
"identity"
],
"fields": {
"ip.dpdm.version_notes": {
"analyzers": [
"text_en"
]
},
"ip.digital.feature_list": {
"analyzers": [
"text_en"
]
},
"ip.common.Available-Views": {
"analyzers": [
"text_en"
]
},
"ip.pipeline.comment": {
"analyzers": [
"text_en"
]
},
"ip.dpdm.eccn": {
"analyzers": [
"text_en"
]
},
"ip.dpdm.description": {
"analyzers": [
"text_en"
]
},
"ip.dpdm.maturity_comments": {
"analyzers": [
"text_en"
]
}
},
"includeAllFields": true,
"storeValues": "none",
"trackListPositions": false
}
},
"primarySortCompression": "lz4"
}
The following query return 58 objects:
for d in pmconfig filter d.ip.dpdm.description like('% synchronous %') return d
But the following doesn't return any items:
for d in v_myView search analyzer(d.ip.dpdm.description in tokens('synchronous', 'text_en'), 'text_en') return d
I'm sure I did something wrong but it looks right to me. A pointer to get me going would be appreciated.
I knew I was missing something. Looks like I shouldn't use nested field paths when specifying the view. All I needed to do was to set the top level field "ip" to the "text_en" analyzer in the view. All is good now.
Related
I have a collection with 5 items in it. They look like this:
[
{
"brands": ["philips"],
"departments": [],
"markets": ["TR"],
"deleted": false,
"public": true,
"name": "test",
"account_id": 2,
"creator_user_id": 7,
"creator_user_name": "Murat"
}
]
To search through this collection I'm using the following query:
db.collection.find(
{
"$and": [
{
"$or": [
{
"brands": {
"$in": ["philips"]
}
},
{
"brands": []
}
]
},
{
"$or": [
{
"departments": {
"$in": ["foo","bar"]
}
},
{
"departments": []
}
]
},
{
"deleted": false
},
{
"account_id": 2
}
]
}
)
But I'm constantly having this error:
MongoServerError: FieldPath field names may not start with '$'. Consider using $getField or $setField.
I am 100% sure that there is no mistake or typo in the query. It looks like a version error, but I tried many mongoose version updates, the error persists.
I'm using mongoose 6.2.9
I am using CouchDB 3.1.1 to perform Mango queries against a database containing a large number of documents. A very common requirement in my application is to perform queries on a very specific and dynamic set of documents. From what I understand at this moment, these are the only choices I have on how to confront my problem:
Make multiple requests to /db/_find each with a distinct "_id"
Make a single call to /db/_find
Of the ways I can accomplish the second choice:
Use an "$or" array on all the "_id": value pairs
Use an "$or" array on all the values of the "_id" key
The second choice is what I would prefer to use since making multiple POST requests would incur overhead. Unfortunately using "$or" seems to get in the way of the query engine making use of the "_id" index.
Thus, choice #1 returns with a speedy 2 ms per transaction but the results are not sorted (requiring my application to do the sorting). Choice #2, given an array of 2 _ids, regardless of the $or syntax, takes over 3 seconds to render.
What is the most efficient way to use a CouchDB Mango query index against a specific set of documents?
Fast Example: Results using a single _id
{
"selector": {
"_id": "184094"
},
"fields": [
"_id"
]
}
documents examined: 26,312
results returned: 1
execution time: 2 ms
Slow Example: Results using $or of key / value pairs
{
"selector": {
"$or": [
{
"_id": "184094"
},
{
"_id": "157533"
}
]
},
"fields": [
"_id"
]
}
documents examined: 26,312
results returned: 2
execution time: 2,454 ms
Slow Example: Results using $or array of values
{
"selector": {
"_id": {
"$or": [
"184094",
"157533"
]
}
},
"fields": [
"_id"
]
}
documents examined: 26,312
results returned: 2
execution time: 2,522 ms
Slow Example: Results using $in (which is illegal but still returns results)
{
"selector": {
"_id": {
"$in": [
"184094",
"157533"
]
}
},
"fields": [
"_id"
]
}
documents examined: 26,312
results returned: 2
execution time: 2,618 ms
Index: The registered index for _id
{
"_id": "_design/508b5b51e6085c2f96444b82aced1e5dfec986b2",
"_rev": "1-f951eb482f9a521752adfdb6718a6a59",
"language": "query",
"views": {
"foo-index": {
"map": {
"fields": {
"_id": "asc"
},
"partial_filter_selector": {}
},
"reduce": "_count",
"options": {
"def": {
"fields": [
"_id"
]
}}}}}
Explain: An 'explain' summary done to one of the slow queries. Note that the registered index was used.
{
"dbname": "dnp_person_comment",
"index": {
"ddoc": "_design/508b5b51e6085c2f96444b82aced1e5dfec986b2",
"name": "foo-index",
"type": "json",
"partitioned": false,
"def": {
"fields": [
{
"_id": "asc"
}
]
}
},
"partitioned": false,
"selector": {
"$or": [
{
"_id": {
"$eq": "184094"
}
},
{
"_id": {
"$eq": "157533"
}
}
]
},
"opts": {
"use_index": [],
"bookmark": "nil",
"limit": 25,
"skip": 0,
"sort": {},
"fields": [
"_id"
],
"partition": "",
"r": [
49
],
"conflicts": false,
"stale": false,
"update": true,
"stable": false,
"execution_stats": false
},
"limit": 25,
"skip": 0,
"fields": [
"_id"
],
"mrargs": {
"include_docs": true,
"view_type": "map",
"reduce": false,
"partition": null,
"start_key": [],
"end_key": [
"<MAX>"
],
"direction": "fwd",
"stable": false,
"update": true,
"conflicts": "undefined"
}
}
{
"_id": "5e28b029a0c8263a8a56980a",
"name": "Recruiter",
"data": [
{
"_id": "5e28b0980f89ba3c0782828f",
"targetLink": "https://www.linkedin.com/in/dan-kelsall-7aa0926b/",
"name": "Dan Kelsall",
"headline": "Content Marketing & Copywriting",
"actions": [
{
"result": 1,
"name": "VISIT"
},
{
"result": 1,
"name": "FOLLOW"
}
]
},
{
"_id": "5e28b0980f89ba3c078283426f",
"targetLink": "https://www.linkedin.com/in/56wergwer/",
"name": "56wergwer",
"headline": "asdgawehethre",
"actions": [
{
"result": 1,
"name": "VISIT"
}
]
}
]
}
Here is one of my mongodb document. I'd like to update data->actions->result
So this is what I've done
Campaign.updateOne({
'data.targetLink': "https://www.linkedin.com/in/dan-kelsall-7aa0926b/",
'data.actions.name': "Follow"
}, {$set: {'data.$.actions.result': 0}})
But it seems not updating anything and even it can't find the document by this 'data.actions.name'
You need the positional filtered operator since the regular positional operator ($) can only be used for one level of nested arrays:
Campaign.updateOne(
{ "_id": "5e28b029a0c8263a8a56980a", "data.targetLink": "https://www.linkedin.com/in/dan-kelsall-7aa0926b/" },
{ $set: { "data.$.actions.$[action].result": 0 } },
{ arrayFilters: [ { "action.name": "Follow" } ] }
)
I have the following view in CouchDB that is reduced via _count:
function (doc) {
if (doc.type === "signature") {
emit([doc.worksite_id, doc.uid, doc.timestamp], doc._id);
}
}
There are cases where rather than using group_level=2 in my query to get my count values sorted by doc.worksite_id and doc.uid pairs (as shown below)...
{
"rows": [
{
"key": [
"worksite-1",
"id-1"
],
"value": 2
},
{
"key": [
"worksite-2",
"id-1"
],
"value": 1
},
{
"key": [
"worksite-2",
"id-2"
],
"value": 26
}
]
}
...I, instead, need to get count values sorted strictly by doc.uid, with an example of something similar to the following:
{
"rows": [
{
"key": [
"id-1"
],
"value": 3
},
{
"key": [
"id-2"
],
"value": 26
}
]
}
Is there an efficient way to do this based on the current view I'm querying from? And if not, what is the most efficient way to do this?
Cloudant is returning error message:
{"error":"invalid_key","reason":"Invalid key use-index for this request."}
whenever I try to query against an index with the combination operator, "$or".
A sample of what my documents look like is:
{
"_id": "28f240f1bcc2fbd9e1e5174af6905349",
"_rev": "1-fb9a9150acbecd105f1616aff88c26a8",
"type": "Feature",
"properties": {
"PageName": "A8",
"PageNumber": 1,
"Lat": 43.051523,
"Long": -71.498852
},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-71.49978935969642,
43.0508382914137
],
[
-71.49978564033566,
43.052210148524
],
[
-71.49791499857444,
43.05220740550381
],
[
-71.49791875962663,
43.05083554852429
],
[
-71.49978935969642,
43.0508382914137
]
]
]
}
}
The index that I created is for field "properties.PageName", which works fine when I'm just querying for one document, but as soon as I try for multiple ones, I would receive the error response as quoted in the beginning.
If it helps any, here is the call:
POST https://xyz.cloudant.com/db/_find
request body:
{
"selector": {
"$or": [
{ "properties.PageName": "A8" },
{ "properties.PageName": "M30" },
{ "properties.PageName": "AH30" }
]
},
"use-index": "pagename-index"
}
In order to perform an $or query you need to create a text (full text) index, rather than a json index. For example, I just created the following index:
{
"index": {
"fields": [
{"name": "properties.PageName", "type": "string"}
]
},
"type": "text"
}
I was then be able to perform the following query:
{
"selector": {
"$or": [
{ "properties.PageName": "A8" },
{ "properties.PageName": "M30" },
{ "properties.PageName": "AH30" }
]
}
}