I'm trying to query documents in a Cloudant.com database (CouchDB). The two following query requests work fine separately:
{ "selector": { "some_field": "value_1" } }
{ "selector": { "some_field": "value_2" } }
Cloudant's documentation seems to indicate I should be able to combine those two queries into a single HTTP request as follows:
{ "selector": { "$or": [ { "some_field": "value_1" },
{ "some_field": "value_2" } ] } }
But when I try that I receive the following response:
{"error":"no_usable_index",
"reason":"There is no operator in this selector can used with an index."}
Can someone tell me what I need to do to get this to work?
There doesn't seem to be a way to achieve this with Cloudant Query at the moment. However, you can use a view query instead using the index created with Cloudant Query. Assuming the index is in a design document named ae97413b0892b3738572e05b2101cdd303701bb8:
curl -X POST \
'https://youraccount.cloudant.com/db/_design/ae97413b0892b3738572e05b2101cdd303701bb8/_view/ae97413b0892b3738572e05b2101cdd303701bb8?reduce=false&include_docs=true' \
-d '
{
"keys":[
["value_1"],
["value_2"]
]
}'
This will give you a response like this:
{
"total_rows": 3,
"offset": 1,
"rows": [
{
"id": "5fcec42ba5cad4fb48a676400dc8f127",
"key": [
"abc"
],
"value": null,
"doc": {
"_id": "5fcec42ba5cad4fb48a676400dc8f127",
"_rev": "1-0042bf88a7d830e9fdb0326ae957e3bc",
"some_field": "value_1"
}
},
{
"id": "955606432c9d3aaa48cab0c34dc2a9c8",
"key": [
"ghi"
],
"value": null,
"doc": {
"_id": "955606432c9d3aaa48cab0c34dc2a9c8",
"_rev": "1-68fac0c180923a2bf133132301b1c15e",
"some_field": "value_2"
}
}
]
}
Related
I have data something like below:
{
"_id": "60708607143b058e101fc189",
"orders": {
"userID": "606eaa5cf67ac70cfa347fcd",
"order": [
{
"productIDs": [ "606f1f37006513258c8b59b9" ],
"vendorID": "60641cf597aed2a9a2971a3f",
"id": "60708607143b058e101fc18a"
},
{
"productIDs": [ "606f1fec006513258c8b59ba", "606f2bb2006513258c8b59bd" ],
"vendorID": "60642991015028ba0a6ce72c",
"id": "60708607143b058e101fc18b"
}
]
},
}
Is there a way to find any given vendorID in this record using nodejs and mongoose?
If you are logged in as a vendor, then you can send the vendor_id in the body of the request. To filter all orders associated to that particular vendor, you can try this:
Model.find({"orders.order":
{
"$elemMatch": {
"vendorID": req.body.vendor_id
}
}
});
My data:
{
"rootElement": {
"names": {
"name": [
"Haseb",
"Anil",
"Ajinkya",
{
"city": "mumbai",
"state": "maharashtra",
"job": {
"second": "bosch",
"first": "infosys"
}
}
]
},
"places": {
"place": {
"origin": "INDIA",
"current": "GERMANY"
}
}
}
}
I created a hash index on job field with the API:
http://localhost:8529/_db/_api/index?collection=Metadata
{
"type": "hash",
"fields": [
"rootElement.names.name[*].jobs"
]
}
And I make the search query with the API:
http://localhost:8529/_db/_api/simple/by-example
{
"collection": "Metadata",
"example": {
"rootElement.names.name[*].jobs ": "bosch"
}
}
Ideally, only the document containing job : bosch should be returned as a result. But for me it gives all the documents in the array name[*]. Where I am doing mistake?
Array asterisk operators are not supported by simple queries.
You need to use AQL for this:
FOR elem IN Metadata FILTER elem.rootElement.names.name[*].jobs = "bosch" RETURN elem
You can also execute AQL via the REST interface - However you should rather try to let a driver do the heavy lifting for you.
Cloudant is returning error message:
{"error":"invalid_key","reason":"Invalid key use-index for this request."}
whenever I try to query against an index with the combination operator, "$or".
A sample of what my documents look like is:
{
"_id": "28f240f1bcc2fbd9e1e5174af6905349",
"_rev": "1-fb9a9150acbecd105f1616aff88c26a8",
"type": "Feature",
"properties": {
"PageName": "A8",
"PageNumber": 1,
"Lat": 43.051523,
"Long": -71.498852
},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-71.49978935969642,
43.0508382914137
],
[
-71.49978564033566,
43.052210148524
],
[
-71.49791499857444,
43.05220740550381
],
[
-71.49791875962663,
43.05083554852429
],
[
-71.49978935969642,
43.0508382914137
]
]
]
}
}
The index that I created is for field "properties.PageName", which works fine when I'm just querying for one document, but as soon as I try for multiple ones, I would receive the error response as quoted in the beginning.
If it helps any, here is the call:
POST https://xyz.cloudant.com/db/_find
request body:
{
"selector": {
"$or": [
{ "properties.PageName": "A8" },
{ "properties.PageName": "M30" },
{ "properties.PageName": "AH30" }
]
},
"use-index": "pagename-index"
}
In order to perform an $or query you need to create a text (full text) index, rather than a json index. For example, I just created the following index:
{
"index": {
"fields": [
{"name": "properties.PageName", "type": "string"}
]
},
"type": "text"
}
I was then be able to perform the following query:
{
"selector": {
"$or": [
{ "properties.PageName": "A8" },
{ "properties.PageName": "M30" },
{ "properties.PageName": "AH30" }
]
}
}
I have an index like following settings and mapping;
{
"settings":{
"index":{
"analysis":{
"analyzer":{
"analyzer_keyword":{
"tokenizer":"keyword",
"filter":"lowercase"
}
}
}
}
},
"mappings":{
"product":{
"properties":{
"name":{
"analyzer":"analyzer_keyword",
"type":"string",
"index": "not_analyzed"
}
}
}
}
}
I am struggling with making an implementation for wildcard search on name field. My example data like this;
[
{"name": "SVF-123"},
{"name": "SVF-234"}
]
When I perform following query;
http://localhost:9200/my_index/product/_search -d '
{
"query": {
"filtered" : {
"query" : {
"query_string" : {
"query": "*SVF-1*"
}
}
}
}
}'
It returns SVF-123,SVF-234. I think, it still tokenizes data. It must return only SVF-123.
Could you please help on this?
Thanks in advance
There's a couple of things going wrong here.
First, you are saying that you don't want terms analyzed index time. Then, there's an analyzer configured (that's used search time) that generates incompatible terms. (They are lowercased)
By default, all terms end up in the _all-field with the standard analyzer. That is where you end up searching. Since it tokenizes on "-", you end up with an OR of "*SVF" and "1*".
Try to do a terms facet on _all and on name to see what's going on.
Here's a runnable Play and gist: https://www.found.no/play/gist/3e5fcb1b4c41cfc20226 (https://gist.github.com/alexbrasetvik/3e5fcb1b4c41cfc20226)
You need to make sure the terms you index is compatible with what you search for. You probably want to disable _all, since it can muddy what's going on.
#!/bin/bash
export ELASTICSEARCH_ENDPOINT="http://localhost:9200"
# Create indexes
curl -XPUT "$ELASTICSEARCH_ENDPOINT/play" -d '{
"settings": {
"analysis": {
"text": [
"SVF-123",
"SVF-234"
],
"analyzer": {
"analyzer_keyword": {
"type": "custom",
"tokenizer": "keyword",
"filter": [
"lowercase"
]
}
}
}
},
"mappings": {
"type": {
"properties": {
"name": {
"type": "string",
"index": "not_analyzed",
"analyzer": "analyzer_keyword"
}
}
}
}
}'
# Index documents
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_bulk?refresh=true" -d '
{"index":{"_index":"play","_type":"type"}}
{"name":"SVF-123"}
{"index":{"_index":"play","_type":"type"}}
{"name":"SVF-234"}
'
# Do searches
# See all the generated terms.
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_search?pretty" -d '
{
"facets": {
"name": {
"terms": {
"field": "name"
}
},
"_all": {
"terms": {
"field": "_all"
}
}
}
}
'
# Analyzed, so no match
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_search?pretty" -d '
{
"query": {
"match": {
"name": {
"query": "SVF-123"
}
}
}
}
'
# Not analyzed according to `analyzer_keyword`, so matches. (Note: term, not match)
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_search?pretty" -d '
{
"query": {
"term": {
"name": {
"value": "SVF-123"
}
}
}
}
'
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_search?pretty" -d '
{
"query": {
"term": {
"_all": {
"value": "svf"
}
}
}
}
'
My solution adventure
I have started my case as you can see in my question. Whenever, I have changed a part of my settings, one part started to work, but another part stop working. Let me give my solution history:
1.) I have indexed my data as default. This means, my data is analyzed as default. This will cause problem on my side. For example;
When user started to search a keyword like SVF-1, system run this query:
{
"query": {
"filtered" : {
"query" : {
"query_string" : {
"analyze_wildcard": true,
"query": "*SVF-1*"
}
}
}
}
}
and results;
SVF-123
SVF-234
This is normal, because name field of my documents are analyzed. This splits query into tokens SVF and 1, and SVF matches my documents, although 1 does not match. I have skipped this way. I have create a mapping for my fields make them not_analyzed
{
"mappings":{
"product":{
"properties":{
"name":{
"type":"string",
"index": "not_analyzed"
},
"site":{
"type":"string",
"index": "not_analyzed"
}
}
}
}
}
but my problem continued.
2.) I wanted to try another way after lots of research. Decided to use wildcard query.
My query is;
{
"query": {
"wildcard" : {
"name" : {
"value" : *SVF-1*"
}
}
},
"filter":{
"term": {"site":"pro_en_GB"}
}
}
}
This query worked, but one problem here. My fields are not_analyzed anymore, and I am making wildcard query. Case sensitivity is problem here. If I search like svf-1, it returns nothing. Since, user can input lowercase version of query.
3.) I have changed my document structure to;
{
"mappings":{
"product":{
"properties":{
"name":{
"type":"string",
"index": "not_analyzed"
},
"nameLowerCase":{
"type":"string",
"index": "not_analyzed"
}
"site":{
"type":"string",
"index": "not_analyzed"
}
}
}
}
}
I have adde one more field for name called nameLowerCase. When I am indexing my document, I am setting my document like;
{
name: "SVF-123",
nameLowerCase: "svf-123",
site: "pro_en_GB"
}
Here, I am converting query keyword to lowercase and make search operation on new nameLowerCase index. And displaying name field.
Final version of my query is;
{
"query": {
"wildcard" : {
"nameLowerCase" : {
"value" : "*svf-1*"
}
}
},
"filter":{
"term": {"site":"pro_en_GB"}
}
}
}
Now it works. There is also one way to solve this problem by using multi_field. My query contains dash(-), and faced some problems.
Lots of thanks to #Alex Brasetvik for his detailed explanation and effort
Adding to Hüseyin answer, we can use AND as the default operator. So SVF and 1* will be joined using AND operator, therefore giving us the correct results.
"query": {
"filtered" : {
"query" : {
"query_string" : {
"default_operator": "AND",
"analyze_wildcard": true,
"query": "*SVF-1*"
}
}
}
}
#Viduranga Wijesooriya as you stated "default_operator" : "AND" will check for presence of both SVF and 1 but exact match alone is still not possible,
but ya this will filter the results in more appropriate way leaving with all combination of SVF and 1 and sorting the results by relevance which will promote SVF-1 up the order
For pulling out the exact result
"settings": {
"analysis": {
"analyzer": {
"analyzer_keyword": {
"type": "custom",
"tokenizer": "keyword",
"filter": [
"lowercase"
]
}
}
}
},
"mappings": {
"type": {
"properties": {
"name": {
"type": "string",
"analyzer": "analyzer_keyword"
}
}
}
}
and the query is
{
"query": {
"bool": {
"must": [
{
"query_string" : {
"fields": ["name"],
"query" : "*svf-1*",
"analyze_wildcard": true
}
}
]
}
}
}
result
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "play",
"_type": "type",
"_id": "AVfXzn3oIKphDu1OoMtF",
"_score": 1,
"_source": {
"name": "SVF-123"
}
}
]
}
}
I have an index of documents that look this:
{
url: "/foo/bar",
html_blocks: [
"<h1>hi</h1>"
],
tags: [
"video",
"text"
],
title: "My title"
}
I'd like to query these documents on the title and html_blocks fields, and for any matches add a boost if they have a video tag.
So far, my query looks like this:
{
"query": {
"query_string": {
"query": "foo",
"fields": [
"title",
"html_blocks"
]
}
}
}
How do I modify it so that it continues to only return results if a match is found in the existing query, but a boost is added to any of the results which have a video tag? Thanks!
You want a custom_filters_score which will just boost on matches. Note that filter input is not analyzed, so you might wrap that in a query if you need it analyzed. Your other options to boost, while not really for this case are the boosting query, which is good for demoting results and the custom_score_query which is good for added boosts based on some calculated value.
See: Custom_filters_score
{
"query": {
"custom_filters_score": {
"query": {
"query_string": {
"query": "foo",
"fields": [
"title",
"html_blocks"
]
}
},
"filters": [
{
"filter": {
"term": {
"tags": "video"
}
},
"boost": 3
}
]
}
}
}
Edit:
This is what I mean by wrapping in a query using a filter query. Trust me, once you get the hang of ES, you'll be nested so knee deep that you'll produce some of the most satisfying queries ever.
{
"query": {
"custom_filters_score": {
"query": {
"query_string": {
"query": "foo",
"fields": [
"title",
"html_blocks"
]
}
},
"filters": [
{
"filter": {
//here comes the filter query, and I changed term to match
//since match analyzes
"query":{
"match": {
"tags": "video"
}
}
},
"boost": 3
}
]
}
}
}