Is there any way we can convert query string using qs npm to elastic cloud query string ? which support all kind of nested query params, greater than, less than, values between ranges all kind of queries ?
any npm module, git hub code or code snippet which does this ?
Example : for my api I am supporting all kind of query params supported by qs npm module, where as my search api is fetching from elastic cloud.
so in case I get query as
?title=Resurgence&causes=General&location.city=Mysuru&latitude=12.9803047&longitude=77.62949349999997&distance=50000&pageNumber=0&pageSize=10& organisation._id=5c36dd152edb5028fd7655a4
then the query string will be converted into json object by qs npm module and it gives a json object as follows
{"title":"Resurgence","causes":"General","location.city":"Mysuru","latitude":"12.9803047","longitude":"77.62949349999997","distance":"50000","pageNumber":"0","pageSize":"10","organisation._id":"5c36dd152edb5028fd7655a4"}
Where as while searching in elastic search I suppose to convert the same into elastic search query object as follows.
So I was just looking for a npm module which will convert query string json of qs module into elastic cloud query body?
{
"index": "researchArticles",
"from": 0,
"size": 10,
"body": {
"query": {
"bool": {
"must": [
{
"match": {
"status": "PUBLISHED"
}
},
{
"match": {
"title": {
"query": "Resurgence",
"operator": "or",
"prefix_length": 1
}
}
},
{
"match": {
"causes": {
"query": "General",
"operator": "or",
"prefix_length": 1
}
}
},
{
"match": {
"location.city": {
"query": "Bengaluru",
"operator": "or",
"prefix_length": 1
}
}
},
{
"match": {
"pageNumber": {
"query": "0",
"operator": "or",
"prefix_length": 1
}
}
},
{
"match": {
"pageSize": {
"query": "10",
"operator": "or",
"prefix_length": 1
}
}
},
{
"match": {
"organisation._id": {
"query": "5c36dd152edb5028fd7655a4",
"operator": "or",
"prefix_length": 1
}
}
}
],
"should": {
"range": {
"startDate": {
"gte": "now"
}
}
},
"filter": {
"geo_distance": {
"distance": "50000",
"location.geoSpatial": {
"lat": "12.9803047",
"lon": "77.62949349999997"
}
}
}
}
},
"sort": [
{
"_geo_distance": {
"location.geoSpatial": "12.9803047,77.62949349999997",
"order": "asc",
"unit": "m"
}
}
]
}
}
Related
I was implementing fuzzy search in my existing elasticsearch where I can't change mappings, I was hoping if there is any way I can convert the following query in fuzzy one i.e add fuzzy search on fields lower_name and album
{
"query": {
"bool": {
"must": [
{
"term": {
"user": "userId"
}
},
{
"bool": {
"should": [
{
"terms": {
"lower_name": ["search", "Text"]
}
},
{
"terms": {
"album": ["search","Text"]
}
}
]
}
}
]
}
}
}
I tried this :
{
"query": {
"bool": {
"must": [
{
"term": {
"user": "userId"
}
},
{
"bool": {
"should": [
{
"fuzzy": {
"lower_name": ["search","Text"]
}
},
{
"fuzzy": {
"album": ["search","Text"]
}
}
]
}
}
]
}
}
}
But this is giving error: [fuzzy] query doesn't support multiple fields
Please help!
Using Elasticsearch 6.3
You can use a multi_match query with fuzziness. Try out the below query
Index Data:
{
"user": "ben",
"lower_name": "def",
"album": "Brenda"
}
{
"user": "ben",
"lower_name": "abc",
"album": "Brenda"
},
{
"user": "ben",
"lower_name": "fgh",
"album": "honda"
}
Search Query:
{
"query": {
"bool": {
"must": [
{
"term": {
"user": "ben"
}
},
{
"bool": {
"should": [
{
"multi_match": {
"query": "abc dey",
"fields": [
"lower_name"
],
"fuzziness": "auto"
}
},
{
"multi_match": {
"query": "brenda",
"fields": [
"album"
],
"fuzziness": "auto"
}
}
]
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "66311552",
"_type": "_doc",
"_id": "2",
"_score": 0.7497801,
"_source": {
"user": "ben",
"lower_name": "def",
"album": "Brenda"
}
},
{
"_index": "66311552",
"_type": "_doc",
"_id": "1",
"_score": 0.7497801,
"_source": {
"user": "ben",
"lower_name": "abc",
"album": "Brenda"
}
}
]
You can easily use the "fuzziness": "AUTO". param in your search query. Refer fuzziness in match query official example
I am new to ElasticSearch and looking for bool query to pass it to get the data from elasticsearch in spark scala code.
Here is my query:
Get all records for the eventName = "XXXXXX" and date between("1438367180542","1738367180542")
Could you please help me to write the elasticsearch query. Below is the one I tried but its giving error.
GET _search
{
"query": {
"bool": {
"must": [
{
"range": {
"date": {
"gte": "1438367180542",
"lte": "1738367180542"
}
}
}
],
"term": {
"eventName.keyword": "XXXXXXX"
}
}
}
}
Here is the error message:
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[bool] query does not support [term]",
"line": 15,
"col": 19
}
],
"type": "parsing_exception",
"reason": "[bool] query does not support [term]",
"line": 15,
"col": 19
},
"status": 400
}
You're almost there! As you can see your range query is surrounded by curly braces, but your term query isn't and is out of the must array. Simply add those and move it to the must array and it will work. Even better use filter instead of must since you only have filters (i.e. you're not interested in scoring here)
GET _search
{
"query": {
"bool": {
"filter": [
{
"range": {
"date": {
"gte": "1438367180542",
"lte": "1738367180542"
}
}
},
{
"term": {
"eventName.keyword": "XXXXXXX"
}
}
]
}
}
}
I have 20 documents and i'm performing aggregation based on reportid. I need top 10 aggregation based on time in descending. But the response is very random. What am i missing? I'm using elasticsearch 6.2.2 and node.js 4.5. Below here is the body search query for elasticsearch request.
{
"size": 0,
"sort": [
{
"triggerDate":
{
"order": "desc"
}
}],
"query":
{
"bool":
{
"must": [
{
"query_string":
{
"query": "*",
"analyze_wildcard": true
}
},
{
"range":
{
"triggerDate":
{
"gte": fromTime,
"lte": toTime
}
}
}
],
"must_not": [
{
"query_string":
{
"query": "reportId.keyword:\"\"",
"analyze_wildcard": true
}
}]
}
},
"_source":
{
"excludes": []
},
"aggs":
{
"reportid":
{
"terms":
{
"field": "reportId.keyword",
"size": 10
}
}
}
I think what you need to do is aggregate on reportId.keyword and sort aggregation by date.
So here is the solution
{
"size": 0,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "*",
"analyze_wildcard": true
}
},
{
"range": {
"triggerDate": {
"gte": fromTime,
"lte": toTime
}
}
}
],
"must_not": [
{
"query_string": {
"query": "reportId.keyword:\"\"",
"analyze_wildcard": true
}
}
]
}
},
"_source": {
"excludes": []
},
"aggs": {
"reportid": {
"terms": {
"field": "reportId.keyword",
"size": 10,
"order": {
"2-orderAgg": "desc"
}
},
"aggs": {
"2-orderAgg": {
"max": {
"field": "triggerDate"
}
}
}
}
}
}
You need to sort the aggregation results by a custom aggregation and not the query results.
I have a query like so:
{
"sort": [
{
"_geo_distance": {
"geo": {
"lat": 39.802763999999996,
"lon": -105.08748399999999
},
"order": "asc",
"unit": "mi",
"mode": "min",
"distance_type": "sloppy_arc"
}
}
],
"query": {
"bool": {
"minimum_number_should_match": 0,
"should": [
{
"match": {
"name": ""
}
},
{
"match": {
"credit": true
}
}
]
}
}
}
I want my search to always return ALL results, just sorted with those which have matching flags closer to the top.
I would like the sorting priority to go something like:
searchTerm (name, a string)
flags (credit/atm/ada/etc, boolean values)
distance
How can this be achieved?
So far, the query you see above is all I've gotten. I haven't been able to figure out how to always return all results, nor how to incorporate the additional queries into the sort.
I don't believe "sort" is the answer you are looking for, actually. I believe you need a trial-and-error approach starting with a simple "bool" query where you put all your criterias (name, flags, distance). Then you give your name criteria more weight (boost) then a little bit less to your flags and even less to the distance calculation.
A "bool" "should" would be able to give you a sorted list of documents based on the _score of each and, depending on how you score each criteria, the _score is being influenced more or less.
Also, returning ALL the elements is not difficult: just add a "match_all": {} to your "bool" "should" query.
This would be a starting point, from my point of view, and, depending on your documents and your requirements (see my comment to your post about the confusion) you would need to adjust the "boost" values and test, adjust again and test again etc:
{
"query": {
"bool": {
"should": [
{ "constant_score": {
"boost": 6,
"query": {
"match": { "name": { "query": "something" } }
}
}},
{ "constant_score": {
"boost": 3,
"query": {
"match": { "credit": { "query": true } }
}
}},
{ "constant_score": {
"boost": 3,
"query": {
"match": { "atm": { "query": false } }
}
}},
{ "constant_score": {
"boost": 3,
"query": {
"match": { "ada": { "query": true } }
}
}},
{ "constant_score": {
"query": {
"function_score": {
"functions": [
{
"gauss": {
"geo": {
"origin": {
"lat": 39.802763999999996,
"lon": -105.08748399999999
},
"offset": "2km",
"scale": "3km"
}
}
}
]
}
}
}
},
{
"match_all": {}
}
]
}
}
}
I'm trying to search my database and be able to use upper/lower case filter terms but I've noticed while query's apply analyzers, I can't figure out how to apply a lowercase analyzer on a filtered search. Here's the query:
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"language": "mandarin" // Returns a doc
}
},
{
"term": {
"language": "Italian" // Does NOT return a doc, but will if lowercased
}
}
]
}
}
}
}
}
I have a type languages that I have lowercased using:
"analyzer": {
"lower_keyword": {
"type": "custom",
"tokenizer": "keyword",
"filter": "lowercase"
}
}
and a corresponding mapping:
"mappings": {
"languages": {
"_id": {
"path": "languageID"
},
"properties": {
"languageID": {
"type": "integer"
},
"language": {
"type": "string",
"analyzer": "lower_keyword"
},
"native": {
"type": "string",
"analyzer": "keyword"
},
"meta": {
"type": "nested"
},
"language_suggest": {
"type": "completion"
}
}
}
}
The problem is that you have a field that you have analyzed during index to lowercase it, but you are using a term filter for the query which is not analyzed:
Term Filter
Filters documents that have fields that contain a term (not analyzed).
Similar to term query, except that it acts as a filter.
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-filter.html
I'd try using a query filter instead:
Query Filter
Wraps any query to be used as a filter. Can be placed within queries
that accept a filter.
Example:
{
"constantScore" : {
"filter" : {
"query" : {
"query_string" : {
"query" : "this AND that OR thus"
}
}
}
} }
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-filter.html#query-dsl-query-filter
This may be achieved by appending .keyword to your field to query against the keyword version of the field. Assuming language was defined in the mapping with type keyword.
Note that now only the exact text would match: mandarin won't match and Italian would.
Your query would end up like this:
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"language.keyword": "mandarin" // Returns Empty
}
},
{
"term": {
"language.keyword": "Italian" // Returns Italian.
}
}
]
}
}
}
}
}
Combining the term values is also allowed:
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"language.keyword":
["mandarin", "Italian"]
}
}
]
}
}
}
}
}