I have the following mapping
{
"cloth": {
"dynamic" : false,
"_source" : {"enabled" : false },
"properties": {
"name": {
"type": "string",
"index": "analyzed"
},
"variation": {
"type": "nested",
"properties": {
"size": {
"type": "string",
"index": "not_analyzed"
},
"color": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
I am not able to figure out a way to retrieve the nested object fields using the fields query.
{
"fields" : ["name" , "variation.size", "variation.color"],
"query" : {
"nested" : {
"path" : "variation",
"query" : {
"bool" : {
"must" : [
{ "term" : { "variation.size" : "XXL" } },
{ "term" : { "variation.color" : "red" } }
]
}
}
}
}
}
The above query returns
"_id" : "1",
"_score" : 1.987628,
"fields" : {
"variation.size" : [ "XXL", "XL" ],
"variation.color" : [ "red", "black" ],
"name" : [ "Test shirt" ]
}
When I tried
"fields" : ["name" , "variation"]
I got the error
status: 400
reason: "ElasticsearchIllegalArgumentException[field [variation] isn't a leaf field]"
Which is as expected.
How can I get the variation object as it is?
Expected Result. I need to retrieve the variable object as a whole so that I can preserve the association of size and color. Like "red" with "XXL".
"variation" : { "XXL" , "red"}
Update: Source is disabled for this Index Type.
If you use Source Filtering it will return the nested objects as a whole, your query would be:
{
"_source": [
"name",
"variation"
],
"query": {
"nested": {
"path": "variation",
"query": {
"bool": {
"must": [
{
"term": {
"variation.size": "XXL"
}
},
{
"term": {
"variation.color": "red"
}
}
]
}
}
}
}
}
You should use this:
"script_fields": {
"variation": {
"script": {
"inline": "doc['variation.size'].value + ' ' + doc['variation.red'].value"
}
}
}
I use elasticsearch v. 5.1.1
Related
I created an index in elasticsearch 6.5.1 successfully loaded the data to that index. there is one field "submitted_date" which is the timestamp. below is the mapping like of this field.
"submitted_date": { "type": "date", "format":"yyyy-MM-dd HH:mm:ss.SSS" },
then I created the index pattern. I used the Time Filter field name as "submitted_date". after that, I tried to check the data in Discover tab, but data are not showing. there is a message saying that No results match your search criteria.
NOTE that I have changed the time in time range picker in every possible way which is on top of the right corner in kibana dashboard.
data appear in Dev Tools tab with elastic queries.
ps : I inserted the data using nodejs with elasticsearch official library, did not used logstash.
I followed this article, but it did not help me.
UPDATE : sample document
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 10480,
"max_score" : 1.0,
"hits" : [
{
"_index" : "test",
"_type" : "tests",
"_id" : "1214334",
"_score" : 1.0,
"_source" : {
"priority" : "4",
"submitted_date" : "2018-01-04T18:32:21.000Z",
"submitted_month" : 0,
"submitted_month_name" : "January",
"submitted_day" : 4,
"submitted_weekday" : "Tuesday",
"submitted_hour" : 18,
"submitted_year_month" : "2018-0",
"submitted_year_month_name" : "2018-January",
"date_key" : "20180104",
"year_month_key" : "201801",
"status" : "Closed"
}
}
]
}
}
Inspect request
{
"version": true,
"size": 500,
"sort": [
{
"_score": {
"order": "desc"
}
}
],
"_source": {
"excludes": []
},
"aggs": {
"2": {
"date_histogram": {
"field": "submitted_date",
"interval": "1d",
"time_zone": "Asia/Kolkata",
"min_doc_count": 1
}
}
},
"stored_fields": [
"*"
],
"script_fields": {},
"docvalue_fields": [
{
"field": "close_date",
"format": "date_time"
},
{
"field": "last_modified_date",
"format": "date_time"
},
{
"field": "last_resolved_date",
"format": "date_time"
},
{
"field": "submitted_date",
"format": "date_time"
},
{
"field": "time_to_resolve",
"format": "date_time"
}
],
"query": {
"bool": {
"must": [
{
"match_all": {}
},
{
"range": {
"submitted_date": {
"gte": 1514745000000,
"lte": 1543937620414,
"format": "epoch_millis"
}
}
}
],
"filter": [],
"should": [],
"must_not": []
}
},
"highlight": {
"pre_tags": [
"#kibana-highlighted-field#"
],
"post_tags": [
"#/kibana-highlighted-field#"
],
"fields": {
"*": {}
},
"fragment_size": 2147483647
}
}
Index pattern
function _putMapping() {
return client.indices.create({
index: process.env.ELASTICSEARCH_INDEX,
body: {
settings:{
index:{
"number_of_shards": 1,
"number_of_replicas": 5
},
"index.mapping.ignore_malformed" : true
},
mappings:{
tests:{
properties:{
"last_modified_date": { "type": "date" },
"last_resolved_date": { "type": "date" },
"time_to_resolve": { "type": "date" },
"submitted_date": { "type": "date", "format":"yyyy-MM-dd HH:mm:ss.SSS" },
"date_key": { "type": "integer" },
"priority": { "type": "long" },
"submitted_hour": { "type": "long" },
"submitted_month": { "type": "long" },
"submitted_year": { "type": "long" },
"submitted_year": { "type": "keyword" },
"submitted_year_month": { "type": "keyword" },
"submitted_year_month_name": { "type": "keyword" },
}
}
}
}
});
}
Your mYour submitted_date is coming like 2018-01-04T18:32:21.000Z but your mapping is set as yyyy-MM-dd HH:mm:ss.SSS.
You need to change it to "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'".
I would like to write something like this in ElasticSearch:
SELECT *
FROM ...
WHERE name IS NULL OR name IN ("a","b","c");
I can write the "IS NULL" part using:
{
"query" :
{
"bool" : {
"must_not": {
"exists": {
"field": "name"
}
}
}
}
}
The "IN list" part:
{
"query" :
{
"bool" : {
"should" : [
{
"terms" : {
"name" : [
"a", "b", "c"
]
}
}
]
}
}
}
But I can't find a way to merge these two queries using a OR (and not a AND of course).
Thanks
You can use bool/should in order to combine both
{
"query": {
"bool": {
"should": [
{
"terms": {
"name": [
"a",
"b",
"c"
]
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "name"
}
}
}
}
]
}
}
}
I have a parent-children relationship in my ES mapping and I want to filter the parents by the value of an aggregation (avg) on their children. That is, I only want to retrieve parents where that value is within a given range.
I tried to do it with aggs and post-filters but couldn't get it to work.
{
"apartments" : {
"mappings" : {
"apartment_availability" : {
"_parent" : {
"type" : "apartment"
},
"_routing" : {
"required" : true
},
"properties" : {
"availability_date" : {
"type" : "date"
},
"apartment_id" : {
"type" : "long"
},
"id" : {
"type" : "long"
},
"price_cents" : {
"type" : "long"
},
"status" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"apartment" : {
"properties" : {
"id" : {
"type" : "long"
},
}
}
}
}
}
If oru users select a period of March 24th-March 31st and a price range of €150-€300, then we want to show them all apartments that are free in that period and whose average price for that period is in the €150-€300 range.
Here's what we have so far:
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [{
"has_child": {
"type": "apartment_availability",
"min_children": 8,
"max_children": 8,
"query": {
"bool": {
"must": [{
"term": {
"status": "available"
}
}, {
"range": {
"availability_date": {
"gte": "2017-03-24",
"lte": "2017-03-31"
}
}
}]
}
}
}
}]
}
}
}
}
}
My suggestion, using bucket_selector aggregation to choose between apartments:
GET /apartments/apartment/_search
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"has_child": {
"type": "apartment_availability",
"query": {
"bool": {
"must": [
{
"term": {
"status": "available"
}
},
{
"range": {
"availability_date": {
"gte": "2017-04-01",
"lte": "2017-04-03"
}
}
}
]
}
}
}
}
]
}
}
}
},
"aggs": {
"apartments_ids": {
"terms": {
"field": "id",
"size": 10
},
"aggs": {
"avails": {
"children": {
"type": "apartment_availability"
},
"aggs": {
"filter_avails": {
"filter": {
"bool": {
"must": [
{
"term": {
"status": "available"
}
},
{
"range": {
"availability_date": {
"gte": "2017-04-01",
"lte": "2017-04-03"
}
}
}
]
}
},
"aggs": {
"average": {
"avg": {
"field": "price_cents"
}
}
}
}
}
},
"avg_bucket_filter": {
"bucket_selector": {
"buckets_path": {
"avg": "avails>filter_avails.average"
},
"script": "params.avg > 150 && params.avg < 300"
}
}
}
}
}
}
Is there any way to convert strings to floats when specifying a histogram aggregation? Because I have documents with fields that are floats but are not parsed by elasticsearch as such, and when I attempt to do a sum using a string field It throws the next error.
ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData
cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}]"
I know I could change the mapping, but for the usage case that I have, it would be more handy if I
could specify something like "script : _value.tofloat()" when writing the
aggregation for the field.
This is my code:
{
"query" : {
"bool": {"
must": [
{"match": { "sensorId": "D14UD021808ARZC" }},
{"match": { "variableName": "CAUDAL"}}
]
}
},
"aggs" : {
"caudal_per_month" : {
"date_histogram" : {
"field" : "timestamp",
"interval" : "month"
},
"aggs": {
"totalmonth": {
"sum": {
"field": "value",
"script" : "_value*1.0"
}
}
}
}
}
}
You need this
{
"query": {
"bool": {
"must": [
{
"match": {
"sensorId": "D14UD021808ARZC"
}
},
{
"match": {
"variableName": "CAUDAL"
}
}
]
}
},
"aggs": {
"caudal_per_month": {
"date_histogram": {
"field": "timestamp",
"interval": "month"
},
"aggs": {
"totalmonth": {
"sum": {
"script": "Float.parseFloat(doc['value'].value)"
}
}
}
}
}
}
For a field that's called value: Float.parseFloat(doc['value'].value)
I am developing an application that uses elastic search, and in some case I want to make a search that according to term and locales. I am testing this on localhost
http://localhost:9200/index/type/_search
and parameters
query : {
wildcard : { "term" : "myterm*" }
},
filter : {
and : [
{
term : { "lang" : "en" }
},
{
term : { "translations.lang" : "tr" } //this is subdocument search
},
]
}
Here is an example document:
{
"_index": "twitter",
"_type": "tweet",
"_id": "5084151d2c6e5d5b11000008",
"_score": null,
"_source": {
"lang": "en",
"term": "photograph",
"translations": [
{
"_id": "5084151d2c6e5d5b11000009",
"lang": "tr",
"translation": "fotoğraf",
"score": "0",
"createDate": "2012-10-21T15:30:37.994Z",
"author": "anonymous"
},
{
"_id": "50850346532b865c2000000a",
"lang": "tr",
"translation": "resim",
"score": "0",
"createDate": "2012-10-22T08:26:46.670Z",
"author": "anonymous"
}
],
"author": "anonymous",
"createDate": "2012-10-21T15:30:37.994Z"
}
}
I am trying to get terms with wildcard(for autocomplete) with input language "en", and output language "tr". It is getting terms that has "myterm" but doesnt apply, and operation on this. Any suggestion would be appreciated
Thanks in advance
I would guess that the translations element has nested type. If this is the case, you should use nested query:
curl -XPOST "http://localhost:9200/twitter/tweet/_search" -d '{
query: {
wildcard: {
"term": "term*"
}
},
filter: {
and: [{
term: {
"lang": "en"
}
}, {
"nested": {
"path": "translations",
"query": {
"term" : { "translations.lang" : "tr" }
}
}
}]
}
}'
I have manage to solve my problem with following query;
query : {
wildcard : { "term" : "myterm*" }
},
filter : {
and : [
{
term : { "lang" : "en" }
},
{
term : { "translations.lang" : "tr" } //this is subdocument search
}
]
},
sort : {
{"term" : "desc"}
}
An important point here is, you need to set your sorting field as not_analyzed. Since, you cannot sort a field that is analyzed.