Elasticsearch: transform date with groovy script - groovy

Question

i have the following (simplified) mapping:
{
"event_date": {
"_source": { "enabled": true },
"_all": { "enabled": true },
"dynamic": "strict",
"properties": {
"start_date_time": {
"type": "date",
"format": "dateOptionalTime"
},
"start_date_day": {
"type": "date",
"format": "dateOptionalTime",
"index": "not_analyzed"
}
}
}
}
The indexed objects will look like this:
{
"start_date_time": "2017-05-08T18:23:45+0200"
}
The property start_date_day should always contain the same date, but with the time set to 00:00:00. In the example above start_date_day must be "2017-05-08T00:00:00+0200".
I think,it is possible to achieve this with a transform mapping and a groovy script, but my developed groovy code did not work in the elasticsearch-context and I am not that familiar with the groovy language.
Maybe someone has an idea on how to solve this?

Answer 1

Yes this is doable, for testing/running you might need to turn on script.groovy.sandbox.enabled: true in ../conf/elasticsearch.yml first.
PUT datetest/
{
"mappings": {
"event_date": {
"_source": { "enabled": true },
"_all": { "enabled": true },
"dynamic": "strict",
"transform" : {
"script" : "ctx._source['start_date_day'] = new Date().parse(\"yyyy-MM-dd\", ctx._source['start_date_time']).format(\"yyyy-MM-dd\");",
"lang": "groovy"
},
"properties": {
"start_date_time": {
"type": "date",
"format": "dateOptionalTime"
},
"start_date_day": {
"type": "date",
"format": "dateOptionalTime",
"index": "not_analyzed",
"store": "yes"
}
}
}
}
}
Sample data:
PUT /datetest/event_date/1
{
"start_date_time": "2017-05-08T18:23:45+0200"
}
Sample output:
GET /datetest/event_date/_search
{
"query": {
"match_all": {}
},
"fields": ["start_date_time","start_date_day"]
}
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "datetest",
"_type": "event_date",
"_id": "1",
"_score": 1,
"fields": {
"start_date_day": [
"2017-05-08T00:00:00.000Z"
],
"start_date_time": [
"2017-05-08T18:23:45+0200"
]
}
}
]
}
}

Answer 2

i think it is the format which isn't good. "date" seems to be MM/DD/YYYY and only this. If you want the time you need datetime format.
I have found this link which can help you : Elastic date format
You can try to change type "date" to "basic_date_time".

Answer 3

Based on your comments, since you don't need the hour part, you can simply define the mapping for the start_date_day field using the date format and use the following transform like this:
{
"event_date": {
"_source": {
"enabled": true
},
"_all": {
"enabled": true
},
"dynamic": "strict",
"transform": {
"script": "ctx._source['start_date_day'] = ctx._source['start_date_time'].split('T')[0]",
"lang": "groovy"
},
"properties": {
"start_date_time": {
"type": "date",
"format": "dateOptionalTime"
},
"start_date_day": {
"type": "date",
"format": "date",
"index": "not_analyzed"
}
}
}
}
ES will only store the date part and leave the hours and timezone out.
However, you should note that when using transform, the original source is stored without modification, the result of the transform will be indexed though, and hence searchable.