handle empty date in elasticsearch dynamically - azure

Question

I have below dynamic mapping template.
PUT my_index
{
"mappings": {
"dynamic_templates": [
{
"objects": {
"match_mapping_type": "object",
"mapping": {
"type": "nested"
}
}
}
],
"dynamic_date_formats": ["yyyy-MM-dd" , "yyyy-MM-dd HH:mm:ss"]
}
}
Only problem is when I am having empty date it is throwing error. I just want to ignore empty dates. My data having multiple date fields hence don't want to do mapping for each date fields.
Below is the error I am getting:
org.elasticsearch.hadoop.rest.EsHadoopRemoteException: illegal_argument_exception: mapper [pb_bureau.applications.accounts.dateclosed] of different type, current_type [text], merged_type [date]
{"index":{"_id":"02ade9b5-1ca5-4006-ab06-9c96439e7d02"}}
below date we are inserting: blank field is null value of date
select date1, date2 from cbl_application_credit_report_account ;
2014-11-14
2018-03-31
2012-07-27 2012-07-23
2015-11-30
2017-08-04 2016-05-13
below is mapping which I am applying:
PUT my_index
{
"mappings": {
"dynamic_templates": [
{
"objects": {
"match_mapping_type": "object",
"mapping": {
"type": "nested"
}
}
},
{
"dates_ignore_malformed": {
"path_match": "*",
"match_mapping_type": "date",
"mapping": {
"format": "yyyy-MM-dd||yyyy-MM-dd HH:mm:ss",
"ignore_malformed": true
}
}
}
],
"dynamic_date_formats": ["yyyy-MM-dd" , "yyyy-MM-dd HH:mm:ss"]
}
}
Is there any way in the dynamic mapping to ignore empty dates?

Answer 1

Mapping:
PUT my_index4
{
"mappings": {
"dynamic_templates": [
{
"objects": {
"match_mapping_type": "object",
"mapping": {
"type": "nested"
}
}
},
{
"dates_ignore_malformed": {
"path_match": "*",
"match_mapping_type": "date",
"mapping": {
"format": "yyyy-MM-dd||yyyy-MM-dd HH:mm:ss" ---> date format on which to be applied ,
"ignore_malformed": true ---> Ignores if field s malformed
}
}
}
],
"dynamic_date_formats": [
"yyyy-MM-dd",
"yyyy-MM-dd HH:mm:ss"
]
}
}
Data:
POST my_index4/_doc
{
"date":"2019-01-01 04:30:22",
"Id":1
}
POST my_index4/_doc
{
"name":2,
"date":"2019-01-01"
}
POST my_index4/_doc
{
"name":2,
"date":""
}
Query:
GET my_index4/_search
Result:
"hits" : [
{
"_index" : "my_index4",
"_type" : "_doc",
"_id" : "NT5XSG0BbzgYofLxTDZ_",
"_score" : 1.0,
"_source" : {
"date" : "2019-01-01 04:30:22",
"Id" : 1
}
},
{
"_index" : "my_index4",
"_type" : "_doc",
"_id" : "Nj5XSG0BbzgYofLxUTaT",
"_score" : 1.0,
"_source" : {
"name" : 2,
"date" : "2019-01-01"
}
},
{
"_index" : "my_index4",
"_type" : "_doc",
"_id" : "Nz5XSG0BbzgYofLxWDYi",
"_score" : 1.0,
"_ignored" : [
"date"
],
"_source" : {
"name" : 2,
"date" : ""
}
}
]