I have documents like this.
{
"a":"test",
"b":"harry"
},
{
"a":""
"b":"jack"
}
I need to update docs with field a==""(empty string) to default value say null in all documents for a given index.
Any help is appreciated. Thanks
Use Update by query with ingest
_update_by_query can also use the Ingest Node feature by specifying a pipeline like this:
define the pipeline
PUT _ingest/pipeline/set-foo
{
"description" : "sets foo",
"processors" : [ {
"set" : {
"field": "a",
"value": null
}
} ]
}
then you can use it like:
POST myindex/_update_by_query?pipeline=set-foo
{
"query": {
"filtered": {
"filter": {
"script": {
"script": "_source._content.length() == 0"
}
}
}
}
}'
OR
POST myindex/_update_by_query?pipeline=set-foo
{
"query": {
"bool" : {
"must" : {
"script" : {
"script" : {
"inline": "doc['a'].empty",
"lang": "painless"
}
}
}
}
}
}
To query a documents with empty string field value, i.e = ''
I did,
"query": {
"bool": {
"must": [
{
"exists": {
"field": "a"
}
}
],
"must_not": [
{
"wildcard": {
"a": "*"
}
}
]
}
}
So overall query to update all docs with field a=="" is,
POST test11/_update_by_query
{
"script": {
"inline": "ctx._source.a=null",
"lang": "painless"
},
"query": {
"bool": {
"must": [
{
"exists": {
"field": "a"
}
}
],
"must_not": [
{
"wildcard": {
"a": "*"
}
}
]
}
}
}
Related
I have this query for my elasticsearch request:
{
"query": {
"bool": {
"filter": {
"bool": {
"should" : [
{
"bool" : {
"must_not": {
"exists": {
"field": "visibility_id"
}
}
}
},
{
"bool" : {
"must": {
"terms": {
"visibility.visibility": ["visible"]
}
}
}
}
]
}
}
}
}
}
The goal is to check if the row visibility_id is in the table. If not it will return true has it reach the "must_not". But if the visibility_id column is present it needs to check that this is set to "visible".
At the moment it works if the visibility_id is null but it does not check the terms. terms can be anything else but visible and it will works.
Can someone help me please, I am new to elasticsearch. (I have tried without the filter, bool, only with the should but it does not work neither.)
Try this query, you're missing minimum_should_match: 1
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must_not": {
"exists": {
"field": "visibility_id"
}
}
}
},
{
"terms": {
"visibility.visibility": [
"visible"
]
}
}
]
}
}
}
If visibility is nested in your mapping, your query needs to be like this instead:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must_not": {
"exists": {
"field": "visibility_id"
}
}
}
},
{
"nested": {
"path": "visibility",
"query": {
"terms": {
"visibility.visibility": [
"visible"
]
}
}
}
}
]
}
}
}
I would like to write something like this in ElasticSearch:
SELECT *
FROM ...
WHERE name IS NULL OR name IN ("a","b","c");
I can write the "IS NULL" part using:
{
"query" :
{
"bool" : {
"must_not": {
"exists": {
"field": "name"
}
}
}
}
}
The "IN list" part:
{
"query" :
{
"bool" : {
"should" : [
{
"terms" : {
"name" : [
"a", "b", "c"
]
}
}
]
}
}
}
But I can't find a way to merge these two queries using a OR (and not a AND of course).
Thanks
You can use bool/should in order to combine both
{
"query": {
"bool": {
"should": [
{
"terms": {
"name": [
"a",
"b",
"c"
]
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "name"
}
}
}
}
]
}
}
}
I want build query for search matching with nested and not equal.
This is my elasticSearch query:
{
"from":0,"size":1000,
"query":{
"nested" : {
"path" : "data",
"query" : {
"match" : {
"data.city" : "california"
}
}
},
"filter":{
"not":{
"filter":{
"term":{
"_id":"01921asda01201"
}
}
}
}
}
}
But I got error, am I write something wrong ? thanks
You can use bool Filter too with must and must_not clause.
{
"from": 0,
"size": 1000,
"filter": {
"bool": {
"must": [
{
"nested": {
"path": "data",
"query": {
"match": {
"data.city": "california"
}
}
}
}
],
"must_not": [
{
"term": {
"_id": "01921asda01201"
}
}
]
}
}
}
You need to use filtered query
GET _search
{
"query": {
"filtered": {
"query": {
"nested": {
"path" : "data",
"query" : {
"match" : {
"data.city" : "california"
}
}
}
},
"filter": {
"bool": {
"must_not": [
{
"term": {
"_id": "01921asda01201"
}
}
]
}
}
}
}
}
You should use a bool query for this, and put your two clauses in the must and must_not sections respectively.
If you don't care about scoring on the data.city field (from your example it's not clear), you might want to use the filter portion instead of the must portion.
{
"from": 0,
"size": 1000,
"query": {
"bool": {
"filter": [
{
"nested": {
"path": "data",
"query": {
"match": {
"data.city": "california"
}
}
}
}
],
"must_not": [
{
"term": {
"_id": "01921asda01201"
}
}
]
}
}
}
Is there any way to convert strings to floats when specifying a histogram aggregation? Because I have documents with fields that are floats but are not parsed by elasticsearch as such, and when I attempt to do a sum using a string field It throws the next error.
ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData
cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}]"
I know I could change the mapping, but for the usage case that I have, it would be more handy if I
could specify something like "script : _value.tofloat()" when writing the
aggregation for the field.
This is my code:
{
"query" : {
"bool": {"
must": [
{"match": { "sensorId": "D14UD021808ARZC" }},
{"match": { "variableName": "CAUDAL"}}
]
}
},
"aggs" : {
"caudal_per_month" : {
"date_histogram" : {
"field" : "timestamp",
"interval" : "month"
},
"aggs": {
"totalmonth": {
"sum": {
"field": "value",
"script" : "_value*1.0"
}
}
}
}
}
}
You need this
{
"query": {
"bool": {
"must": [
{
"match": {
"sensorId": "D14UD021808ARZC"
}
},
{
"match": {
"variableName": "CAUDAL"
}
}
]
}
},
"aggs": {
"caudal_per_month": {
"date_histogram": {
"field": "timestamp",
"interval": "month"
},
"aggs": {
"totalmonth": {
"sum": {
"script": "Float.parseFloat(doc['value'].value)"
}
}
}
}
}
}
For a field that's called value: Float.parseFloat(doc['value'].value)
I'm trying to search my database and be able to use upper/lower case filter terms but I've noticed while query's apply analyzers, I can't figure out how to apply a lowercase analyzer on a filtered search. Here's the query:
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"language": "mandarin" // Returns a doc
}
},
{
"term": {
"language": "Italian" // Does NOT return a doc, but will if lowercased
}
}
]
}
}
}
}
}
I have a type languages that I have lowercased using:
"analyzer": {
"lower_keyword": {
"type": "custom",
"tokenizer": "keyword",
"filter": "lowercase"
}
}
and a corresponding mapping:
"mappings": {
"languages": {
"_id": {
"path": "languageID"
},
"properties": {
"languageID": {
"type": "integer"
},
"language": {
"type": "string",
"analyzer": "lower_keyword"
},
"native": {
"type": "string",
"analyzer": "keyword"
},
"meta": {
"type": "nested"
},
"language_suggest": {
"type": "completion"
}
}
}
}
The problem is that you have a field that you have analyzed during index to lowercase it, but you are using a term filter for the query which is not analyzed:
Term Filter
Filters documents that have fields that contain a term (not analyzed).
Similar to term query, except that it acts as a filter.
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-filter.html
I'd try using a query filter instead:
Query Filter
Wraps any query to be used as a filter. Can be placed within queries
that accept a filter.
Example:
{
"constantScore" : {
"filter" : {
"query" : {
"query_string" : {
"query" : "this AND that OR thus"
}
}
}
} }
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-filter.html#query-dsl-query-filter
This may be achieved by appending .keyword to your field to query against the keyword version of the field. Assuming language was defined in the mapping with type keyword.
Note that now only the exact text would match: mandarin won't match and Italian would.
Your query would end up like this:
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"language.keyword": "mandarin" // Returns Empty
}
},
{
"term": {
"language.keyword": "Italian" // Returns Italian.
}
}
]
}
}
}
}
}
Combining the term values is also allowed:
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"language.keyword":
["mandarin", "Italian"]
}
}
]
}
}
}
}
}