I have a certain use case that I'm researching involving creating a query that returns events whenever a failed login occurs from two separate accounts on one machine.
I've created the following query, but I'm receiving errors whenever I try to run it.
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter" : {
"term": {
"EventID": "4625"
}
},
"filter" : {
"range" : {
"_timestamp" : {
"gt": "now-15m"
}
}
}
}
},
aggs: {
group_by_host: {
terms: {
field: 'hostname'
},
aggs: {
group_by_user: {
terms: {
field: 'username'
}
}
}
}
}
Any
You need to wrap your filters inside bool. You can refer to docs for more info.
Try this
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"EventID": "4625"
}
},
{
"range": {
"_timestamp": {
"gt": "now-15m"
}
}
}
]
}
}
}
},
"aggs": {
"group_by_host": {
"terms": {
"field": "hostname"
},
"aggs": {
"group_by_user": {
"terms": {
"field": "username"
}
}
}
}
}
}
Related
I have this query for my elasticsearch request:
{
"query": {
"bool": {
"filter": {
"bool": {
"should" : [
{
"bool" : {
"must_not": {
"exists": {
"field": "visibility_id"
}
}
}
},
{
"bool" : {
"must": {
"terms": {
"visibility.visibility": ["visible"]
}
}
}
}
]
}
}
}
}
}
The goal is to check if the row visibility_id is in the table. If not it will return true has it reach the "must_not". But if the visibility_id column is present it needs to check that this is set to "visible".
At the moment it works if the visibility_id is null but it does not check the terms. terms can be anything else but visible and it will works.
Can someone help me please, I am new to elasticsearch. (I have tried without the filter, bool, only with the should but it does not work neither.)
Try this query, you're missing minimum_should_match: 1
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must_not": {
"exists": {
"field": "visibility_id"
}
}
}
},
{
"terms": {
"visibility.visibility": [
"visible"
]
}
}
]
}
}
}
If visibility is nested in your mapping, your query needs to be like this instead:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must_not": {
"exists": {
"field": "visibility_id"
}
}
}
},
{
"nested": {
"path": "visibility",
"query": {
"terms": {
"visibility.visibility": [
"visible"
]
}
}
}
}
]
}
}
}
I have documents like this.
{
"a":"test",
"b":"harry"
},
{
"a":""
"b":"jack"
}
I need to update docs with field a==""(empty string) to default value say null in all documents for a given index.
Any help is appreciated. Thanks
Use Update by query with ingest
_update_by_query can also use the Ingest Node feature by specifying a pipeline like this:
define the pipeline
PUT _ingest/pipeline/set-foo
{
"description" : "sets foo",
"processors" : [ {
"set" : {
"field": "a",
"value": null
}
} ]
}
then you can use it like:
POST myindex/_update_by_query?pipeline=set-foo
{
"query": {
"filtered": {
"filter": {
"script": {
"script": "_source._content.length() == 0"
}
}
}
}
}'
OR
POST myindex/_update_by_query?pipeline=set-foo
{
"query": {
"bool" : {
"must" : {
"script" : {
"script" : {
"inline": "doc['a'].empty",
"lang": "painless"
}
}
}
}
}
}
To query a documents with empty string field value, i.e = ''
I did,
"query": {
"bool": {
"must": [
{
"exists": {
"field": "a"
}
}
],
"must_not": [
{
"wildcard": {
"a": "*"
}
}
]
}
}
So overall query to update all docs with field a=="" is,
POST test11/_update_by_query
{
"script": {
"inline": "ctx._source.a=null",
"lang": "painless"
},
"query": {
"bool": {
"must": [
{
"exists": {
"field": "a"
}
}
],
"must_not": [
{
"wildcard": {
"a": "*"
}
}
]
}
}
}
I have a parent-children relationship in my ES mapping and I want to filter the parents by the value of an aggregation (avg) on their children. That is, I only want to retrieve parents where that value is within a given range.
I tried to do it with aggs and post-filters but couldn't get it to work.
{
"apartments" : {
"mappings" : {
"apartment_availability" : {
"_parent" : {
"type" : "apartment"
},
"_routing" : {
"required" : true
},
"properties" : {
"availability_date" : {
"type" : "date"
},
"apartment_id" : {
"type" : "long"
},
"id" : {
"type" : "long"
},
"price_cents" : {
"type" : "long"
},
"status" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"apartment" : {
"properties" : {
"id" : {
"type" : "long"
},
}
}
}
}
}
If oru users select a period of March 24th-March 31st and a price range of €150-€300, then we want to show them all apartments that are free in that period and whose average price for that period is in the €150-€300 range.
Here's what we have so far:
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [{
"has_child": {
"type": "apartment_availability",
"min_children": 8,
"max_children": 8,
"query": {
"bool": {
"must": [{
"term": {
"status": "available"
}
}, {
"range": {
"availability_date": {
"gte": "2017-03-24",
"lte": "2017-03-31"
}
}
}]
}
}
}
}]
}
}
}
}
}
My suggestion, using bucket_selector aggregation to choose between apartments:
GET /apartments/apartment/_search
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"has_child": {
"type": "apartment_availability",
"query": {
"bool": {
"must": [
{
"term": {
"status": "available"
}
},
{
"range": {
"availability_date": {
"gte": "2017-04-01",
"lte": "2017-04-03"
}
}
}
]
}
}
}
}
]
}
}
}
},
"aggs": {
"apartments_ids": {
"terms": {
"field": "id",
"size": 10
},
"aggs": {
"avails": {
"children": {
"type": "apartment_availability"
},
"aggs": {
"filter_avails": {
"filter": {
"bool": {
"must": [
{
"term": {
"status": "available"
}
},
{
"range": {
"availability_date": {
"gte": "2017-04-01",
"lte": "2017-04-03"
}
}
}
]
}
},
"aggs": {
"average": {
"avg": {
"field": "price_cents"
}
}
}
}
}
},
"avg_bucket_filter": {
"bucket_selector": {
"buckets_path": {
"avg": "avails>filter_avails.average"
},
"script": "params.avg > 150 && params.avg < 300"
}
}
}
}
}
}
I try to get data from elasticsearch with the exact filtering that I have on a Kibana3 dashboard, and for some reason can't make it make sense.
this is what I have on Kibana (which returns docs):
and this is a query that I try to run against ES and get no results:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool" : {
"must" : [
{ "term" : { "domain_name.raw": "foo.bar" } },
{ "term" : { "method": "get" } }
],
"should" : {
"or" : [
{
"term" : { "pathname.raw" : "/xxxxx" }
},
{
"term" : { "pathname.raw" : "/yyyyy" }
},
{
"term" : { "pathname.raw" : "/zzzzz" }
}
]
}
}
}
}
}
}
I've been cracking my head on it for a few hours and can't get it to work.
when I remove one of the must terms (either the domain_name or the method), I do get results. but then I'ts not exactly the filtering I need...
any idea where i'm going wrong?
It seems like your syntax is a bit off, when using should you don't need to use or, try this:
I try to get data from elasticsearch with the exact filtering that I have on a Kibana3 dashboard, and for some reason can't make it make sense.
this is what I have on Kibana (which returns docs): kibana filters
and this is a query that I try to run against ES and get no results:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [{
"term": {
"domain_name.raw": "foo.bar"
}
}, {
"term": {
"method": "get"
}
}],
"should": [{
"term": {
"pathname.raw": "/xxxxx"
}
}, {
"term": {
"pathname.raw": "/yyyyy"
}
}, {
"term": {
"pathname.raw": "/zzzzz"
}
}]
}
}
}
}
}
}
I want build query for search matching with nested and not equal.
This is my elasticSearch query:
{
"from":0,"size":1000,
"query":{
"nested" : {
"path" : "data",
"query" : {
"match" : {
"data.city" : "california"
}
}
},
"filter":{
"not":{
"filter":{
"term":{
"_id":"01921asda01201"
}
}
}
}
}
}
But I got error, am I write something wrong ? thanks
You can use bool Filter too with must and must_not clause.
{
"from": 0,
"size": 1000,
"filter": {
"bool": {
"must": [
{
"nested": {
"path": "data",
"query": {
"match": {
"data.city": "california"
}
}
}
}
],
"must_not": [
{
"term": {
"_id": "01921asda01201"
}
}
]
}
}
}
You need to use filtered query
GET _search
{
"query": {
"filtered": {
"query": {
"nested": {
"path" : "data",
"query" : {
"match" : {
"data.city" : "california"
}
}
}
},
"filter": {
"bool": {
"must_not": [
{
"term": {
"_id": "01921asda01201"
}
}
]
}
}
}
}
}
You should use a bool query for this, and put your two clauses in the must and must_not sections respectively.
If you don't care about scoring on the data.city field (from your example it's not clear), you might want to use the filter portion instead of the must portion.
{
"from": 0,
"size": 1000,
"query": {
"bool": {
"filter": [
{
"nested": {
"path": "data",
"query": {
"match": {
"data.city": "california"
}
}
}
}
],
"must_not": [
{
"term": {
"_id": "01921asda01201"
}
}
]
}
}
}