Nested Filter Query with Exclusionary ORs - search

Question

I am unable to restrict the result set to documents that match both kol_tags.scored.name and kol_tags.scored.score range for both the or options below.
I would like to match documents that have the kol_tags.scored.name of "Core Grower" and kol_tags.scored.score between 1 and 100 unless they also have kol_tags.scored.name of "Connectivity" where kol_tags.scored.score is NOT in the range of 35 to 65.
Given the following mapping (non nested fields omitted for brevity):
GET /production_users/user/_mapping
{
"user": {
"_all": {
"enabled": false
},
"properties": {
"kol_tags": {
"type": "nested",
"properties": {
"scored": {
"type": "nested",
"properties": {
"name": {
"type": "string",
"index": "not_analyzed",
"omit_norms": true,
"index_options": "docs"
},
"score": {
"type": "integer"
}
}
}
}
}
}
}
}
I am executing the following query:
{
"filter": {
"nested": {
"path": "kol_tags.scored",
"filter": {
"or": [
{
"and": [
{
"terms": {
"kol_tags.scored.name": [
"Core Grower"
]
}
},
{
"range": {
"kol_tags.scored.score": {
"gte": 1,
"lte": 100
}
}
}
]
},
{
"and": [
{
"terms": {
"kol_tags.scored.name": [
"Connectivity"
]
}
},
{
"range": {
"kol_tags.scored.score": {
"gte": 35,
"lte": 65
}
}
}
]
}
]
}
}
}
}
With the query above I get documents that match kol_tags.scored.name of "Core Grower" and kol_tags.scored.score between 1 and 100 and ALSO that have kol_tags.scored.name of "Connectivity" and kol_tags.scored.score in any range.
What I need is documents that match:
kol_tags.scored.name of "Core Grower" and kol_tags.scored.score between 1 and 100
kol_tags.scored.name of "Connectivity" and kol_tags.scored.score between 35 and 65
Exclude any documents that have kol_tags.scored.name of "Connectivity" and kol_tags.scored.score less than 34 and greater than 66

Answer 1

There's some ambiguity in your description, but I've tried to make a runnable example that should work here: https://www.found.no/play/gist/8940202 (also embedded below)
Here's a few things I did:
Put the filter in a filtered-query. A top level filter (renamed to post_filter in Elasticsearch 1.0) should only be used if you want to filter hits, but not facets.
Use bool instead of and and or, since the filters are cachable. More here: http://www.elasticsearch.org/blog/all-about-elasticsearch-filter-bitsets/
And most importantly, put the nested inside the bool, so the logic gets right wrt. what should match on the nested vs. the parent document.
Added a must_not to account for your last point. Not sure if you can have two sub-documents with name "Connectivity", but if you can, that should account for it. If you'll only ever have one, you can remove the must_not.
You didn't provide any sample documents, so I made some I think should fit your description. I don't think you need two levels of nested.
#!/bin/bash
export ELASTICSEARCH_ENDPOINT="http://localhost:9200"
# Create indexes
curl -XPUT "$ELASTICSEARCH_ENDPOINT/play" -d '{
"mappings": {
"type": {
"properties": {
"kol_tags": {
"properties": {
"scored": {
"type": "nested",
"properties": {
"name": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
}
}'
# Index documents
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_bulk?refresh=true" -d '
{"index":{"_index":"play","_type":"type"}}
{"kol_tags":{"scored":[{"name":"Core Grower","score":36},{"name":"Connectivity","score":42}]}}
{"index":{"_index":"play","_type":"type"}}
{"kol_tags":{"scored":[{"name":"Connectivity","score":34},{"name":"Connectivity","score":42}]}}
{"index":{"_index":"play","_type":"type"}}
{"kol_tags":{"scored":[{"name":"Core Grower","score":36}]}}
{"index":{"_index":"play","_type":"type"}}
{"kol_tags":{"scored":[{"name":"Connectivity","score":36}]}}
'
# Do searches
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_search?pretty" -d '
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"nested": {
"path": "kol_tags.scored",
"filter": {
"bool": {
"must": [
{
"term": {
"name": "Core Grower"
}
},
{
"range": {
"score": {
"gte": 1,
"lte": 100
}
}
}
]
}
}
}
},
{
"nested": {
"path": "kol_tags.scored",
"filter": {
"bool": {
"must": [
{
"term": {
"name": "Connectivity"
}
},
{
"range": {
"score": {
"gte": 35,
"lte": 65
}
}
}
]
}
}
}
}
],
"must_not": [
{
"nested": {
"path": "kol_tags.scored",
"filter": {
"bool": {
"must": [
{
"term": {
"name": "Connectivity"
}
},
{
"not": {
"range": {
"score": {
"gte": 35,
"lte": 65
}
}
}
}
]
}
}
}
}
]
}
}
}
}
}
'
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_search?pretty" -d '
{
"filter": {
"nested": {
"path": "kol_tags.scored",
"filter": {
"or": [
{
"and": [
{
"terms": {
"kol_tags.scored.name": [
"Core Grower"
]
}
},
{
"range": {
"kol_tags.scored.score": {
"gte": 1,
"lte": 100
}
}
}
]
},
{
"and": [
{
"terms": {
"kol_tags.scored.name": [
"Connectivity"
]
}
},
{
"range": {
"kol_tags.scored.score": {
"gte": 35,
"lte": 65
}
}
}
]
}
]
}
}
}
}
'