Elasticsearch: boost the absence of certain terms - search

Question

How do I positive-boost the absence of certain terms? I've asked this question before here but the response was not satisfactory because it wasn't generalizable enough.
Lets try again, with more nuances.
I want to be able to distinguish laptops from their accessories. In human language this is done by the absense of terms. That is, when you say lenovo thinkpad you know that by omitting the word battery you mean you want the actual laptop. Compare this with when a person says lenovo thinkpad battery, where they mean the battery.
So suppose we have the index:
PUT test_index
{
"settings": {
"index": {
"number_of_shards": 1,
"number_of_replicas": 1
}
}
}
with mapping:
PUT test_index/_mapping/merchant
{
"properties": {
"title": {
"type": "string"
},
"category": {
"type": "string",
"index": "not_analyzed"
}
}
}
put two items into it:
PUT test_index/merchant/3
{
"title": "macbook battery",
"category": "laptops accessories"
}
PUT test_index/merchant/2
{
"title": "lenovo thinkpad battery",
"category": "laptops accessories"
}
PUT test_index/merchant/1
{
"title": "lenovo thinkpad white/black",
"category": "laptops"
}
Now search lenovo thinkpad:
POST test_index/_search
{
"query":{
"match": { "title": "lenovo thinkpad" }
}
}
The result is:
"hits": [
{
"_index": "test_index",
"_type": "merchant",
"_id": "2",
"_score": 0.70710677,
"_source": {
"title": "lenovo thinkpad battery",
"category": "laptops accessories"
}
},
{
"_index": "test_index",
"_type": "merchant",
"_id": "1",
"_score": 0.70710677,
"_source": {
"title": "lenovo thinkpad white/black",
"category": "laptops"
}
}
]
where notice that lenovo thinkpad battery is higher up than lenovo thinkpad white/black.
Now, I can see at least two reasonable ways to do this.
A) Use term frequency on a per-category basis to influence relevance of title match. For example, if for each category you extract the 95% percentile terms, you get that battery is a high frequency term in laptops accessories and so the word battery should be negative-boosted on all title queries.
B) Use term frequency on a per-category basis to influence relevance of category match. For example, in addition of the title match, you automatically negative-boost results whose categories have 95% percentile terms which aren't contained in your title match.
A and B aren't quite the same, but they both rely on the idea that certain absent words should be taken into account for relevance.
So...... thoughts?

Answer 1

My vote would be
C)
Fix the categories so that a battery doesn't have 'laptops' as a category (it's a 'laptopAccessory' or just 'accessory') Alternatively create an additional category (not called 'laptops') to indicate the actual machines themselves.
In your search, instead of trying to down-rank the accessories, you apply a boost to the 'laptops' category (no longer ambiguous). This will cause initial searches as in your example of 'lenovo thinkpad' to bring the actual machines up above the accessories. A more precise search ('lenovo thinkpad battery') will still work as you'd expect also.
Another nice UI/UX experience is to take the total set of categories returned in your results, and provide easy filter links. So if your initial search returns 'laptops' 'accessories' 'payment plans', then you'd have each of those as a link to a re-query that uses the original search plus a filter on that category.
Good luck!

Answer 2

Boost "that" category.
GET /test_index/merchant/_search
{
"from": 0,
"query": {
"bool": {
"must": [
{"match": {"title": "lenovo thinkpad"}}
],
"should": [
{
"match": {
"category": {
"boost": "2",
"query": "laptops"
}
}
}
]
}
},
"size": "10"
}
Result:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1.573319,
"hits": [
{
"_index": "index",
"_type": "merchant",
"_id": "1",
"_score": 1.573319,
"_source": {
"title": "lenovo thinkpad white/black",
"category": "laptops"
}
},
{
"_index": "index",
"_type": "merchant",
"_id": "2",
"_score": 0.15889977,
"_source": {
"title": "lenovo thinkpad battery",
"category": "laptops accessories"
}
}
]
}
}
More on boosting, can be found here

Answer 3

We can update the absence of certain terms using boost property which was provided while query for that term.
Please check below query with boost property set to 10.
GET /test_index/students/_search
{
"from": 0,
"query": {
"bool": {
"must": [
{"match": {"age": "20"}}
],
"should": [
{
"match": {
"category": {
"boost": "10",
"query": "students"
}
}
}
]
}
},
"size": "10"
}