AzureSearch edgeNGram search matching too many documents - azure

Question

I'm trying to implement a prefix search using a field analyzed with an edge ngram analyzer.
However, whenever I do a search, it returns similar matches, but that do not contain the searched term.
The following query
POST /indexes/resources/docs/search?api-version=2020-06-30
{
"queryType": "full",
"searchMode": "all",
"search": "short_text_prefix:7024032"
}
Returns
{
"#odata.context": ".../indexes('resources')/$metadata#docs(*)",
"#search.nextPageParameters": {
"queryType": "full",
"searchMode": "all",
"search": "short_text_prefix:7024032",
"skip": 50
},
"value": [
{
"#search.score": 4.669537,
"short_text_prefix": "7024032 "
},
{
"#search.score": 4.6333756,
"short_text_prefix": "7024030 "
},
{
"#search.score": 4.6333756,
"short_text_prefix": "7024034 "
},
{
"#search.score": 4.6333756,
"short_text_prefix": "7024031 "
},
{
"#search.score": 4.6319494,
"short_text_prefix": "7024033 "
},
... omitted for brevity ...
],
"#odata.nextLink": ".../indexes('resources')/docs/search.post.search?api-version=2020-06-30"
}
Which includes a bunch of documents which almost match my term. And the "correct" document with the highest score on top.
The custom analyzer tokenizes "7024032 " like this
"#odata.context": "/$metadata#Microsoft.Azure.Search.V2020_06_30.AnalyzeResult",
"tokens": [
{
"token": "7",
"startOffset": 0,
"endOffset": 7,
"position": 0
},
{
"token": "70",
"startOffset": 0,
"endOffset": 7,
"position": 0
},
{
"token": "702",
"startOffset": 0,
"endOffset": 7,
"position": 0
},
{
"token": "7024",
"startOffset": 0,
"endOffset": 7,
"position": 0
},
{
"token": "70240",
"startOffset": 0,
"endOffset": 7,
"position": 0
},
{
"token": "702403",
"startOffset": 0,
"endOffset": 7,
"position": 0
},
{
"token": "7024032",
"startOffset": 0,
"endOffset": 7,
"position": 0
}
]
}
How do I exclude the documents which did not match the term exactly?

Answer 1

Ngram is not the right way in this case as the prefix '702403' appears in all those documents. You can use it if you specify the minimum length to be the length of the term you're searching for.
Here's an example:
token length: 3
sample content:
234
1234
2345
3456
001234
99234345
searching for '234'
it would return items 1 (234), 2 (1234), 3 (2345), 4 (001234) and 5 (99234345)
Another option, if you're 100% the content is stored in the way you presented, you could use regular expression to retrieve the way you want:
/.*7024032\s+/

Answer 2

I figured out the problem:
I had created the field with the "analyzer" property referring to my custom analyzer ("edge_nGram_analyzer"). Setting this field means the string are tokenized both on indexing and when searching. So searching for "7024032" meant I was searching for all tokens, split according to the egde n-gram analyzer: "7", "70", "702", "7024", "7024032", "70240", "702403", "7024032"
The indexAnalyzer and searchAnalyzer properties can instead be used, to handle index-tokenizing separately from search-tokenizing. When I used them separately:
{ "indexAnalyzer": "edge_nGram_analyzer", "searchAnalyzer": "whitespace" }
everything worked as expected.