Azure Search - Partial Phrase match - azure

Question

I'm trying to improve the ranking of results that come back from an Azure Search index. The search index basically contains a list of band names and members.
Exact match is important to us, but also a partial match, but likewise, so is a partial word within the query.
If I use the example of trying to find a band called Black Flag. In user input area, I have got as far as typing black fl.
I currently structure the query as: "black fl"|black fl* (exact match on whole phrase and partial match on fl).
This brings back the following results in following order:
Flourescent Black
Florence Black
Black Flag
At the moment, there is the single text field being searched against, using the Standard - Lucene Analyzer.
I've looked at Scoring Profiles but these don't appear to be relevant to such a small dataset in terms of fields available.
I have also explored the full lucene search, by adding things like ^10 on the word black to make it more important - and have changed my query string in many ways, all of which don't seem to give the effect i'm after.
I would expect that Black Flag would match better as the word order is more correct than that of the results that come above it.
Is there a way to change the scoring method to handle this? I now imagine that I'm looking at dealing with a custom analyzer (https://learn.microsoft.com/en-gb/azure/search/index-add-custom-analyzers) but not really sure where to start with this or how I would want the analyzer to behave.
Any thoughts or examples on how best to handle this scenario would be greatly appreciated.
EDIT - More Info
The current solution consists of the following, but it involves having to manipulate the results that come back from the search index.
The index is created as follows:
{
"fields": [
{"name": "id", "type": "Edm.String", "key": true, "filterable": false, "searchable": false, "sortable": false, "facetable": false},
{"name": "entityId", "type": "Edm.Int64", "filterable": false, "searchable": false, "sortable": false, "facetable": false},
{"name": "entityType", "type": "Edm.Int32", "sortable": false, "facetable": false},
{"name": "sortableName", "type": "Edm.String", "filterable": false, "facetable": false, "searchable": false},
{"name": "name", "type": "Edm.String", "filterable": false, "retrievable": false, "sortable": false, "facetable": false, "analyzer":"keyword_analyzer"},
{"name": "town", "type": "Edm.String", "filterable": false, "retrievable": false, "sortable": false, "facetable": false, "analyzer":"keyword_analyzer"},
{"name": "tags", "type": "Collection(Edm.String)", "filterable": false, "retrievable": false, "sortable": false, "facetable": false, "analyzer":"keyword_analyzer"}
],
"defaultScoringProfile": "default_score",
"scoringProfiles": [
{
"name": "default_score",
"text":{
"weights": {
"name": 3.5,
"tags": 2,
"town": 1
}
}
}
],
"analyzers":[
{
"name": "keyword_analyzer",
"#odata.type":"#Microsoft.Azure.Search.CustomAnalyzer",
"charFilters":[
"map_dash",
"map_space"
],
"tokenizer":"keyword_tokenizer",
"tokenFilters":[
"asciifolding",
"lowercase",
"trim",
"delimiter_filter"
]
}
],
"charFilters":[
{
"name":"map_dash",
"#odata.type":"#Microsoft.Azure.Search.MappingCharFilter",
"mappings":["-=>_"]
},
{
"name":"map_space",
"#odata.type":"#Microsoft.Azure.Search.MappingCharFilter",
"mappings":["\\u0020=>_"]
}
],
"tokenizers":[
{
"name": "keyword_tokenizer",
"#odata.type":"#Microsoft.Azure.Search.KeywordTokenizerV2"
}
],
"tokenFilters":[
{
"name": "stopwords_filter",
"#odata.type":"#Microsoft.Azure.Search.StopwordsTokenFilter",
"removeTrailing": false
},
{
"name": "delimiter_filter",
"#odata.type":"#Microsoft.Azure.Search.WordDelimiterTokenFilter",
"generateWordParts": true,
"generateNumberParts": true,
"splitOnCaseChange": false,
"preserveOriginal": true,
"splitOnNumerics": false
}
]
}
Before uploading data to the index we need to normalize it - Black Flag becomes black flag. We also have to remove any preceeding words of the so this means that The Killers becomes killers - also any non standard characters are replaced to remove accents etc.
When performing a search, in code we need to now remove any preceeding the if it exists, and perform the same normalization - I can accept doing this.
We then build up the query which changes depending upon how many words there are in the initial query.
List<string> splitQ = queryPhrase.SplitToList(" ");
if (splitQ.Count > 0)
{
if (splitQ.Count == 1)
{
search.Append($"(\"{splitQ[0]}\" || {this.EscapeSpecialCharacters(splitQ[0])}*)");
}
else
{
for (int i = 0; i < splitQ.Count; i++)
{
if (i == splitQ.Count - 1)
{
search.Append($"+{this.EscapeSpecialCharacters(splitQ[i])}*");
}
else
search.Append($"+\"{splitQ[i]}\"");
}
search.Insert(0, $"(\"{queryPhrase}\"||(");
search.Append("))");
}
}
A single word black would mean the main query is: ("black" || black*)
However, as soon as additional words come in it has to change. black fl becomes: ("black fl"||(+"black"+fl*))
A three word search would be: ("one two three"||(+"one"+"two"+three*))
On top of this we then add any filter options.
Search is sent to the index with the query type set to full
The above has got us as close as we can to having decent and accurate results. However, the scoring is all messed up.
Processing the results...
Firstly we now normalise the score given by the Azure Search Index, depending upon the search query, the scores range massively, so we normalize this as a percentage based on the maximum scoring item.
We now have to apply our own enhancer to the score based on the tag or name field. An exact match to the query gives an enhancer of 5, and a startswith query gets and enhancement of 3.
We then provide a score that uses the enhancement to increase the results position in the rankings.
This final seciton of processing the results seems as though it is something that should be done automatically within the search index system.