Azure Search decode base64 file contents for Index - azure

Question

I am attempting to use Azure search on a blob container that contains a ton of .htm files. Each one of these files is entirely encoded in base64 with padding. One of these files may be "example.htm", and if you opened it you would see:
//This decodes to html
PCEtLSBBIHNlZ21lbnQgb2YgYSBzd2VldCBib2R5IC0tPg0KPGRpdiBjbGFzcz0iYS1uaWNlLWNsYXNzIiBpZD0iaW1tYS1pZCI+DQoJPHA+Q2F0J3MgYXJlIGhhcmQgdG8gZGVjb2RlPC9wPg0KPC9kaXY+
I have tried to add a field mapping to decode this in my indexer. If I set "useHttpServerUtilityUrlTokenDecode": true then I get “Error applying mapping function ‘base64Decode’ to field ‘NAME’: Array cannot be null.\r\nParameter name: bytes”. , and if I set it to false then no files are indexed even though it says "success".
{
"name":"demoindexer",
"dataSourceName" : "demodata",
"targetIndexName" : "demoindex",
"fieldMappings" : [
{
"sourceFieldName" : "content",
"targetFieldName" : "content",
"mappingFunction" :
{ "name" : "base64Decode", "parameters" : {
"useHttpServerUtilityUrlTokenDecode" : false } }
}
],
"parameters":
{
"maxFailedItems":-1,
"maxFailedItemsPerBatch":-1
}
}
It seems that a clue may be a note on the field mappings page for Azure where it says that for "Base64 encoding with padding, use URL-safe characters and remove padding through additional processing after library encoding". I am not sure if this can be done through the Azure Search API and if so how to go about it, or if it is just really saying before uploading into Azure storage encode differently.
How would I go about decoding the contents of these files for my index so that search results will not return base64 stings?

Answer 1

If I understand your situation correctly, your blobs contain only base64-encoded text. If so, you should use text parsing mode to preserve your text as-is so it can be decoded. See Indexing plain text.
After you modify your indexer to use text parsing mode, don't forget to reset to the indexer so that it will start indexing your blobs from scratch. It's necessary because previously the indexer skipped over the blobs since you set "maxFailedItems" : -1.