Azur Functions is not authorized to list blobs from afolder in a container in Data Lake - azure

Question

I would like to list blobs in a folder in a container in Azure Data Lake in an Azure Functions.
Für authenticating I would like to use system assigned managed identity of Azure Functions. I have activate it in my azure Functions and on Data Lake side give it Storage Blob Data Contributor role.
Here is my Code:
string dfsUri = "https://<myDataLake>.dfs.core.windows.net";
DataLakeClientOptions options = new DataLakeClientOptions(DataLakeClientOptions.ServiceVersion.V2019_07_07);
DataLakeServiceClient dataLakeServiceClient = new DataLakeServiceClient(new Uri(dfsUri), new Azure.Identity.DefaultAzureCredential(),options);
DataLakeFileSystemClient dataLakeFileSystemClient = dataLakeServiceClient.GetFileSystemClient("my-file-system");
IAsyncEnumerator<PathItem> enumerator = dataLakeFileSystemClient.GetPathsAsync("testfolder").GetAsyncEnumerator();
await enumerator.MoveNextAsync();
PathItem item = enumerator.Current;
while (item != null)
{
log.LogInformation($"File Name {item.Name}.");
if (!await enumerator.MoveNextAsync())
{
break;
}
item = enumerator.Current;
}
If I run my Code I get this error message:
This request is not authorized to perform this operation using this permission.
RequestId:cd00a570-401f-0024-4d21-35badb000000
Time:2021-04-19T13:39:30.8429070Z
Status: 403 (This request is not authorized to perform this operation using this permission.)
ErrorCode: AuthorizationPermissionMismatch
Headers:
Server: Windows-Azure-HDFS/1.0,Microsoft-HTTPAPI/2.0
x-ms-error-code: AuthorizationPermissionMismatch
x-ms-request-id: cd00a570-401f-0024-4d21-35badb000000
x-ms-version: 2019-07-07
x-ms-client-request-id: b0510f6a-5798-476c-a95e-6f206bf2a9cc
Date: Mon, 19 Apr 2021 13:39:29 GMT
Content-Length: 227
Content-Type: application/json; charset=utf-8
but on the container level the following code works fine for me:
string dfsUri = "https://<myDataLake>.dfs.core.windows.net";
DataLakeClientOptions options = new DataLakeClientOptions(DataLakeClientOptions.ServiceVersion.V2019_07_07);
DataLakeServiceClient dataLakeServiceClient = new DataLakeServiceClient(new Uri(dfsUri), new Azure.Identity.DefaultAzureCredential(),options);
DataLakeFileSystemClient dataLakeFileSystemClient = dataLakeServiceClient.GetFileSystemClient("my-file-system");
IAsyncEnumerator<PathItem> enumerator = dataLakeFileSystemClient.GetPathsAsync("").GetAsyncEnumerator();
await enumerator.MoveNextAsync();
PathItem item = enumerator.Current;
while (item != null)
{
log.LogInformation($"File Name {item.Name}.");
if (!await enumerator.MoveNextAsync())
{
break;
}
item = enumerator.Current;
}
can someone tell me what should I do to list blobs from a folder in a container?

Answer 1

PART 1
I have done something similar with a user assigned identity, but it should be the same for both. It sounds like your configuration should be correct, but you can do the following to confirm. In the Function App's Identity section, click on the "Azure Role Assignments":
Do you see anything listed? If not, then your current configuration it isn't recognized. You can try to the "Add role assignment" feature, which is a pretty straightforward wizard.
PART 2
Again, not exactly your scenario, but it may help. I recently wrote some Azure Functions that connect to Synapse via Managed Identity. I used a ManagedIdentityCredential, not a DefaultAzureCredential:
// Build the credentials
var clientId = SettingsHelper.Get(Settings.ManagedIdentityClientId);
var credential = new ChainedTokenCredential(new ManagedIdentityCredential(clientId),
new AzureCliCredential());
As you can see, I stored the Managed Identity Client ID value in the Function App Settings. The ChainedTokenCredential is there to support local development via the AzureCliCredential. I used the exact same code to connect to Key Vault.

Answer 2

Update:
Tested, the document description is indeed problematic, and 'Storage Blob Data Contributor' can give the required permissions:
var uri = new Uri("https://datalakename.dfs.core.windows.net/");
var tokenCredential = new ManagedIdentityCredential();
DataLakeServiceClient dataLakeServiceClient = new DataLakeServiceClient(uri,tokenCredential);
var fileSystemClient = dataLakeServiceClient.GetFileSystemClient("test");
Pageable<PathItem> items = fileSystemClient.GetPaths();
foreach (var item in items) {
Console.WriteLine(item.Name);
}
After add Storage Blob Data Contributor RBAC role of function app identity to data lake, I still get the error:
But few minutes later, I can fetch the blobs with no problem. I suspect you just need to wait for minutes and it will be OK。