Deployment of queue, blob and ADLS2 private endpoints via Bicep goes wrong - azure

Question

I am trying to deploy a number of three azure storage resources under two storage accounts, and I want to implement three private endpoints as to only allow connection to these resources from VMs in the same VNET. Type of resources that need to be connected to in two separate storage accounts are (per storage account):
storageAccountTemp
Azure blob queue
Azure blob storage
storageAccountDatalake
ADLS 2 containers (datalake)
I have the following Azure Bicep code for deploying the permanent and temporary stores:
param location string
param environmentType string
param storageAccountSku string
param privateEndpointsSubnetId string
var privateEndpointNameTmpstBlob = 'pe-tmpst-blob-${environmentType}-001'
var privateEndpointNameTmpstQueue = 'pe-tmpst-queue-{environmentType}-001'
var privateEndpointNamePst = 'pe-pst-${environmentType}-001'
/// Temp storage ///
resource storageAccountTemp 'Microsoft.Storage/storageAccounts#2021-08-01' = {
name: 'tmpst${environmentType}'
location: location
sku: {
name: storageAccountSku
}
kind: 'StorageV2'
properties: {
allowBlobPublicAccess: false
accessTier: 'Hot'
minimumTlsVersion: 'TLS1_0'
publicNetworkAccess: 'Disabled'
}
}
resource blobContainerForQueue 'Microsoft.Storage/storageAccounts/blobServices/containers#2021-08-01' = {
name: '${storageAccountTemp.name}/default/claimcheck-storage-${environmentType}'
properties: {
publicAccess: 'None'
}
}
resource storageQueueMain 'Microsoft.Storage/storageAccounts/queueServices/queues#2019-06-01' = {
name: '${storageAccountTemp.name}/default/queue-main-${environmentType}'
}
/// Persistant storage datalake ///
resource storageAccountDatalake 'Microsoft.Storage/storageAccounts#2021-08-01' = {
name: 'pstdatalake${environmentType}'
location: location
sku: {
name: storageAccountSku
}
kind: 'StorageV2'
properties: {
allowBlobPublicAccess: false
accessTier: 'Hot'
minimumTlsVersion: 'TLS1_0'
isHnsEnabled: true
publicNetworkAccess: 'Disabled'
}
}
/// Data///
resource ContainerForData 'Microsoft.Storage/storageAccounts/blobServices/containers#2021-08-01' = {
name: '${storageAccountDatalake.name}/default/data-${environmentType}'
properties: {
publicAccess: 'None'
}
}
/// Private endpoints configuration for tempblob, queue and datalake ///
resource privateEndpointTmpstBlob 'Microsoft.Network/privateEndpoints#2021-05-01' = if (environmentType == 'dev' || environmentType == 'prd') {
name: privateEndpointNameTmpstBlob
location: location
properties: {
subnet: {
id: privateEndpointsSubnetId
}
privateLinkServiceConnections: [
{
name: privateEndpointNameTmpstBlob
properties: {
privateLinkServiceId: storageAccountTemp.id
groupIds: ['blob']
}
}
]
}
}
resource privateEndpointTmpstQueue 'Microsoft.Network/privateEndpoints#2021-05-01' = if (environmentType == 'dev' || environmentType == 'prd') {
name: privateEndpointNameTmpstQueue
location: location
properties: {
subnet: {
id: privateEndpointsSubnetId
}
privateLinkServiceConnections: [
{
name: privateEndpointNameTmpstQueue
properties: {
privateLinkServiceId: storageAccountTemp.id
groupIds: ['queue']
}
}
]
}
}
resource privateEndpointPst 'Microsoft.Network/privateEndpoints#2021-05-01' = if (environmentType == 'dev' || environmentType == 'prd') {
name: privateEndpointNamePst
location: location
properties: {
subnet: {
id: privateEndpointsSubnetId
}
privateLinkServiceConnections: [
{
name: privateEndpointNamePst
properties: {
privateLinkServiceId: storageAccountDatalake.id
groupIds: ['blob']
}
}
]
}
}
As you can see, for the storage account, IsHnsEnabled is set to true, as to enable HierarchicalNamespace and thus ADLS2 functionality. The problem is, if I include the privateEndpointPst resource deployment in the Bicep deployment, and then try to view a datalake container in the portal from a VM that is in the same VNET as the private endpoint (which are in the subnet that makes the privateEndpointsSubnetId variable), I get the following message when trying to look at files in one of the datalake containers:
I believe it is not the problems in the picture. The reason for this is that when I deploy all three endpoints together, they all show this same problem when trying to look at blob/queue/datalake in storageAccountTemp and storageAccountDatalake when I deploy all three endpoints.
However, only deploying the two endpoints for the storageAccountTemp resources and not the one for Datalake, I can see the data in the portal when running from the VM in the VNET and code running from this VM can also reach the queue + blob. So not only does the deployment of the privateEndpointPst seem to mess up datalake reachability, it also in some way does the same to the reachability of my other queue and blob in the storageAccountTemp if I deploy them altogether. My mind is boggled as to why this is happening, and why I cannot seem to deploy the datalake endpoint in the right way. Also, sometimes, deploying the endpoints altogether WILL make the datalake endpoint work, and break the other two, which is even more mind-boggling. Clicking do you want to do some checks to detect common connectivity issues gives me the following information, which does not make me much wiser as to what is causing the issue (since I'm pretty sure it's not firewalls; sometimes I can access, sometimes not):
Does anyone see what could be wrong with my Bicep code for deploying the endpoint that might be causing this issue? I'm at quite a loss here. Also tried to replace groupIds: ['blob'] with groupIds: ['dfs'], but that does not seem to solve my problem.

Answer 1

I seem to have found the issue. For connecting to a datalake resource, one needs to have both a private endpoint with groupIds: ['blob'] and groupIds: ['dfs], since the blob API is still used for getting some meta-info about the containers (as far as I can understand).
So adding:
resource privateEndpointPstDfs 'Microsoft.Network/privateEndpoints#2021-05-01' = if (environmentType == 'dev' || environmentType == 'prd') {
name: privateEndpointNamePstDfs
location: location
properties: {
subnet: {
id: privateEndpointsSubnetId
}
privateLinkServiceConnections: [
{
name: privateEndpointNamePstDfs
properties: {
privateLinkServiceId: storageAccountDatalake.id
groupIds: ['dfs']
}
}
]
}
}
Made the deployment work successfully.