Get Azure Storage to Return blob URL when listing storage container contents - azure

Question

I'm trying to use a service principal to access and list the contents of an Azure storage container using PowerShell. I've been able to get a token for the service principal and use that to access the storage account but when I run this command
$iwrParams = #{
'Uri' = 'https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list'
'UseBasicParsing' = $true
'ContentType' = 'application/xml'
'Headers' = #{
'Authorization' = "Bearer $($token.access_token)"
'x-ms-version' = '2017-11-09'
}
}
Invoke-WebRequest #iwrParams
I get a return from this but the return xml content never shows the URL for each returned blob. If you look at the return at https://learn.microsoft.com/en-us/rest/api/storageservices/enumerating-blob-resources#list-blobs-and-snapshots it shows that for each blob it should return a Name, Url and various other properties. When I run the code above I get everything but the Url.
Now the really interesting thing is if I change the container access to anonymous public and run this code, I get the Url returned as expected.
$iwrParams = #{
'Uri' = 'https://myaccount.blob.core.windows.net/mycontainer?comp=list'
'UseBasicParsing' = $true
'ContentType' = 'application/xml'
}
Invoke-WebRequest #iwrParams
The issue honestly seems to be having to include restype=container when accessing with any sort of authentication.
My question is does anyone know a way to get the URL for each blob returned when not using anonymous access to list container contents?
I'd like to do this without resorting to the Az modules.

Answer 1

The reason you're not able to see the URL returned in the response is because of the storage REST API version used by your code (2017-11-09). Essentially the Blob URL property was removed from the response starting with REST API version 2013-08-15. From this link:
In version 2013-08-15 and newer, the EnumerationResults element
contains a ServiceEndpoint attribute specifying the blob endpoint, and
a ContainerName field specifying the name of the container. In
previous versions these two attributes were combined together in the
ContainerName field. Also in version 2013-08-15 and newer, the Url
element under Blob has been removed.
Regarding your comment about why you can see the URL property if you list blobs anonymously, this is happening because if no REST API version is specified in the request, Storage Service uses the oldest REST API version to process the request if default version has not been set. From this link:
If an anonymous request to a general-purpose storage account does not
specify the x-ms-version header, and the default version for the
service has not been set using Set Blob Service Properties, then the
service uses the earliest possible version to process the request.
Considering you would want to use Azure AD based authorization, the earliest version you will be able to use is 2017-11-09 thus it will not be possible to get the Blob URL returned in the response body.
One option would be to manually construct the Blob URL by using Blob Container URL and Blob name. Other option would be to use Shared Key Authorization instead of Azure AD authorization and specifying a version earlier than 2013-08-15 for x-ms-version request header in your requests. You will need to manually compute Authorization header value in this case using instructions provided here.