Download parquet file from ADL Gen2 using powershell - azure

Question

I am using below powershell command to download TSV file from ADL to local system and it works absolutely fine
#this appid has access to ADL
[string] $AppID = "bbb88818-aaaa-44fb-q2345678901y"
[string] $TenantId = "ttt88888-xxxx-yyyy-q2345678901y"
[string] $SubscriptionName = "Sub Sample"
[string] $AzureDataLakeAccountName = "sample.blob.core.windows.net"
[string] $AzureDataLakeSrcFilePath = "/accounts/sample/test.tsv"
[string] $LocalTargetFilePathName = "D:\MoveToModern"
Write-Host "AppID = " $AppID
Write-Host "TenantId = " $TenantId
Write-Host "SubscriptionName = " $SubscriptionName
Write-Host "AzureDataLakeAccountName = " AzureDataLakeAccountName
Write-Host "AzureDataLakeSrcFilePath = " $AzureDataLakeSrcFilePath
Write-Host "LocalTargetFilePathName = " $LocalTargetFilePathName
#this is the access key of the appid
$AccessKeyValue = "1234567=u-r.testabcdefaORYsw5AN5"
$azurePassword = ConvertTo-SecureString $AccessKeyValue -AsPlainText -Force
$psCred = New-Object System.Management.Automation.PSCredential($AppID, $azurePassword)
Login-AzureRmAccount -Credential $psCred -ServicePrincipal -Tenant $TenantId
Get-AzureRmSubscription
Get-AzureRmSubscription -SubscriptionName $SubscriptionName | Set-AzureRmContext
Export-AzureRmDataLakeStoreItem -AccountName $AzureDataLakeAccountName -Path $AzureDataLakeSrcFilePath -Destination $LocalTargetFilePathName -Force
But when I replace sourcefile path with value /accounts/sample/test-V4.parquet, I get below error:
ADLSException: Error in getting metadata for path /accounts/Partner/Non-PII/Account/Account-V4.parquet.
Operation: GETFILESTATUS failed with HttpStatus:Forbidden Error: Unexpected error in JSON parsing of the error stream. Content-Type of error response: application/xml.
ExceptionType: Newtonsoft.Json.JsonReaderException ExceptionMessage: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
AuthenticationFailedServer failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:de2e4b16-001e-002e-625f-c92785000000
Can you please let me know how can I download parquet file from ADL Gen2 using powershell.

Answer 1

According to the information you provide, you do not have enough permissions to do that. So you get Forbidden error. If you want to download a file from Azure data lake store with a service principal, we need to grant the security principal read access to the file and give the security principal Execute permissions to the container and each folder in the hierarchy of folders that lead to the file.
For example
Please check it and set the right ACL. Regarding how to do that, please refer to here
If you want to download files from Azure Data Lake Gen2, I suggest you use PowerShell module Az.Storage. For more details, please refer to here
For example
Configure ACL for the service principal.
If you want to download a file from Azure data lake Gen2 with a service principal, we need to grant the security principal read access to the file and give the security principal Execute permissions to the container and each folder in the hierarchy of folders that lead to the file. Regarding how to configure it, please refer to here.
Script
$AppID = ""
$AccessKeyValue = ""
$TenantId=""
$azurePassword = ConvertTo-SecureString $AccessKeyValue -AsPlainText -Force
$psCred = New-Object System.Management.Automation.PSCredential($AppID, $azurePassword)
Connect-AzAccount -Credential $psCred -ServicePrincipal -Tenant $TenantId
$AzureDataLakeAccountName = ""
$ctx =New-AzStorageContext -StorageAccountName $AzureDataLakeAccountName -UseConnectedAccount
$filesystemName="<your container name>"
$path="<your blob name>"
$LocalTargetFilePathName = ""
Get-AzDataLakeGen2ItemContent -Context $ctx -FileSystem $filesystemName -Path $path -Destination $LocalTargetFilePathName