I'm trying to create a SAS URL for a blob storage container. I've tried multiple storage accounts and multiple methods of creating the SAS, and all of them give this result when I test the SAS URL in a browser:
<Error>
<Code>AuthenticationFailed</Code>
<Message>
Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:d95bf34f-0001-0022-4430-b1a25b000000 Time:2016-05-18T18:12:30.5552096Z
</Message>
<AuthenticationErrorDetail>
Signature did not match. String to sign used was rl 2016-05-18T18:10:00Z 2016-05-19T18:10:00Z /blob/cloudappmanager/$root 2015-04-05
</AuthenticationErrorDetail>
</Error>
I tried Storage Explorer (right-click container, Get SAS, click OK with defaults):
I tried the old Storage Explorer:
And I tried PowerShell:
PS C:\Users\virklba> $context = New-AzureStorageContext -StorageAccountName msuscoreaprod
cmdlet New-AzureStorageContext at command pipeline position 1
Supply values for the following parameters:
(Type !? for Help.)
StorageAccountKey: xxxxxxxxx
PS C:\Users\virklba> New-AzureStorageContainerSASToken -Name aadlogs -Context $context -FullUri -Permission rl
https://msuscoreaprod.blob.core.windows.net/aadlogs?sv=2015-04-05&sr=c&sig=xxxxxxxx&se=2016-05-18T19%3A47%3A56Z&sp=rl
All with the same result. Is anyone else seeing this behavior, or is it just me?
You are creating a SAS on the container, and it looks like you are trying to read the container in the browser. When I paste the container SAS into the browser, I get the same error you are getting.
The container SAS (with read permissions) gives you read access to the blobs in the container. So you need to append a blob name to the SAS before you paste it into the browser, in order to read a blob.
For example, this will not work:
https://myaccount.blob.core.windows.net/lotsofblobs?st=2016-05-18T22%3A49%3A00Z&se=2016-05-19T22%3A59%3A00Z&sp=rl&sv=2015-04-05&sr=c&sig=62WHwaZGI60ub1hYcQyKg1%2FE%2F1w9HUrOPGorzoWDLvE%3D
This does work, with myblob.txt appended to the base URL:
https://myaccount.blob.core.windows.net/lotsofblobs/myblob.txt?st=2016-05-18T22%3A49%3A00Z&se=2016-05-19T22%3A59%3A00Z&sp=rl&sv=2015-04-05&sr=c&sig=62WHwaZGI60ub1hYcQyKg1%2FE%2F1w9HUrOPGorzoWDLvE%3D
Please also see Gaurav Mantri's detailed explanation here: Azure Shared Access Signature - Signature did not match
To fix this, try connecting the storage account first, then the blob.
Related
I have created a storage_account with a container named data.
In that container I have a single .zip file.
I'm generating an Account Key SAS Token with Read permission directly on the data container :
The Blob SAS URL looks like this :
https://<STORAGE_ACCOUNT>.blob.core.windows.net/data?sp=r&st=2022-06-06T15:23:31Z&se=2022-06-06T23:23:31Z&spr=https&sv=2020-08-04&sr=c&sig=<SIGNATURE>
How am I supposed to download my zip file from that URI?
I'm always running into some Authorization error whereas I though having the link was enough and unfortunately documentation didn't help me to figure out what's wrong.
I would like to download the file from a HTTP call, not using az copy or powershell.
from your description and the URL you provided, I guess the issue is that you didn't reference the name of the zip file in the URL
so instead of
https://<STORAGE_ACCOUNT>.blob.core.windows.net/data?sp=r&st=2022-06-06T15:23:31Z&se=2022-06-06T23:23:31Z&spr=https&sv=2020-08-04&sr=c&sig=<SIGNATURE>
try
https://<STORAGE_ACCOUNT>.blob.core.windows.net/data/zipName?sp=r&st=2022-06-06T15:23:31Z&se=2022-06-06T23:23:31Z&spr=https&sv=2020-08-04&sr=c&sig=<SIGNATURE>
I try to copy file from Linux virtual machine on Azure (in a virtual network) to an account storage.
With Azcopy login It's working but I want to make it with SAS token.
I add my virtual network in "Networking".
image
And I generate a SAS key in "Shared acces signature"
image
On my linux virtual machine I have 10.0.3.4 ip adress.
image
I run this command sudo azcopy cp ./myFile https://backupscanqa.blob.core.windows.net/backup/?[mySASKey]
image
In my log I have this:
image
I dont know where is the problem because when I try the same thing with oAuth2 connexion with azcopy login it's working.
Thanks for your help !
Edit:
I try to generate a SAS key in my container with all grants:
When I use it it's the same error:
My sas key tranfom to sp=racwdli
From the logs I could see the SAS token you are using is incorrect. In your image its only sp=r in SAS token , whereas it should be something like this in the image if you are generating the SAS token as you have mentioned.
I tested the same in my environment , added firewall in Storage account like :
Using the generated SAS token as you have mentioned , the operation get successful using the below command :
./azcopy copy "/home/ansuman/azcopy_linux_amd64_10.13.0/NOTICE.txt" "https://testansumansa123.blob.core.windows.net/backup?sv=2020-08-04&ss=bfqt&srt=sco&sp=rwdlacupitfx&se=2022-01-27T15:18:31Z&st=2022-01-27T07:18:31Z&spr=https&sig=XXXXXX"
Which is in format of
./azcopy copy "SourcePath" "storageaccounturl/container<SASToken>"
As you can see if SAS is generated by the method in your image then it will have permissions as sp=rwdlacupitfx which is all permissions in the Storage account.
To resolve the issue , Please check the SAS token you are using .
If you are generating from Storage account like you have shown in image then you can use the SAS token by appending it behind your storage account url/container.
If you are generating the SAS token from inside the container , Please make sure to have selected the necessary permissions from the drop down as shown below and then you can use the Blob SAS URl :
#AnsumanBal-MT put me on the right track.
As he very well noticed in the logs, my SAS key does not appear.
However, I did copy my key.
So I understood that from the '&' in the URL the characters were not taken into account.
After adding '' before each & the command worked correctly!
Thanks you #AnsumanBal-MT!
I am trying to copy blobs between storage accounts using Copy-AzStorageBlob command.
$srcCtx = New-AzStorageContext -StorageAccountName $srcStorageAccountName -SasToken $srcSasToken
$destCtx = New-AzStorageContext -StorageAccountName $destStorageAccountName -SasToken $destSasToken
Copy-AzStorageBlob -SrcBlob "blobPath" -SrcContainer "src" -Context $srcCtx -DestContainer "dest" -DestContext $destCtx
and I get an error that the following parameter is invalid:
QueryParameterName: comp
QueryParameterValue: tags
but I am not using them directly. Do you have any idea what is wrong? I would like to underline, that source and destination context are working - I have read and write some blobs using mentioned contexts. The issue occurs only during blob copying.
The error message:
Copy-AzStorageBlob: Value for one of the query parameters specified in the request URI is invalid.
RequestId:d705aed0-b01e-0013-12c6-244430000000
Time:2022-02-18T12:56:40.1603279Z
Status: 400 (Value for one of the query parameters specified in the request URI is invalid.)
ErrorCode: InvalidQueryParameterValue
Additional Information:
QueryParameterName: comp
QueryParameterValue: tags
Reason:
Content:
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidQueryParameterValue</Code><Message>Value for one of the query parameters specified in the request URI is invalid.
RequestId:d705aed0-b01e-0013-12c6-244430000000
Time:2022-02-18T12:56:40.1603279Z</Message><QueryParameterName>comp</QueryParameterName><QueryParameterValue>tags</QueryParameterValue><Reason /></Error>
Headers:
Server: Microsoft-HTTPAPI/2.0
x-ms-request-id: d705aed0-b01e-0013-12c6-244430000000
x-ms-client-request-id: f76e9297-504e-42a7-8c86-54ad81bedad7
x-ms-error-code: InvalidQueryParameterValue
Date: Fri, 18 Feb 2022 12:56:40 GMT
Content-Length: 375
Content-Type: application/xml
Please try freshly generating a new SAS key through Azure Portal or Azure Storage Explorer to ensure that the SAS key is fresh and not expired and copy that in url without whitespaces and also make sure to have the correct permissions ( read, write, list etc) which are required for certain operations.
Make sure to have one of any of below roles tried.
1. Storage Blob Data Contributor
2. Storage Blob Data Owner
Try to upgrade(or downgrade) the package versions of azure.storage.blob
Try to provide only required headers like x-ms-blob-type in your request by removing all other unneccessary headers .Check if x-ms-tags header is present and remove if not required.
References:
Error -Value for one of
the query parameters specified in the request URI is invalid - Stack
Overflow
Get Blob Tags (REST API) - Azure Storage | Microsoft Docs
Copy Blob (REST API) - Azure Storage | Microsoft Docs
I'm trying to use a service principal to access and list the contents of an Azure storage container using PowerShell. I've been able to get a token for the service principal and use that to access the storage account but when I run this command
$iwrParams = #{
'Uri' = 'https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list'
'UseBasicParsing' = $true
'ContentType' = 'application/xml'
'Headers' = #{
'Authorization' = "Bearer $($token.access_token)"
'x-ms-version' = '2017-11-09'
}
}
Invoke-WebRequest #iwrParams
I get a return from this but the return xml content never shows the URL for each returned blob. If you look at the return at https://learn.microsoft.com/en-us/rest/api/storageservices/enumerating-blob-resources#list-blobs-and-snapshots it shows that for each blob it should return a Name, Url and various other properties. When I run the code above I get everything but the Url.
Now the really interesting thing is if I change the container access to anonymous public and run this code, I get the Url returned as expected.
$iwrParams = #{
'Uri' = 'https://myaccount.blob.core.windows.net/mycontainer?comp=list'
'UseBasicParsing' = $true
'ContentType' = 'application/xml'
}
Invoke-WebRequest #iwrParams
The issue honestly seems to be having to include restype=container when accessing with any sort of authentication.
My question is does anyone know a way to get the URL for each blob returned when not using anonymous access to list container contents?
I'd like to do this without resorting to the Az modules.
The reason you're not able to see the URL returned in the response is because of the storage REST API version used by your code (2017-11-09). Essentially the Blob URL property was removed from the response starting with REST API version 2013-08-15. From this link:
In version 2013-08-15 and newer, the EnumerationResults element
contains a ServiceEndpoint attribute specifying the blob endpoint, and
a ContainerName field specifying the name of the container. In
previous versions these two attributes were combined together in the
ContainerName field. Also in version 2013-08-15 and newer, the Url
element under Blob has been removed.
Regarding your comment about why you can see the URL property if you list blobs anonymously, this is happening because if no REST API version is specified in the request, Storage Service uses the oldest REST API version to process the request if default version has not been set. From this link:
If an anonymous request to a general-purpose storage account does not
specify the x-ms-version header, and the default version for the
service has not been set using Set Blob Service Properties, then the
service uses the earliest possible version to process the request.
Considering you would want to use Azure AD based authorization, the earliest version you will be able to use is 2017-11-09 thus it will not be possible to get the Blob URL returned in the response body.
One option would be to manually construct the Blob URL by using Blob Container URL and Blob name. Other option would be to use Shared Key Authorization instead of Azure AD authorization and specifying a version earlier than 2013-08-15 for x-ms-version request header in your requests. You will need to manually compute Authorization header value in this case using instructions provided here.
I tried creating SAS like this (ADDING "Read" permission changes nothing):
But it didnt work for me. I only want my script to get blob list, read metadata and delete old blobs.
Get-AzureStorageContainer : The remote server returned an error: (403)
Forbidden. HTTP Status Code: 403 - HTTP Error Message: This request is
not authorized to perform this operation.
Also, I'd like to know whats the minimum possible permissions to achieve my goal.
$ctx = New-AzureStorageContext -StorageAccountName xxx -SasToken zzz
$Containers = Get-AzureStorageContainer -Context $ctx
sample sas token:
?sv=2017-07-29&ss=b&srt=co&sp=dl&se=2018-03-31T21:24:06Z&st=2018-03-31T09:24:06Z&spr=https&sig=bWsg5sSPZF%2FaBXxfW6RoCH%2BlcFKBT6MFyMKTRM3I2jI%3D
So there are two things here:
You're getting 403 error: Assuming you're using the same SAS token as you have mentioned in the question along with Get-AzureStorageContainer Cmdlet, you will get this error. The reason for this is the purpose of this Cmdlet is to list blob containers in a storage account and for that you need to have Service permission in your SAS token (srt value in your SAS token should be sco instead of co). Because the required permission is not there in your SAS token, you are getting this 403 error. However if you use the same token along with Get-AzureStorageBlob, you should not get any error.
Necessary permissions for get blob list, read metadata and delete old blobs: For this, you would need the following permissions:
Allowed Services: Blobs (b)
Allowed resource types: Container (c) and Object (o)
Allowed permissions: List (l), Read (r) and Delete (d)
With this combination you should be able to list blobs from a blob container using Get-AzureStorageBlob, read its metadata and delete the blobs.
UPDATE
So what I did was I followed your steps and tried to list the blob containers using Get-AzureStorageContainer Cmdlet. I also got the same error :).
Then I ran the Cmdlet with Debug and Verbose switches and found that for each blob container, this Cmdlet tries to get the ACL.
_https://account.blob.core.windows.net/my-container?sv=2017-07-29&ss=b&srt=sco&sp=dl&se=2018-03-31T23:28:27Z&st=2018-03-31T15:2
8:27Z&spr=https&sig=signature&api-version=2017-04-17&restype=container&comp=acl.
Confirm The remote server returned an error: (403) Forbidden. HTTP
Status Code: 403 - HTTP Error Message: This request is not authorized
to perform this operation. [Y] Yes [A] Yes to All [H] Halt Command
[S] Suspend [?] Help (default is "Y"): y Get-AzureStorageContainer :
The remote server returned an error: (403) Forbidden. HTTP Status
Code: 403 - HTTP Error Message: This request is not authorized to
perform this operation. At line:1 char:1
+ Get-AzureStorageContainer -Context $ctx -Debug -Verbose
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Get-AzureStorageContainer], StorageException
+ FullyQualifiedErrorId : StorageException,Microsoft.WindowsAzure.Commands.Storage.Blob.Cmdlet.GetAzureStorageCont
ainerCommand
Now the problem is that you can't fetch ACL for a container using a shared access signature, you would need to use the account key (same thing goes for creating a shared access signature). This is the reason you're getting 403 error back from the service.
Not sure you would classify this as a bug in Get-AzureStorageContainer or would want to put in a feature request allowing you to list blob containers without getting its ACL but they way things are today, you can't list blob containers using this Cmdlet and a SAS token.