Python: Upload a package to Azure using SAS URI - python-3.x

I am using the Microsoft's Hardware dashboard API to automate the submission of my (.CAB) package for signing. I have followed the steps in this documentation: https://learn.microsoft.com/en-us/windows-hardware/drivers/dashboard/create-a-new-submission-for-a-product
The response of new submission contains the SAS(Shared Access Signature) URI
like this: (changed the sig and accnt_name for security)
'''https://accnt_name.blob.core.windows.net/scsjc/cexxxxxxxxxx?sv=2017-04-17&sr=b&sig=xxxxxxxxxxxxxx&se=2019-07-10T18:15:58Z&sp=rwl&rscd=attachment%3B filename%3Dinitial_xxxxxxxx.cab'''
I need to use this SAS URI to upload by package to azure blob storage.
The examples in documentation shows C# or .NET as follows:
string sasUrl =
"https://productingestionbin1.blob.core.windows.net/ingestion/26920f66-
b592-4439-9a9d-fb0f014902ec?sv=2014-02-
14&sr=b&sig=usAN0kNFNnYE2tGQBI%2BARQWejX1Guiz7hdFtRhyK%2Bog%3D&se=2016-
06-17T20:45:51Z&sp=rwl";
Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob blockBob =
new Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob(new
System.Uri(sasUrl));
await blockBob.UploadFromStreamAsync(stream);
I want to use the SAS URI obtained from submission resource JSON Response to upload the package.
This link Download file from AZURE BLOB CONTAINER using SAS URI in PYTHON suggests that there is no equivalent method in python and BlockBlobService can be used.
from azure.storage.blob import BlockBlobService
blobservice = BlockBlobService("storage_account",sas_token="?sv=2018-03-
28&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-24T10:01:58Z&st=2019-04-
23T02:01:58Z&spr=https&sig=xxxxxxxxx")
blobservice.create_blob_from_path(container_name, local_file_name,
full_path_to_file)
However I am not sure of what is storage_account name and container name from the SAS URI obtained from submission resource.
Also I have created a separate azure storage account and added a new container, blob in it. I have tried passing the new container and storage account name with SAS access token from SAS URI (obtained from submission JSON response micorsoft hardware api) but always get below ERROR
'''
AzureHttpError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode: AuthenticationFailed
AuthenticationFailedServer failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:5463b7d2-901e-0068-6994-36782e000000
Time:2019-07-09T20:23:04.5760736ZSignature did not match. String to sign used was rwl
2019-07-10T18:15:58Z
/blob/evcertautomation/ev2/initial_1152921504628106590.cab
2017-04-17
attachment; filename=initial_1152921504628106563.cab
'''
Thanks in advance

If you have a blob SAS URI as you post below, you can easily upload a file to the blob in Python with requests.
https://accnt_name.blob.core.windows.net/scsjc/cexxxxxxxxxx?sv=2017-04-17&sr=b&sig=xxxxxxxxxxxxxx&se=2019-07-10T18:15:58Z&sp=rwl&rscd=attachment%3B filename%3Dinitial_xxxxxxxx.cab
First, you must have to inspect the values of parameters se and sp. The se parameter means the expire time of the blob SAS URI, and the sp parameter means the operation permisson of the blob SAS URL like w for Blob Write Permission
So for your blob SAS URL above, you have the blob write permission to upload a file to this blob before the time 2019-07-10T18:15:58Z.
Here is my sample code for uploading via a blob sas uri.
import requests
blob_sas_uri = '<your blob sas uri which must includes `sp=w` and do the write operation before `se`>'
local_file_name = '<your local file name>'
headers = {
'x-ms-blob-type': 'BlockBlob'
}
data = open(local_file_name).read()
r = requests.put(blob_sas_uri, headers=headers, data=data)
print(r.status_code)
If you see the result is 201, it works fine and succeed for uploading.
As reference, there is a similar offical sample Example: Upload a Blob using a Container’s Shared Access Signature which using a wide container permission.

As per the SAS URI you provided: '''https://accnt_name.blob.core.windows.net/scsjc/cexxxxxxxxxx?sv=2017-04-17&sr=b&sig=xxxxxxxxxxxxxx&se=2019-07-10T18:15:58Z&sp=rwl&rscd=attachment%3B filename%3Dinitial_xxxxxxxx.cab'''
The account name should be accnt_name, the container should be scsjc.
So your code should look like below:
from azure.storage.blob import BlockBlobService
storage_account ="accnt_name"
token="?sv=2018-03-
28&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-24T10:01:58Z&st=2019-04-
23T02:01:58Z&spr=https&sig=xxxxxxxxx"
container="scsjc"
blobservice = BlockBlobService(storage_account,sas_token=token)
blobservice.create_blob_from_path(container, local_file_name,
full_path_to_file)

Related

Azure python sdk authentication using SAS url [duplicate]

We got error of Authentication fail, when we try to create an azure blob client from connection string, using python v12 sdk with Azure Blob Storage v12.5.0, and Azure core 1.8.2.
I used
azure-storate-blob == 12.5.0
azure-core == 1.8.2
I tried to access my blob storage account using connection string with Python v12 SDK and received the error above. The environment I'm running in is python venv in NixShell.
The code for calling the blob_upload is as following:
blob_service_client = BlobServiceClient(account_url=<>,credential=<>)
blob_client = blob_service_client.get_blob_client(container=container_name,
blob=file)
I printed out blob_client, and it looks normal. But the next line of upload_blob gives error.
with open(os.path.join(root,file), "rb") as data:
blob_client.upload_blob(data)
The error message is as follows
File "<local_address>/.venv/lib/python3.8/site-packages/azure/storage/blob/_upload_helpers.py", in upload_block_blob
return client.upload(
File "<local_address>/.venv/lib/python3.8/site-packages/azure/storage/blob/_generated/operations/_block_blob_operations.py", in upload
raise models.StorageErrorException(response, self._deserialize)
azure.storage.blob._generated.models._models_py3.StorageErrorException: Operation returned an invalid status 'Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.'
So I printed out the http put request to azure blob storage, and get the response value of [403]
I can work the following code well with the version the same as yours.
from azure.storage.blob import BlobServiceClient
blob=BlobServiceClient.from_connection_string(conn_str="your connect string in Access Keys")
with open("./SampleSource.txt", "rb") as data:
blob.upload_blob(data)
Please check your connect-string, and check your PC's time.
There is a similar issue about the error: AzureStorage Blob Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature
UPDATE:
I tried with this code, and get the same error:
from azure.storage.blob import BlobServiceClient
from azure.identity import DefaultAzureCredential
token_credential = DefaultAzureCredential()
blob_service_client = BlobServiceClient(account_url="https://pamelastorage123.blob.core.windows.net/",credential=token_credential)
blob_client = blob_service_client.get_blob_client(container="pamelac", blob="New Text Document.txt")
with open("D:/demo/python/New Text Document.txt", "rb") as data:
blob_client.upload_blob(data)
Then I use AzureCliCredential() instead of DefaultAzureCredential(). I authenticate via the Azure CLI with az login. And it works.
If you use environment credential, you need to set the variables. Anyway, I recommend you to use the specific credentials instead DefaultAzureCredential.
For more details about Azure Identity, see here.

I want to generate SAS URL dynamically via C# code for Azure Blob Container. Using this SAS URL we must be able to upload the files

I want to generate SAS URL dynamically via C# code for Azure Blob Container. Using this SAS URL we must be able to upload the files to the Azure Blob Container. I have tried multiple ways to generate the SAS URL by following the Microsoft docs. But I am always getting AuthorizationResourceTypeMismatch Error or AuthorizationPermissionMismatch.
Error: AuthorizationPermissionMismatch This request is not authorized to perform this operation using this permission.
private static Uri GetServiceSasUriForContainer(BlobContainerClient containerClient,
string storedPolicyName = null)
{
// Check whether this BlobContainerClient object has been authorized with Shared Key.
if (containerClient.CanGenerateSasUri)
{
// Create a SAS token that's valid for one hour.
BlobSasBuilder sasBuilder = new BlobSasBuilder()
{
BlobContainerName = containerClient.Name,
Resource = "c"
};
if (storedPolicyName == null)
{
sasBuilder.ExpiresOn = DateTimeOffset.UtcNow.AddHours(1);
sasBuilder.SetPermissions(BlobContainerSasPermissions.Read);
}
else
{
sasBuilder.Identifier = storedPolicyName;
}
Uri sasUri = containerClient.GenerateSasUri(sasBuilder);
Console.WriteLine("SAS URI for blob container is: {0}", sasUri);
Console.WriteLine();
return sasUri;
}
else
{
Console.WriteLine(#"BlobContainerClient must be authorized with Shared Key
credentials to create a service SAS.");
return null;
}
}
Error: AuthenticationFailed Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
I need this sas url because I use this url in my javascript to upload files into the Azure Blob Container.
Can someone help me out achieving this goal?
The reason you're getting this error is because you are creating the SAS token with Read permission (BlobContainerSasPermissions.Read).
In order to upload a blob in a container using SAS URL, the SAS token needs either Write (BlobContainerSasPermissions.Write) or Create (BlobContainerSasPermissions.Create) permission. Please create a SAS token with one of these permissions and you should not get this error.
To learn more about the permissions, please see this link: https://learn.microsoft.com/en-us/rest/api/storageservices/create-service-sas#permissions-for-a-directory-container-or-blob.

Get Azure Storage to Return blob URL when listing storage container contents

I'm trying to use a service principal to access and list the contents of an Azure storage container using PowerShell. I've been able to get a token for the service principal and use that to access the storage account but when I run this command
$iwrParams = #{
'Uri' = 'https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list'
'UseBasicParsing' = $true
'ContentType' = 'application/xml'
'Headers' = #{
'Authorization' = "Bearer $($token.access_token)"
'x-ms-version' = '2017-11-09'
}
}
Invoke-WebRequest #iwrParams
I get a return from this but the return xml content never shows the URL for each returned blob. If you look at the return at https://learn.microsoft.com/en-us/rest/api/storageservices/enumerating-blob-resources#list-blobs-and-snapshots it shows that for each blob it should return a Name, Url and various other properties. When I run the code above I get everything but the Url.
Now the really interesting thing is if I change the container access to anonymous public and run this code, I get the Url returned as expected.
$iwrParams = #{
'Uri' = 'https://myaccount.blob.core.windows.net/mycontainer?comp=list'
'UseBasicParsing' = $true
'ContentType' = 'application/xml'
}
Invoke-WebRequest #iwrParams
The issue honestly seems to be having to include restype=container when accessing with any sort of authentication.
My question is does anyone know a way to get the URL for each blob returned when not using anonymous access to list container contents?
I'd like to do this without resorting to the Az modules.
The reason you're not able to see the URL returned in the response is because of the storage REST API version used by your code (2017-11-09). Essentially the Blob URL property was removed from the response starting with REST API version 2013-08-15. From this link:
In version 2013-08-15 and newer, the EnumerationResults element
contains a ServiceEndpoint attribute specifying the blob endpoint, and
a ContainerName field specifying the name of the container. In
previous versions these two attributes were combined together in the
ContainerName field. Also in version 2013-08-15 and newer, the Url
element under Blob has been removed.
Regarding your comment about why you can see the URL property if you list blobs anonymously, this is happening because if no REST API version is specified in the request, Storage Service uses the oldest REST API version to process the request if default version has not been set. From this link:
If an anonymous request to a general-purpose storage account does not
specify the x-ms-version header, and the default version for the
service has not been set using Set Blob Service Properties, then the
service uses the earliest possible version to process the request.
Considering you would want to use Azure AD based authorization, the earliest version you will be able to use is 2017-11-09 thus it will not be possible to get the Blob URL returned in the response body.
One option would be to manually construct the Blob URL by using Blob Container URL and Blob name. Other option would be to use Shared Key Authorization instead of Azure AD authorization and specifying a version earlier than 2013-08-15 for x-ms-version request header in your requests. You will need to manually compute Authorization header value in this case using instructions provided here.

Creating an Azure Blob Container using a SAS

I'm trying to create a new container in a Blob Storage account using the Create Container API.
https://myaccount.blob.core.windows.net/mycontainer?restype=container
I can't get this to work, I'm struggling to get the format of the Authorization header right. Other blob services I've used allow this to be passed as a query parameter.
I have the SAS token, similar to ?sv=2019-12-12&ss=bfqt&srt=sco&sp=rwdlacupx&se=2022-02-01T16:52:59Z&st=2021-02-02T08:52:59Z&spr=https&sig=r4%2B7dlSfSO8kyd8mKawHhXNtRzInq7YI%2FIbqSr1g%2FqE%3D
How do I form the Authorization header correctly to pass this?
Thanks.
To create a blob container by using Create Container rest api, if you're using "sas token", then you don't need to add "Authorization" in the Headers.
Assume you have a correct "sas token", then you the request url should look like this(Note: you should remove the first "?" from the "sas token"):
https://myaccount.blob.core.windows.net/mycontainer?restype=container&your_sas_token(note that remove the first ? from sas token)
And in the Headers, you just need pass x-ms-date and x-ms-version.
Here is the test by using Postman:
By the way, here is the screenshot about how to generate the "sas token" for creating blob container:

Generating Azure Shared Access Signatures with BlobService.getBlobURL() in Azure SDK for Node.js

I am attempting to generate a url to a blob with a Shared Access Signature using BlobService.getBlobURL() in the Azure library on Node.js on my local machine. But when I try to retrieve the blob via the generated URL, I'm getting an Authentication Error saying that the "Signature did not match". Downloading the same blob from the Azure Management Portal works fine.
Below is the code I'm using to generate the URL:
process.env['AZURE_STORAGE_ACCOUNT'] = "[MY_ACCOUNT_NAME]";
process.env['AZURE_STORAGE_ACCESS_KEY'] = "[MY_ACCESS_KEY]";
var azure = require('azure');
var blobs = azure.createBlobService();
blobs.getBlobUrl('[CONTAINER_NAME]', "[BLOB_NAME]", { AccessPolicy: {
Start: Date.now(),
Expiry: azure.date.minutesFromNow(60),
Permissions: azure.Constants.BlobConstants.SharedAccessPermissions.READ
}});
The URL generated by this function is:
https://[MY_ACCOUNT_NAME].blob.core.windows.net:443/[CONTAINER_NAME]/
[ENCODED_BLOB_NAME]
?st=2013-10-28T18%3A34%3A23Z
&se=2013-10-28T19%3A34%3A23Z
&sp=r
&sr=b
&sv=2012-02-12
&sig=rLB%2FEOAWzijkkWcseju8TJLAxzeE5e3Pvq1i68i5Erc%3D
When I try to paste this URL into a browser, I get the following error message:
<Error>
<Code>AuthenticationFailed</Code>
<Message>
Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:9fe3d3ed-97f4-43d1-8c65-c95ce6b15a08 Time:2013-10-28T18:34:43.3015398Z
</Message>
<AuthenticationErrorDetail>
Signature did not match. String to sign used was r 2013-10-28T18:34:23Z 2013-10-28T19:34:23Z /[MY_ACCOUNT_NAME]/[CONTAINER_NAME]/[BLOB_NAME] 2012-02-12
</AuthenticationErrorDetail>
</Error>
Then I tried logging on to the Azure Management Portal, selecting the same blob, and downloading it. This worked. URL provided from the Management Portal was:
http://[MY_ACCOUNT_NAME].blob.core.windows.net/[CONTAINER_NAME]/
[ENCODED_BLOB_NAME]
?sv=2012-02-12
&st=2013-10-28T18%3A35%3A16Z
&se=2013-10-28T18%3A42%3A16Z
&sr=b
&sp=r
&sig=kcjV%2BkrNAaWOj%2F7NFwmHefXJEiEyu61U7mUTsw3pw7w%3D
It appears that as of the Azure Node.js Library version 0.7.16, there is a bug causing this behavior. When a Blob name includes spaces, BlobService.getBlobURL() fails to generate a correct signature. To resolve, upload a new blob without any spaces in its name, and call BlobService.getBlobURL() again with the name of the new blob. The URL produced this time will be valid. You can check in on this issue on Github.

Resources