How to programmatically find out what operations I can do in a blob storage? - azure

I am using libraries Microsoft.Azure.Storage.Blob 11.2.3.0 and Microsoft.Azure.Storage.Common 11.2.3.0 to connect to an Azure BlobStorage from a .NET Core 3.1 application.
When I started working on this, I had been given connection strings that gave me full access to the BlobStorage (or rather, the entire cloud storage account). Based upon those, I chose to write my connection code "defensively", making use of Exists() and CreateIfNotExists() from the CloudBlobContainer class to ensure the application would not fail when a container was not yet existing.
Now, I'm connecting a BlobStorage container using a SAS. While I can freely retrieve and upload blobs within the container like this, unfortunately, it seems that I am not allowed to do anything on the container level. Not only CreateIfNotExists, but even the mere querying of existence by Exists() throws a StorageException saying
This request is not authorized to perform this operation.
The documentation does not mention the exception.
Is there any way to check preemptively whether I am allowed to check the container's existence?
I have tried looking into the container permissions retrieved from GetPermissions, but that will throw an exception, as well.
The only other alternative I can see is to check for container existence within a try-catch-block and assume existence if an exception is thrown ...

There's a no definitive way to identify if an operation can be performed using a SAS token other than performing that operation and catching any exception that may be thrown by the operation. The exception that is of your interest is Unauthorized (403).
However you can try to predict if an operation can be performed by looking at the SAS token. If it is a Service SAS Token and not an Account SAS Token, that means all the account related operations are not not allowed. The way to distinguish between an Account SAS token and a Service SAS token is that the former will contain attributes like SignedServices (ss) and SignedResourceTypes (srt).
Next thing you would want to do is look for SignedPermissions (sp) attribute in your SAS token. This attribute will tell you what all operations are possible with the SAS token. For example, if your SAS token is a Service SAS token and if it includes Delete (d) permission, that would mean you can use this SAS token to delete a blob.
Please see these tables for the permissions/allowed operations combinations:
Service SAS Token: https://learn.microsoft.com/en-us/rest/api/storageservices/create-service-sas#permissions-for-a-directory-container-or-blob
Account SAS Token: https://learn.microsoft.com/en-us/rest/api/storageservices/create-service-sas#permissions-for-a-directory-container-or-blob
Please note that the operation might still fail for any number of reasons like SAS token has expired, account key has changed since the generation of SAS token, IP restrictions etc.

I tried in in my system to check whether the container exist or not able check it and if container not exists created container and able to upload file.
You need to give proper permission for your SAS Token
const string sasToken = “SAS Token”
const string accountName = "teststorage65";
const string blobContainerName = "example";
const string blobName = "test.txt";
const string myFileLocation = #"Local Path ";
var storageAccount = new CloudStorageAccount(storageCredentials, accountName, null, true);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer blobContainer = blobClient.GetContainerReference(blobContainerName);
var result=blobContainer.Exists();
if (result == true)
{
Console.WriteLine("Container exists");
}
else
{
// blobContainer.CreateIfNotExists();
Console.WriteLine("Conatiner not exists");
Console.WriteLine("Creating Container "+ blobContainerName);
blobContainer.CreateIfNotExists();
}
// blobContainer.CreateIfNotExists();
//Console.WriteLine("Creating Container ");
CloudBlockBlob cloudBlob = blobContainer.GetBlockBlobReference(blobName);
cloudBlob.UploadFromFile(myFileLocation);
OUTPUT

Related

AzureBlobStorage fails when enumerating Pageable<BlobContainerItem> result from GetBlobContainers with RequestFailedException, not authorized

I am building an infrastructure wrapper around Azure Blob Storage for a shared component in a .Net Core 6 suite of applications, both web and desktop-based applications. I’m using version 12.14.1 of Azure.Storage.Blobs.
public BlobContainerItem[] GetContainers(string sasToken)
{
blobUri = "https://spiffyaccountname.blob.core.windows.net?" + sasToken;
var client = new BlobServiceClient(blobUri);
var containers = client.GetBlobContainers(); // Succeeds!
var array = containers.array() // Fails with Azure.RequestFailedException
return array;
}
Exception Details:
Azure.RequestFailedException: This request is not authorized to perform this operation using this resource type.
RequestId:...
Time:...
Status: 403 (This request is not authorized to perform this operation using this resource type.)
ErrorCode: AuthorizationResourceTypeMismatch
I don't think it's a code issue but rather a configuration issue, as creating and deleting containers work fine, though I haven't been able to find the setting to fix it.
More than likely the reason you are getting this error is because of incorrect permissions in your SAS token.
Please check for 2 things:
Use Account SAS - Listing containers require that you use Account SAS instead of Service SAS.
Use proper permissions - Listing containers require that your account SAS has SignedServices as Blob (ss=b), SignedResourceTypes as Service (srt=s) and SignedPermission should have at least List permission (sp=l).

I want to generate SAS URL dynamically via C# code for Azure Blob Container. Using this SAS URL we must be able to upload the files

I want to generate SAS URL dynamically via C# code for Azure Blob Container. Using this SAS URL we must be able to upload the files to the Azure Blob Container. I have tried multiple ways to generate the SAS URL by following the Microsoft docs. But I am always getting AuthorizationResourceTypeMismatch Error or AuthorizationPermissionMismatch.
Error: AuthorizationPermissionMismatch This request is not authorized to perform this operation using this permission.
private static Uri GetServiceSasUriForContainer(BlobContainerClient containerClient,
string storedPolicyName = null)
{
// Check whether this BlobContainerClient object has been authorized with Shared Key.
if (containerClient.CanGenerateSasUri)
{
// Create a SAS token that's valid for one hour.
BlobSasBuilder sasBuilder = new BlobSasBuilder()
{
BlobContainerName = containerClient.Name,
Resource = "c"
};
if (storedPolicyName == null)
{
sasBuilder.ExpiresOn = DateTimeOffset.UtcNow.AddHours(1);
sasBuilder.SetPermissions(BlobContainerSasPermissions.Read);
}
else
{
sasBuilder.Identifier = storedPolicyName;
}
Uri sasUri = containerClient.GenerateSasUri(sasBuilder);
Console.WriteLine("SAS URI for blob container is: {0}", sasUri);
Console.WriteLine();
return sasUri;
}
else
{
Console.WriteLine(#"BlobContainerClient must be authorized with Shared Key
credentials to create a service SAS.");
return null;
}
}
Error: AuthenticationFailed Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
I need this sas url because I use this url in my javascript to upload files into the Azure Blob Container.
Can someone help me out achieving this goal?
The reason you're getting this error is because you are creating the SAS token with Read permission (BlobContainerSasPermissions.Read).
In order to upload a blob in a container using SAS URL, the SAS token needs either Write (BlobContainerSasPermissions.Write) or Create (BlobContainerSasPermissions.Create) permission. Please create a SAS token with one of these permissions and you should not get this error.
To learn more about the permissions, please see this link: https://learn.microsoft.com/en-us/rest/api/storageservices/create-service-sas#permissions-for-a-directory-container-or-blob.

Unable to Get/Post Blob on Azure Storage via Rest API in C#

Upon successful consumption of Azure Rest api in c# code. I'm able to create, fetch and fetch list of containers but not blob. While accessing or uploading blobs, this gives permission issue and i.e.,
you are not authorized to perform this request with assigned permission
Progress that i have made so far:
Able to create and fetch the container.
Able to fetch the list of all the containers of storage account.
When tries to Get/Put a blob via below source code it give me error:
This request is not authorized to perform this operation using this permission.
string endPointUri = $"{azureApplicationConfiguration.Resource}/{inpContainerName}/{inpBlobName}";
var request = (HttpWebRequest)WebRequest.Create(endPointUri);
request.Method = HTTPMethod.GET.ToString();
request.Headers.Add("Authorization", $"Bearer {sasToken.access_token}");
request.Headers.Add("x-ms-date", DateTime.UtcNow.ToString("R"));
request.Headers.Add("x-ms-version", "2018-03-28");
request.ProtocolVersion = HttpVersion.Version11;
using (HttpWebResponse resp = (HttpWebResponse)request.GetResponse())
{
Console.WriteLine(resp.StatusCode.ToString());
Console.ReadKey();
}
The most likely answer is that the SAS token you're using to authenticate does not support object level access. You can confirm this by checking if there is a letter o in the Signed Resource Types srt= parameter in your SAS token. Based on the fact you can list and create containers, I'd guess the current value is srt=sc. To be able to perform GET/PUT operations on a blob as well, you'll need to generate a new SAS token that includes object level permissions, which should give you a token containing srt=sco.
In case you aren't aware of it already, there is a .NET Azure Storage SDK available, which provides a layer of abstraction over the REST API and may save you some time in the long run.

How to use SharedAccessSignature to access blobs

I am trying to access a blob stored in a private container in Windows Azure. The container has a Shared Access Signature but when I try
to access the blob I get a StorgeClientException "Server failed to authenticate the request. Make sure the Authorization header is formed
correctly including the signature".
The code that created the container and uploaded the blob looks like this:
// create the container, set a Shared Access Signature, and share it
// first this to do is to create the connnection to the storage account
// this should be in app.config but as this isa test it will just be implemented
// here:
// add a reference to Microsoft.WindowsAzure.StorageClient
// and Microsoft.WindowsAzure.StorageClient set up the objects
//storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
storageAccount = CloudStorageAccount.Parse(ConfigurationManager.AppSettings["ConnectionString"]);
blobClient = storageAccount.CreateCloudBlobClient();
// get a reference tot he container for the shared access signature
container = blobClient.GetContainerReference("blobcontainer");
container.CreateIfNotExist();
// now create the permissions policy to use and a public access setting
var permissions = container.GetPermissions();
permissions.SharedAccessPolicies.Remove("accesspolicy");
permissions.SharedAccessPolicies.Add("accesspolicy", new SharedAccessPolicy
{
// this policy is live immediately
// if the policy should be delatyed then use:
//SharedAccessStartTime = DateTime.Now.Add(T); where T is some timespan
SharedAccessExpiryTime =
DateTime.UtcNow.AddYears(2),
Permissions =
SharedAccessPermissions.Read | SharedAccessPermissions.Write
});
// turn off public access
permissions.PublicAccess = BlobContainerPublicAccessType.Off;
// set the permission on the ocntianer
container.SetPermissions(permissions);
var sas = container.GetSharedAccessSignature(new SharedAccessPolicy(), "accesspolicy");
StorageCredentialsSharedAccessSignature credentials = new StorageCredentialsSharedAccessSignature(sas);
CloudBlobClient client = new CloudBlobClient(storageAccount.BlobEndpoint,
new StorageCredentialsSharedAccessSignature(sas));
CloudBlob sasblob = client.GetBlobReference("blobcontainer/someblob.txt");
sasblob.UploadText("I want to read this text via a rest call");
// write the SAS to file so I can use it later in other apps
using (var writer = new StreamWriter(#"C:\policy.txt"))
{
writer.WriteLine(container.GetSharedAccessSignature(new SharedAccessPolicy(), "securedblobpolicy"));
}
The code I have been trying to use to read the blob looks like this:
// the storace credentials shared access signature is copied directly from the text file "c:\policy.txt"
CloudBlobClient client = new CloudBlobClient("https://my.azurestorage.windows.net/", new StorageCredentialsSharedAccessSignature("?sr=c&si=accesspolicy&sig=0PMoXpht2TF1Jr0uYPfUQnLaPMiXrqegmjYzeg69%2FCI%3D"));
CloudBlob blob = client.GetBlobReference("blobcontainer/someblob.txt");
Console.WriteLine(blob.DownloadText());
Console.ReadLine();
I can make the above work by adding the account credentials but that is exactly what I'm trying to avoid. I do not want something
as sensitive as my account credentials just sitting out there and I have no idea on how to get the signature into the client app without having the account credentials.
Any help is greatly appreciated.
Why this?
writer.WriteLine(container.GetSharedAccessSignature(new SharedAccessPolicy(), "securedblobpolicy"));
and not writing the sas string you already created?
It's late and I could easily be missing something but it seems that you might not be saving the same access signature that you're using to write the file in the first place.
Also perhaps not relevant here but I believe there is a limit on the number of container-wide policies you can have. Are you uploading multiple files to the same container with this code and creating a new container sas each time?
In general I think it would be better to request a sas for an individual blob at the time you need it with a short expiry time.
Is "my.azurestorage.windows.net" just a typo? I would expect something there like "https://account.blob.core.windows.net".
Otherwise the code looks pretty similar to the code in http://blog.smarx.com/posts/shared-access-signatures-are-easy-these-days, which works.

Azure Storage Connection Check

I have a console application which uploads jobs to the workers running in the cloud. The application connects to Azure Storage and uploads some files to blobs and put some messages to queues. Currently, I am using the development storage. I actually want to know at which state my client application connects to the storage. Can I create a queueClient even though I do not have any connection at all? At which step it actually tries to send some network packages? I actually need some kind of a mechanism in order to check the existence of the connection and the validity of the storage account.
The client doesn't send any messages until you call a command on the storage - e.g. until you try to get or put a property of a blob, container, or queue - e.g. in the sample code below (from http://msdn.microsoft.com/en-us/library/gg651129.aspx) then messages are sent in 3 specific places:
// Variables for the cloud storage objects.
CloudStorageAccount cloudStorageAccount;
CloudBlobClient blobClient;
CloudBlobContainer blobContainer;
BlobContainerPermissions containerPermissions;
CloudBlob blob;
// Use the local storage account.
cloudStorageAccount = CloudStorageAccount.DevelopmentStorageAccount;
// If you want to use Windows Azure cloud storage account, use the following
// code (after uncommenting) instead of the code above.
// cloudStorageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=http;AccountName=your_storage_account_name;AccountKey=your_storage_account_key");
// Create the blob client, which provides
// authenticated access to the Blob service.
blobClient = cloudStorageAccount.CreateCloudBlobClient();
// Get the container reference.
blobContainer = blobClient.GetContainerReference("mycontainer");
// Create the container if it does not exist.
// MESSAGE SENT
blobContainer.CreateIfNotExist();
// Set permissions on the container.
containerPermissions = new BlobContainerPermissions();
// This sample sets the container to have public blobs. Your application
// needs may be different. See the documentation for BlobContainerPermissions
// for more information about blob container permissions.
containerPermissions.PublicAccess = BlobContainerPublicAccessType.Blob;
// MESSAGE SENT
blobContainer.SetPermissions(containerPermissions);
// Get a reference to the blob.
blob = blobContainer.GetBlobReference("myfile.txt");
// Upload a file from the local system to the blob.
Console.WriteLine("Starting file upload");
// MESSAGE SENT
blob.UploadFile(#"c:\myfiles\myfile.txt"); // File from local storage.
Console.WriteLine("File upload complete to blob " + blob.Uri);
One way to check whether you have connectivity is to use some of the simple functions like CreateIfNotExist() or GetPermissions() on a known container - these give you a simple quick method to check connectivity. You can also use the BlobRequestOptions to specify a timeout to make sure your app doesn't hang (http://msdn.microsoft.com/en-us/library/ee772886.aspx)
Be careful not to check connectivity too frequently - every 10000 successful checks will cost you $0.01
Adding to what Stuart has written above, what you can do is try and list 1 queue from your storage account using CloudQueueClient.ListQueuesSegmented Method (http://msdn.microsoft.com/en-us/library/ff361716.aspx). We're really not interested in the result as such. What we're more interested in is if we get a storage client exception or not which you will get if the credentials are not correct. Even if you don't have a queue in your storage account, as long as you have passed correct credentials you will not get an error.
Somehow I don't feel creating an object would be the right thing to test storage account credentials. In the past we've seen situations where a storage account was made read only and even if you pass valid credentials and try to see if the credentials are correct by creating an object, you would get an error.
Hope this helps.
Here's how we check if the connection is ready:
public async Task<bool> IsReady()
{
var options = new BlobRequestOptions { ServerTimeout = TimeSpan.FromSeconds(2), MaximumExecutionTime = TimeSpan.FromSeconds(2) };
var container = GetStorageClient().GetContainerReference("status");
var blobRef = container.GetBlockBlobReference("health-check");
try
{
await blobRef.DownloadTextAsync(null, options, null);
}
catch (StorageException e)
{
if (e.RequestInformation.HttpStatusCode != 404)
return false;
try
{
await container.CreateIfNotExistsAsync(options, null);
await blobRef.UploadTextAsync("health check", AccessCondition.GenerateEmptyCondition(), options, null);
}
catch (Exception)
{
return false;
}
}
return true;
}
GetStorageClient() is a custom method, replace with your own
What it does:
We try to read a test file because reading is much cheaper than write operations
If the exception is "file not found" let's try to create the file
If exception is other than 404 we can stop here, trying to write should make no sense
Note: When using "cool" storage using an "other operation" would be cheaper. See https://azure.microsoft.com/en-us/pricing/details/storage/blobs/

Resources