I'm having trouble with Azure Blobs and Shared Access Signatures when they expire. I need to grant access to a blob for longer than 1 hour (7 days), so I'm using a named container policy, but unfortunately I can't seem to generate new urls once those 7 days are up.
I have the following code to create the "default" policy. Note in this code, I'm setting the expiration to be 1 minute from now, to make it easier to test:
CloudStorageAccount account = new CloudStorageAccount(credentials, true);
CloudBlobClient client = new CloudBlobClient(account.BlobEndpoint, credentials);
CloudBlobContainer container = client.GetContainerReference("files");
SharedAccessPolicy sharedAccessPolicy = new SharedAccessPolicy();
sharedAccessPolicy.Permissions = SharedAccessPermissions.Read;
sharedAccessPolicy.SharedAccessStartTime = DateTime.UtcNow;
sharedAccessPolicy.SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(1);
BlobContainerPermissions blobContainerPermissions = new BlobContainerPermissions();
blobContainerPermissions.SharedAccessPolicies.Add("default", sharedAccessPolicy);
container.SetPermissions(blobContainerPermissions);
I then create a SharedAccessSignature url with the following:
CloudStorageAccount account = new CloudStorageAccount(credentials, true);
CloudBlobClient client = new CloudBlobClient(account.BlobEndpoint, credentials);
CloudBlobContainer container = client.GetContainerReference("files");
CloudBlob blob = container.GetBlobReference(path);
string sas = blob.GetSharedAccessSignature(new SharedAccessPolicy(), "default");
Console.WriteLine(blob.Uri.AbsoluteUri + sas);
This generates a url, and the url works properly for the next minute (or 7 days in the real code). Once the one minute is over, the url is invalid and no longer works, as expected.
But once that expiration is past, I run the code again to generate a new url. Unfortunately, it generates the same url, which is still invalid.
Are the start/end times for container policies absolute, meaning when I set that policy right now:
sharedAccessPolicy.SharedAccessStartTime = DateTime.UtcNow;
sharedAccessPolicy.SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(1);
anything using that policy is only valid from 10:10am (EDT) to 10:11am (EDT) today?
One thing you could do is create your access policy without expiry date. You specify the expiry date when you're creating the signed URL.
So your code would look something like:
SharedAccessPolicy sharedAccessPolicy = new SharedAccessPolicy();
sharedAccessPolicy.Permissions = SharedAccessPermissions.Read;
sharedAccessPolicy.SharedAccessStartTime = DateTime.UtcNow;
//sharedAccessPolicy.SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(1); No need to define expiry time here.
BlobContainerPermissions blobContainerPermissions = new BlobContainerPermissions();
blobContainerPermissions.SharedAccessPolicies.Add("default", sharedAccessPolicy);
container.SetPermissions(blobContainerPermissions);
Console.WriteLine("Press any key to continue....");
Console.ReadLine();
CloudBlob blob = container.GetBlobReference(path);
string sas = blob.GetSharedAccessSignature(new SharedAccessPolicy()
{
SharedAccessExpiryTime = DateTime.UtcNow.AddDays(7),//add expiry date only when you're creating the signed URL
}
, "default");
Console.WriteLine(blob.Uri.AbsoluteUri + sas);
Process.Start(new ProcessStartInfo(blob.Uri.AbsoluteUri + sas));
Console.WriteLine("Press any key to continue....");
Console.ReadLine();
Will this work for you? Obviously you would need to regenerate the URL after 7 days but you don't have to make any changes to your access policy.
Hope this helps.
With a 1 minute expiration you might be hitting clock skew effects between the SAS generation box and Windows Azure Storage. You should use a longer interval. I did a post going into the gory depths of shared access signatures, that you might find helpful.
You may be hitting the maximum on container level access policies.
A stored access policy includes a name up to 64 characters long that is unique within the container. This name appears in the signedidentifier field on Shared Access Signatures that link to a stored access policy. A container can include up to 5 stored access policies. Each policy can be used by any number of Shared Access Signatures.
Using a Stored Access Policy
Related
I am trying to figure out how to generate a time based token for users to access data that is stored in an Azure Storage Account Blob container. Users upload various data (PDFs, images) but I don't want links to this data to be public. The recommended strategy was to use a SAS token which I was able to get this working under .Net using the following function which I found on the MS site about a year ago:
//Function for getting a temporary Azure SAS
public static string GetAzureSASToken(string userhashid, int minutes)
{
//Get Azure SAS Token so we can allow them to temporarily view the photos
UploadedFileInfo uploadedfileinfo = new UploadedFileInfo();
//Azure User containter must be all lowercase!!
uploadedfileinfo.usercontainer = "user-" + userhashid;
uploadedfileinfo.azureurl = ConfigurationManager.AppSettings["AzureURL"].ToString() + uploadedfileinfo.usercontainer + "/";
// Retrieve storage account from connection string.
string azureconnection = CloudConfigurationManager.GetSetting("StorageConnectionString");
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(azureconnection);
CloudBlobClient client = storageAccount.CreateCloudBlobClient();
CloudBlobContainer blobContainer = client.GetContainerReference(uploadedfileinfo.usercontainer);
//Set the expiry time and permissions for the container.
//In this case no start time is specified, so the shared access signature becomes valid immediately.
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessExpiryTime = DateTimeOffset.UtcNow.AddMinutes(minutes);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
//Generate the shared access signature on the container, setting the constraints directly on the signature.
string sasContainerToken = blobContainer.GetSharedAccessSignature(sasConstraints);
return sasContainerToken;
}
The problem is that I now need to access these files from an Asp.Net Core 2.2 app and I can't seem to figure out how to replicate the code to get the token (the Core libraries are different)
Any suggestions on how to accomplish this in .Net Core 2.2?
Thanks!
Just install the latest nuget package Microsoft.Azure.Storage.Blob -Version 11.1.0.
Then your .net core code( same as .net framework code in your post) can work well as .net framework code.
Here is an sample code of .net core 2.2. I didn't read the settings from configure file, so it's a little different:
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Auth;
using Microsoft.Azure.Storage.Blob;
using System;
namespace ConsoleApp5
{
class Program
{
static void Main(string[] args)
{
string sas = GetAzureSASToken();
Console.WriteLine(sas);
Console.ReadLine();
}
public static string GetAzureSASToken()
{
string accountName = "xxx";
string accountKey = "xxx";
CloudStorageAccount storageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
CloudBlobClient client = storageAccount.CreateCloudBlobClient();
CloudBlobContainer blobContainer = client.GetContainerReference("test1");
//Set the expiry time and permissions for the container.
//In this case no start time is specified, so the shared access signature becomes valid immediately.
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessExpiryTime = DateTimeOffset.UtcNow.AddMinutes(5);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
//Generate the shared access signature on the container, setting the constraints directly on the signature.
string sasContainerToken = blobContainer.GetSharedAccessSignature(sasConstraints);
return sasContainerToken;
}
}
}
And the test result:
I'm trying to create SAS URIs / Tokens to allow download of my Azure Storage Blobs.
I'd like to do this on a blob-level, in order to not inadvertently give access to an unintended resource.
The current code I use to do this is:
public static string GetBlobSasUri(string containerName, string reference)
{
// Create the CloudBlobContainer object
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
container.CreateIfNotExists();
// Get a reference to a blob within the container.
CloudBlockBlob blob = container.GetBlockBlobReference(reference);
// Set the expiry time and permissions for the blob.
// In this case, the start time is specified as a few minutes in the past, to mitigate clock skew.
// The shared access signature will be valid immediately.
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessStartTime = DateTimeOffset.UtcNow.AddMinutes(-5);
sasConstraints.SharedAccessExpiryTime = DateTimeOffset.UtcNow.AddMonths(1);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
// Generate the shared access signature on the blob, setting the constraints directly on the signature.
string sasBlobToken = blob.GetSharedAccessSignature(sasConstraints);
// Return the URI string for the container, including the SAS token.
return blob.Uri + sasBlobToken;
}
This is largely based on the example in Documentation here:
Generate a shared access signature URI for a blob
This works. However, I see in other SAS documentation that it is possible to restrict to a certain IP range as well:
Service SAS Uri Example
My understanding of SAS tokens is that the signature signs all parameters, so I don't think this is as easy as just appending my IP range to the SAS URI returned from the code I pasted above, since the signature would then not match.
However, the SharedAccessBlobPolicy only has three fields, which are the start/end times of the access, as well as the permissions. I don't see anything about IP ranges.
Is it possible to set these permitted ranges when generating SAS URIs at the blob level, not for a full account?
Please use the code below:
public static string GetBlobSasUri(string ipAddressFrom, string ipAddressTo)
{
CloudStorageAccount storageAccount = new CloudStorageAccount(new StorageCredentials("account_name", "account_key"), true);
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
var cloudBlobContainer = cloudBlobClient.GetContainerReference("test-1");
cloudBlobContainer.CreateIfNotExists();
CloudBlockBlob blob = cloudBlobContainer.GetBlockBlobReference("a.txt");
var ipAddressRange = new IPAddressOrRange(ipAddressFrom, ipAddressTo);
var sasBlobToken = blob.GetSharedAccessSignature(new SharedAccessBlobPolicy()
{
Permissions = SharedAccessBlobPermissions.List,
SharedAccessExpiryTime = new DateTimeOffset(DateTime.UtcNow.AddHours(1))
}, null, null,null, ipAddressRange);
return blob.Uri + sasBlobToken;
}
I need an advice here first, then tech details.
My app generated PDF's and stores them in Azure Storage in a private container. But when user authenticates (I use Azure AD B2C) and goes to his personal page I need to show links to those PDF's. Now, those links must not be public, so I think I need:
1) some kind of middleware to auth the user when he accesses those kind of links
2) to request the file from Storage and pass it on to the response
What's the best way of doing this? (considering performance too)
My first idea was to use SAS tokens and just limit time for about 5-10 minutes. But what if user opens the page and leaves his browser open for an hour, then comes back and click on PDF link?
I agree with #Federico Dipuma. I used this way in one of my projects. I am here to share my code.
Code of generate SAS URL.
public string GetBlobSasUri(string containerName, string blobName, string connectionstring)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionstring);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference(containerName);
CloudBlockBlob blockBlob = container.GetBlockBlobReference(blobName);
//Set the expiry time and permissions for the blob.
//In this case no start time is specified, so the shared access signature becomes valid immediately.
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(10);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
//Generate the shared access signature on the blob, setting the constraints directly on the signature.
string sasContainerToken = blockBlob.GetSharedAccessSignature(sasConstraints);
//Return the URI string for the blob, including the SAS token.
return blockBlob.Uri + sasContainerToken;
}
Redirect to the URL in your web application.
public ActionResult FileDownload()
{
string blobURL = GetBlobSasUri("blob name","container name", "connection string");
return Redirect(blobURL);
}
i have written this below code to get the blob url with cache expiry token, actually have set 2 hours to expire the blob url,
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
CloudBlockBlob blockBlob = container.GetBlockBlobReference("blobname");
//Create an ad-hoc Shared Access Policy with read permissions which will expire in 2 hours
SharedAccessBlobPolicy policy = new SharedAccessBlobPolicy()
{
Permissions = SharedAccessBlobPermissions.Read,
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(2),
};
SharedAccessBlobHeaders headers = new SharedAccessBlobHeaders()
{
ContentDisposition = string.Format("attachment;filename=\"{0}\"", "blobname"),
};
var sasToken = blockBlob.GetSharedAccessSignature(policy, headers);
blobUrl = blockBlob.Uri.AbsoluteUri + sasToken;
using this above code i get the blob url with valid expiry token, now i want to check blob url is valid or not in one client application.
I tried web request and http client approach by passing the URL and get the response status code. if the response code is 404 then I assuming the URL is expired if not the URL is still valid,but this approach taking more time.
Please suggest me any other way.
I tried running code very similar to yours, and I am getting a 403 error, which is actually what is expected in this case. Based on your question, I am not sure whether the 403 is more helpful to you than the 404. Here is code running in a console application that returns a 403:
class Program
{
static void Main(string[] args)
{
string blobUrl = CreateSAS();
CheckSAS(blobUrl);
Console.ReadLine();
}
//This method returns a reference to the blob with the SAS, and attempts to read it.
static void CheckSAS(string blobUrl)
{
CloudBlockBlob blob = new CloudBlockBlob(new Uri(blobUrl));
//If the DownloadText() method is run within the two minute period that the SAS is valid, it succeeds.
//If it is run after the SAS has expired, it returns a 403 error.
//Sleep for 3 minutes to trigger the error.
System.Threading.Thread.Sleep(180000);
Console.WriteLine(blob.DownloadText());
}
//This method creates the SAS on the blob.
static string CreateSAS()
{
string containerName = "forum-test";
string blobName = "blobname";
string blobUrl = "";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
container.CreateIfNotExists();
CloudBlockBlob blockBlob = container.GetBlockBlobReference(blobName + DateTime.Now.Ticks);
blockBlob.UploadText("Blob for forum test");
//Create an ad-hoc Shared Access Policy with read permissions which will expire in 2 hours
SharedAccessBlobPolicy policy = new SharedAccessBlobPolicy()
{
Permissions = SharedAccessBlobPermissions.Read,
SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(2),
};
SharedAccessBlobHeaders headers = new SharedAccessBlobHeaders()
{
ContentDisposition = string.Format("attachment;filename=\"{0}\"", blobName),
};
var sasToken = blockBlob.GetSharedAccessSignature(policy, headers);
blobUrl = blockBlob.Uri.AbsoluteUri + sasToken;
return blobUrl;
}
}
There are cases in which SAS failures do return a 404, which can create problems for troubleshooting operations using SAS. The Azure Storage team is aware of this issue and in future releases SAS failures may return a 403 instead. For help troubleshooting a 404 error, see http://azure.microsoft.com/en-us/documentation/articles/storage-monitoring-diagnosing-troubleshooting/#SAS-authorization-issue.
I also ran into the same issue a few days back. I was actually expecting storage service to return a 403 error code when the SAS token has expired but storage service returns 404 error.
Given that we don't have any other option, the way you're doing it is the only viable way but it is still not correct because you could get 404 error if the blob is not present in the storage account.
Maybe you can parse "se" argument from the generated SAS, which means expiry time, e.g. "se=2013-04-30T02%3A23%3A26Z". However, since the server time might not be the same as client time, the solution may be unstable.
http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-shared-access-signature-part-1/
You're using UTC time for SharedAccessExpiryTime (see "Expiry Time" in https://learn.microsoft.com/en-us/azure/storage/common/storage-dotnet-shared-access-signature-part-1#parameters-common-to-account-sas-and-service-sas-tokens).
The expiry time then is registered under the se claim in the token the value of which can be checked against current UTC time on the client side before actually using the token. This way you'd save yourself from making an extra call to the Blob storage only to find out whether the token is expired.
I have a fairly simple method that uses the NEW Storage API to create a SAS and copy a blob from one container to another.
I am trying to use this to Copy blob BETWEEN STORAGE ACCOUNTS. So I have TWo Storage accounts, with the exact same Containers, and I am trying to copy a blob from the Storage Account's Container to another Storage Account's Container.
I don't know if the SDK is built for that, but it seems like it would be a common scenario.
Some additional information:
I create the token on the Destination Container.
Does that token need to be created on both the source and destination? Does it take time to register the token? Do I need to create it for each request, or only once per token "lifetime"?
I should mention a 403 is an Unauthorized Result http error code.
private static string CreateSharedAccessToken(CloudBlobClient blobClient, string containerName)
{
var container = blobClient.GetContainerReference(containerName);
var blobPermissions = new BlobContainerPermissions();
// The shared access policy provides read/write access to the container for 10 hours:
blobPermissions.SharedAccessPolicies.Add("SolutionPolicy", new SharedAccessBlobPolicy()
{
// To ensure SAS is valid immediately we don’t set start time
// so we can avoid failures caused by small clock differences:
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(1),
Permissions = SharedAccessBlobPermissions.Write |
SharedAccessBlobPermissions.Read
});
blobPermissions.PublicAccess = BlobContainerPublicAccessType.Blob;
container.SetPermissions(blobPermissions);
return container.GetSharedAccessSignature(new SharedAccessBlobPolicy(), "SolutionPolicy");
}
Down the line I use this token to call a copy operation, which returns a 403:
var uri = new Uri(srcBlob.Uri.AbsoluteUri + blobToken);
destBlob.StartCopyFromBlob(uri);
My version of Azure.Storage is 2.1.0.2.
Here is the full copy method in case that helps:
private static void CopyBlobs(
CloudBlobContainer srcContainer, string blobToken,
CloudBlobContainer destContainer)
{
var srcBlobList
= srcContainer.ListBlobs(string.Empty, true, BlobListingDetails.All); // set to none in prod (4perf)
//// get the SAS token to use for all blobs
//string token = srcContainer.GetSharedAccessSignature(
// new SharedAccessBlobPolicy(), "SolutionPolicy");
bool pendingCopy = true;
foreach (var src in srcBlobList)
{
var srcBlob = src as ICloudBlob;
// Determine BlobType:
ICloudBlob destBlob;
if (srcBlob.Properties.BlobType == BlobType.BlockBlob)
{
destBlob = destContainer.GetBlockBlobReference(srcBlob.Name);
}
else
{
destBlob = destContainer.GetPageBlobReference(srcBlob.Name);
}
// Determine Copy State:
if (destBlob.CopyState != null)
{
switch (destBlob.CopyState.Status)
{
case CopyStatus.Failed:
log.Info(destBlob.CopyState);
break;
case CopyStatus.Aborted:
log.Info(destBlob.CopyState);
pendingCopy = true;
destBlob.StartCopyFromBlob(destBlob.CopyState.Source);
return;
case CopyStatus.Pending:
log.Info(destBlob.CopyState);
pendingCopy = true;
break;
}
}
// copy using only Policy ID:
var uri = new Uri(srcBlob.Uri.AbsoluteUri + blobToken);
destBlob.StartCopyFromBlob(uri);
//// copy using src blob as SAS
//var source = new Uri(srcBlob.Uri.AbsoluteUri + token);
//destBlob.StartCopyFromBlob(source);
}
}
And finally the account and client (vetted) code:
var credentials = new StorageCredentials("BAR", "FOO");
var account = new CloudStorageAccount(credentials, true);
var blobClient = account.CreateCloudBlobClient();
var sasToken = CreateSharedAccessToken(blobClient, "content");
When I use a REST client this seems to work... any ideas?
Consider also this problem:
var uri = new Uri(srcBlob.Uri.AbsoluteUri + blobToken);
Probably you are calling the "ToString" method of Uri that produce a "Human redable" version of the url. If the blobToken contain special caracters like for example "+" this will cause a token malformed error on the storage server that will refuse to give you the access.
Use this instead:
String uri = srcBlob.Uri.AbsoluteUri + blobToken;
Shared Access Tokens are not required for this task. I ended up with two accounts and it works fine:
var accountSrc = new CloudStorageAccount(credsSrc, true);
var accountDest = new CloudStorageAccount(credsSrc, true);
var blobClientSrc = accountSrc.CreateCloudBlobClient();
var blobClientDest = accountDest.CreateCloudBlobClient();
// Set permissions on the container.
var permissions = new BlobContainerPermissions {PublicAccess = BlobContainerPublicAccessType.Blob};
srcContainer.SetPermissions(permissions);
destContainer.SetPermissions(permissions);
//grab the blob
var sourceBlob = srcContainer.GetBlockBlobReference("FOO");
var destinationBlob = destContainer.GetBlockBlobReference("BAR");
//create a new blob
destinationBlob.StartCopyFromBlob(sourceBlob);
Since both CloudStorageAccount objects point to the same account, copying without a SAS token would work just fine as you also mentioned.
On the other hand, you need either a publicly accessible blob or a SAS token to copy from another account. So what you tried was correct, but you established a container-level access policy, which can take up to 30 seconds to take effect as also documented in MSDN. During this interval, a SAS token that is associated with the stored access policy will fail with status code 403 (Forbidden), until the access policy becomes active.
One more thing that I would like to point is; when you call Get*BlobReference to create a new blob object, the CopyState property will not be populated until you do a GET/HEAD operation such as FetchAttributes.