I have a SAS token which will expire within 2 minutes.
SAS = AzureClient.GetCloudContainer().GetSharedAccessSignature(new SharedAccessPolicy()
{
SharedAccessExpiryTime = DateTime.UtcNow + TimeSpan.FromMinutes(1)
}, "readonly");
var sasCreds = new StorageCredentialsSharedAccessSignature(SAS);
CloudStorageAccount _storageAccount = AzureClient.GetCloudStorageAccount();
CloudBlobClient sasBlobClient = new CloudBlobClient(_storageAccount.BlobEndpoint, sasCreds);
CloudBlob sasBlob = sasBlobClient.GetBlobReference("blobname");
Where readonly is the policy name.
Now I am doing the following operation:
using (BlobStream stream = sasBlob.OpenRead())
{
using (FileStream fileStream = File.OpenWrite(#"Smething.txt"))
{
BlobStreamReader(stream,fileStream);
}
}
private void BlobStreamReader(BlobStream blob,Stream OutputStream)
{
int buffersize = 4194304; // 4MB
byte[] data = new byte[buffersize];
do
{
int bytesRead = blob.Read(data,0,buffersize);
if (bytesRead == 0) break;
OutputStream.Write(data,0,bytesRead);
}
while (true);
}
The problem is that download is failing when the SAS is expired. I had an understanding that the SAS token is needed only for authentication and if download is started with its expiry time then download will continue even if the SAS is expired.
It is correct that the SAS token is needed only for authentication. However, in your case, BlobStream will issue a new request whenever it needs more data from the server. Because each request needs to be authenticated separately and your SAS token expires before the entire download is finished, it is expected to fail.
If you want to download the entire blob, DownloadToStream is actually a better alternative, because it will only issue a single request to the server and then download the entire blob.
Related
My application generates SAS tokens to access existing blobs within my container. However, my SAS token dose not look like it is expiring. I am able to view and get blob from container way past expiration time I am claiming.
Here is the code :
public string GenerateSasToken([NotNull] string containerName, [NotNull] string blobName)
{
var startTime = DateTimeOffset.UtcNow
var expiredTime = startTime.AddSeconds(20);
var blobClient = new BlobClient(_options.Value.ConnectionString, containerName, blobName);
var sasBuilder = new BlobSasBuilder(BlobContainerSasPermissions.Read, expiredTime)
{
BlobName = blobName,
BlobContainerName = containerName,
StartsOn = startTime,
ExpiresOn = expiredTime
};
var uri = blobClient.GenerateSasUri(sasBuilder);
return uri.ToString();
}
Token been generated is valid and I am able to use it, but it dose not expire after 20 seconds in fact it dose not expire even after 15 minutes.
Am I missing something within this API?
Thank you!
Edit:
I am attaching SAS token that was generated.
?sv=2020-08-04&st=2022-01-24T21%3A20%3A41Z&se=2022-01-24T21%3A21%3A01Z&sr=b&sp=r&sig=signature-here
Even though the SAS token is expired, because of the browser caching, you would still be able to access the blob storage using the same SAS token
In order to avoid this, you can override the cache-control header in the SAS token as suggested by #Gaurav Mantri
You need to set CacheControl value in your BlobSasBuilder function to override the cache-control header
Your BlobSasBuilder function can be as below:
var sasBuilder = new BlobSasBuilder(BlobContainerSasPermissions.Read, expiredTime)
{
BlobName = blobName,
BlobContainerName = containerName,
StartsOn = startTime,
ExpiresOn = expiredTime,
CacheControl = "max-age=" + expiredTime
};
I am trying to figure out how to generate a time based token for users to access data that is stored in an Azure Storage Account Blob container. Users upload various data (PDFs, images) but I don't want links to this data to be public. The recommended strategy was to use a SAS token which I was able to get this working under .Net using the following function which I found on the MS site about a year ago:
//Function for getting a temporary Azure SAS
public static string GetAzureSASToken(string userhashid, int minutes)
{
//Get Azure SAS Token so we can allow them to temporarily view the photos
UploadedFileInfo uploadedfileinfo = new UploadedFileInfo();
//Azure User containter must be all lowercase!!
uploadedfileinfo.usercontainer = "user-" + userhashid;
uploadedfileinfo.azureurl = ConfigurationManager.AppSettings["AzureURL"].ToString() + uploadedfileinfo.usercontainer + "/";
// Retrieve storage account from connection string.
string azureconnection = CloudConfigurationManager.GetSetting("StorageConnectionString");
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(azureconnection);
CloudBlobClient client = storageAccount.CreateCloudBlobClient();
CloudBlobContainer blobContainer = client.GetContainerReference(uploadedfileinfo.usercontainer);
//Set the expiry time and permissions for the container.
//In this case no start time is specified, so the shared access signature becomes valid immediately.
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessExpiryTime = DateTimeOffset.UtcNow.AddMinutes(minutes);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
//Generate the shared access signature on the container, setting the constraints directly on the signature.
string sasContainerToken = blobContainer.GetSharedAccessSignature(sasConstraints);
return sasContainerToken;
}
The problem is that I now need to access these files from an Asp.Net Core 2.2 app and I can't seem to figure out how to replicate the code to get the token (the Core libraries are different)
Any suggestions on how to accomplish this in .Net Core 2.2?
Thanks!
Just install the latest nuget package Microsoft.Azure.Storage.Blob -Version 11.1.0.
Then your .net core code( same as .net framework code in your post) can work well as .net framework code.
Here is an sample code of .net core 2.2. I didn't read the settings from configure file, so it's a little different:
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Auth;
using Microsoft.Azure.Storage.Blob;
using System;
namespace ConsoleApp5
{
class Program
{
static void Main(string[] args)
{
string sas = GetAzureSASToken();
Console.WriteLine(sas);
Console.ReadLine();
}
public static string GetAzureSASToken()
{
string accountName = "xxx";
string accountKey = "xxx";
CloudStorageAccount storageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
CloudBlobClient client = storageAccount.CreateCloudBlobClient();
CloudBlobContainer blobContainer = client.GetContainerReference("test1");
//Set the expiry time and permissions for the container.
//In this case no start time is specified, so the shared access signature becomes valid immediately.
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessExpiryTime = DateTimeOffset.UtcNow.AddMinutes(5);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
//Generate the shared access signature on the container, setting the constraints directly on the signature.
string sasContainerToken = blobContainer.GetSharedAccessSignature(sasConstraints);
return sasContainerToken;
}
}
}
And the test result:
I had my project working in Core 1, but when I changed to Core 2 it,no longer uploads the images to Azure.I get a 404 response code and "Problem sending file to Azure" message. The correct URL is returned from the server, but then,finuploader calls a URL with current URL in Front
The URL in chrome Console returning the 404 is shown as
https://localhost:44348/House/%22https://Customstorage.blob.core.windows.net/images/b0975cc7-fae5-4130-bced-c26272d0a21c.jpeg?sv=2017-04-17&sr=b&sig=UFUEsHT1GWI%2FfMd9cuHmJsl2j05W1Acux52UZ5IsXso%3D&se=2017-09-16T04%3A06%3A36Z&sp=rcwd%22
This is being added to the URL somewhere
https://localhost:44348/House/%22
I create the SAS with
public async Task<string> SAS(string bloburi, string name)
{
CloudStorageAccount storageAccount = new CloudStorageAccount(
new Microsoft.WindowsAzure.Storage.Auth.StorageCredentials(
"<storage-account-name>",
"<access-key>"), true)
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("images");
return await Task.FromResult<string>(GetBlobSasUri(container, bloburi));
}
private static string GetBlobSasUri(CloudBlobContainer container, string blobName, string policyName = null)
{
string sasBlobToken;
// Get a reference to a blob within the container.
// Note that the blob may not exist yet, but a SAS can still be created for it.
var uri = new Uri(blobName);
//get the name of the file from the path
string filename = System.IO.Path.GetFileName(uri.LocalPath);
CloudBlockBlob blob = container.GetBlockBlobReference(filename);
if (policyName == null)
{
// Create a new access policy and define its constraints.
// Note that the SharedAccessBlobPolicy class is used both to define the parameters of an ad-hoc SAS, and
// to construct a shared access policy that is saved to the container's shared access policies.
SharedAccessBlobPolicy adHocSAS = new SharedAccessBlobPolicy()
{
// When the start time for the SAS is omitted, the start time is assumed to be the time when the storage service receives the request.
// Omitting the start time for a SAS that is effective immediately helps to avoid clock skew.
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(1),
Permissions = SharedAccessBlobPermissions.Read | SharedAccessBlobPermissions.Write | SharedAccessBlobPermissions.Create | SharedAccessBlobPermissions.Delete
};
// Generate the shared access signature on the blob, setting the constraints directly on the signature.
sasBlobToken = blob.GetSharedAccessSignature(adHocSAS);
}
else
{
// Generate the shared access signature on the blob. In this case, all of the constraints for the
// shared access signature are specified on the container's stored access policy.
sasBlobToken = blob.GetSharedAccessSignature(null, policyName);
}
// Return the URI string for the container, including the SAS token.
return blob.Uri + sasBlobToken;
}
Fine-uploader -
var uploader = new qq.azure.FineUploader({
element: document.getElementById('uploader'),
template: 'qq-template',
autoUpload: false,
request: {
endpoint:'https://customstorage.blob.core.windows.net/images'
},
signature: {
endpoint: '/House/SAS'
},
uploadSuccess: {
endpoint: '/House/UploadImage'
}
});
i have written this below code to get the blob url with cache expiry token, actually have set 2 hours to expire the blob url,
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
CloudBlockBlob blockBlob = container.GetBlockBlobReference("blobname");
//Create an ad-hoc Shared Access Policy with read permissions which will expire in 2 hours
SharedAccessBlobPolicy policy = new SharedAccessBlobPolicy()
{
Permissions = SharedAccessBlobPermissions.Read,
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(2),
};
SharedAccessBlobHeaders headers = new SharedAccessBlobHeaders()
{
ContentDisposition = string.Format("attachment;filename=\"{0}\"", "blobname"),
};
var sasToken = blockBlob.GetSharedAccessSignature(policy, headers);
blobUrl = blockBlob.Uri.AbsoluteUri + sasToken;
using this above code i get the blob url with valid expiry token, now i want to check blob url is valid or not in one client application.
I tried web request and http client approach by passing the URL and get the response status code. if the response code is 404 then I assuming the URL is expired if not the URL is still valid,but this approach taking more time.
Please suggest me any other way.
I tried running code very similar to yours, and I am getting a 403 error, which is actually what is expected in this case. Based on your question, I am not sure whether the 403 is more helpful to you than the 404. Here is code running in a console application that returns a 403:
class Program
{
static void Main(string[] args)
{
string blobUrl = CreateSAS();
CheckSAS(blobUrl);
Console.ReadLine();
}
//This method returns a reference to the blob with the SAS, and attempts to read it.
static void CheckSAS(string blobUrl)
{
CloudBlockBlob blob = new CloudBlockBlob(new Uri(blobUrl));
//If the DownloadText() method is run within the two minute period that the SAS is valid, it succeeds.
//If it is run after the SAS has expired, it returns a 403 error.
//Sleep for 3 minutes to trigger the error.
System.Threading.Thread.Sleep(180000);
Console.WriteLine(blob.DownloadText());
}
//This method creates the SAS on the blob.
static string CreateSAS()
{
string containerName = "forum-test";
string blobName = "blobname";
string blobUrl = "";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
container.CreateIfNotExists();
CloudBlockBlob blockBlob = container.GetBlockBlobReference(blobName + DateTime.Now.Ticks);
blockBlob.UploadText("Blob for forum test");
//Create an ad-hoc Shared Access Policy with read permissions which will expire in 2 hours
SharedAccessBlobPolicy policy = new SharedAccessBlobPolicy()
{
Permissions = SharedAccessBlobPermissions.Read,
SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(2),
};
SharedAccessBlobHeaders headers = new SharedAccessBlobHeaders()
{
ContentDisposition = string.Format("attachment;filename=\"{0}\"", blobName),
};
var sasToken = blockBlob.GetSharedAccessSignature(policy, headers);
blobUrl = blockBlob.Uri.AbsoluteUri + sasToken;
return blobUrl;
}
}
There are cases in which SAS failures do return a 404, which can create problems for troubleshooting operations using SAS. The Azure Storage team is aware of this issue and in future releases SAS failures may return a 403 instead. For help troubleshooting a 404 error, see http://azure.microsoft.com/en-us/documentation/articles/storage-monitoring-diagnosing-troubleshooting/#SAS-authorization-issue.
I also ran into the same issue a few days back. I was actually expecting storage service to return a 403 error code when the SAS token has expired but storage service returns 404 error.
Given that we don't have any other option, the way you're doing it is the only viable way but it is still not correct because you could get 404 error if the blob is not present in the storage account.
Maybe you can parse "se" argument from the generated SAS, which means expiry time, e.g. "se=2013-04-30T02%3A23%3A26Z". However, since the server time might not be the same as client time, the solution may be unstable.
http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-shared-access-signature-part-1/
You're using UTC time for SharedAccessExpiryTime (see "Expiry Time" in https://learn.microsoft.com/en-us/azure/storage/common/storage-dotnet-shared-access-signature-part-1#parameters-common-to-account-sas-and-service-sas-tokens).
The expiry time then is registered under the se claim in the token the value of which can be checked against current UTC time on the client side before actually using the token. This way you'd save yourself from making an extra call to the Blob storage only to find out whether the token is expired.
I have a fairly simple method that uses the NEW Storage API to create a SAS and copy a blob from one container to another.
I am trying to use this to Copy blob BETWEEN STORAGE ACCOUNTS. So I have TWo Storage accounts, with the exact same Containers, and I am trying to copy a blob from the Storage Account's Container to another Storage Account's Container.
I don't know if the SDK is built for that, but it seems like it would be a common scenario.
Some additional information:
I create the token on the Destination Container.
Does that token need to be created on both the source and destination? Does it take time to register the token? Do I need to create it for each request, or only once per token "lifetime"?
I should mention a 403 is an Unauthorized Result http error code.
private static string CreateSharedAccessToken(CloudBlobClient blobClient, string containerName)
{
var container = blobClient.GetContainerReference(containerName);
var blobPermissions = new BlobContainerPermissions();
// The shared access policy provides read/write access to the container for 10 hours:
blobPermissions.SharedAccessPolicies.Add("SolutionPolicy", new SharedAccessBlobPolicy()
{
// To ensure SAS is valid immediately we don’t set start time
// so we can avoid failures caused by small clock differences:
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(1),
Permissions = SharedAccessBlobPermissions.Write |
SharedAccessBlobPermissions.Read
});
blobPermissions.PublicAccess = BlobContainerPublicAccessType.Blob;
container.SetPermissions(blobPermissions);
return container.GetSharedAccessSignature(new SharedAccessBlobPolicy(), "SolutionPolicy");
}
Down the line I use this token to call a copy operation, which returns a 403:
var uri = new Uri(srcBlob.Uri.AbsoluteUri + blobToken);
destBlob.StartCopyFromBlob(uri);
My version of Azure.Storage is 2.1.0.2.
Here is the full copy method in case that helps:
private static void CopyBlobs(
CloudBlobContainer srcContainer, string blobToken,
CloudBlobContainer destContainer)
{
var srcBlobList
= srcContainer.ListBlobs(string.Empty, true, BlobListingDetails.All); // set to none in prod (4perf)
//// get the SAS token to use for all blobs
//string token = srcContainer.GetSharedAccessSignature(
// new SharedAccessBlobPolicy(), "SolutionPolicy");
bool pendingCopy = true;
foreach (var src in srcBlobList)
{
var srcBlob = src as ICloudBlob;
// Determine BlobType:
ICloudBlob destBlob;
if (srcBlob.Properties.BlobType == BlobType.BlockBlob)
{
destBlob = destContainer.GetBlockBlobReference(srcBlob.Name);
}
else
{
destBlob = destContainer.GetPageBlobReference(srcBlob.Name);
}
// Determine Copy State:
if (destBlob.CopyState != null)
{
switch (destBlob.CopyState.Status)
{
case CopyStatus.Failed:
log.Info(destBlob.CopyState);
break;
case CopyStatus.Aborted:
log.Info(destBlob.CopyState);
pendingCopy = true;
destBlob.StartCopyFromBlob(destBlob.CopyState.Source);
return;
case CopyStatus.Pending:
log.Info(destBlob.CopyState);
pendingCopy = true;
break;
}
}
// copy using only Policy ID:
var uri = new Uri(srcBlob.Uri.AbsoluteUri + blobToken);
destBlob.StartCopyFromBlob(uri);
//// copy using src blob as SAS
//var source = new Uri(srcBlob.Uri.AbsoluteUri + token);
//destBlob.StartCopyFromBlob(source);
}
}
And finally the account and client (vetted) code:
var credentials = new StorageCredentials("BAR", "FOO");
var account = new CloudStorageAccount(credentials, true);
var blobClient = account.CreateCloudBlobClient();
var sasToken = CreateSharedAccessToken(blobClient, "content");
When I use a REST client this seems to work... any ideas?
Consider also this problem:
var uri = new Uri(srcBlob.Uri.AbsoluteUri + blobToken);
Probably you are calling the "ToString" method of Uri that produce a "Human redable" version of the url. If the blobToken contain special caracters like for example "+" this will cause a token malformed error on the storage server that will refuse to give you the access.
Use this instead:
String uri = srcBlob.Uri.AbsoluteUri + blobToken;
Shared Access Tokens are not required for this task. I ended up with two accounts and it works fine:
var accountSrc = new CloudStorageAccount(credsSrc, true);
var accountDest = new CloudStorageAccount(credsSrc, true);
var blobClientSrc = accountSrc.CreateCloudBlobClient();
var blobClientDest = accountDest.CreateCloudBlobClient();
// Set permissions on the container.
var permissions = new BlobContainerPermissions {PublicAccess = BlobContainerPublicAccessType.Blob};
srcContainer.SetPermissions(permissions);
destContainer.SetPermissions(permissions);
//grab the blob
var sourceBlob = srcContainer.GetBlockBlobReference("FOO");
var destinationBlob = destContainer.GetBlockBlobReference("BAR");
//create a new blob
destinationBlob.StartCopyFromBlob(sourceBlob);
Since both CloudStorageAccount objects point to the same account, copying without a SAS token would work just fine as you also mentioned.
On the other hand, you need either a publicly accessible blob or a SAS token to copy from another account. So what you tried was correct, but you established a container-level access policy, which can take up to 30 seconds to take effect as also documented in MSDN. During this interval, a SAS token that is associated with the stored access policy will fail with status code 403 (Forbidden), until the access policy becomes active.
One more thing that I would like to point is; when you call Get*BlobReference to create a new blob object, the CopyState property will not be populated until you do a GET/HEAD operation such as FetchAttributes.