I've uploaded a text blob to azure storage (using UploadTextASync). Everything went OK, even downloading it again on the same workstation.
The problem is when I try downloading it from other server (stage):
The call to blob.ExistsAsync reports the blob is there;
The call to blob.DownloadTextAsync returns an empty string
Setup:
the uploaded text is an html, UTF8 encoded;
upload uses BlobRequestOptions with an IKey obtained from a CachingKeyResolver which generates a secret key in the key vault;
download uses BlobRequestOptions with a CachingKeyResolver which obtaines the same secret key
Things checked so far:
blob connection strings are the same between local and the stage server;
key vault name, client id, client secret and blob url are all the same between local and the stage server;
Updated WindowsAzure.Storage to the latest version (7.2.1) even the old one used (6.2.0) was behaving the same;
encryption mechanism is being successfully used in the same project (to upload/download octet streams); the only difference is in the method used
Recap:
DownloadText (and its async method) returns empty string when called from a different machine
Code:
public async Task<bool> UploadTextAsync(UploadTextBlobParameter parameter)
{
var container = await EnsureBlobContainerAsync(parameter);
var blob = container.GetBlockBlobReference(parameter.Path);
var blobRequestOptions = await _blobRequestOptionsManager.GetUploadBlobRequestOptions().ConfigureAwait(false);
var exists = await blob.ExistsAsync(blobRequestOptions, null).ConfigureAwait(false);
if (!parameter.Overwrite && exists)
{
return false;
}
await blob.UploadTextAsync(parameter.InputStream, parameter.Encoding, null, blobRequestOptions, null).ConfigureAwait(false);
return true;
}
public async Task<string> DownloadTextAsync(DownloadTextBlobParameter parameter)
{
var container = await EnsureBlobContainerAsync(parameter);
var blob = container.GetBlockBlobReference(parameter.Path);
var blobRequestOptions = _blobRequestOptionsManager.GetDownloadBlobRequestOptions();
var exists = await blob.ExistsAsync(blobRequestOptions, null).ConfigureAwait(false);
if (exists)
{
return await blob.DownloadTextAsync(parameter.Encoding, null, blobRequestOptions, null).ConfigureAwait(false);
}
return null;
}
Thanks in advance!
Related
I get this error when trying to upload files to blob storage. The error is present both when I run on localhost and when I run in Azure Function.
My connection string looks like:
DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xxx;EndpointSuffix=core.windows.net
Authentication information is not given in the correct format. Check the value of the Authorization header.
Time:2021-10-14T15:56:26.7659660Z
Status: 400 (Authentication information is not given in the correct format. Check the value of Authorization header.)
ErrorCode: InvalidAuthenticationInfo
But this used to work in the past but recently its started throwing this error for a new storage account I created. My code looks like below
public AzureStorageService(IOptions<AzureStorageSettings> options)
{
_connectionString = options.Value.ConnectionString;
_containerName = options.Value.ImageContainer;
_sasCredential = new StorageSharedKeyCredential(options.Value.AccountName, options.Value.Key);
_blobServiceClient = new BlobServiceClient(new BlobServiceClient(_connectionString).Uri, _sasCredential);
_containerClient = _blobServiceClient.GetBlobContainerClient(_containerName);
}
public async Task<string> UploadFileAsync(IFormFile file, string location, bool publicAccess = true)
{
try
{
await _containerClient.CreateIfNotExistsAsync(publicAccess
? PublicAccessType.Blob
: PublicAccessType.None);
var blobClient = _containerClient.GetBlobClient(location);
await using var fileStream = file.OpenReadStream();
// throws Exception here
await blobClient.UploadAsync(fileStream, true);
return blobClient.Uri.ToString();
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
// To be able to do this, I have to create the container client via the blobService client which was created along with the SharedStorageKeyCredential
public Uri GetSasContainerUri()
{
if (_containerClient.CanGenerateSasUri)
{
// Create a SAS token that's valid for one hour.
var sasBuilder = new BlobSasBuilder()
{
BlobContainerName = _containerClient.Name,
Resource = "c"
};
sasBuilder.ExpiresOn = DateTimeOffset.UtcNow.AddHours(1);
sasBuilder.SetPermissions(BlobContainerSasPermissions.Write);
var sasUri = _containerClient.GenerateSasUri(sasBuilder);
Console.WriteLine("SAS URI for blob container is: {0}", sasUri);
Console.WriteLine();
return sasUri;
}
else
{
Console.WriteLine(#"BlobContainerClient must be authorized with Shared Key
credentials to create a service SAS.");
return null;
}
}
Please change the following line of code:
_blobServiceClient = new BlobServiceClient(new BlobServiceClient(_connectionString).Uri, _sasCredential);
to
_blobServiceClient = new BlobServiceClient(_connectionString);
Considering your connection string has all the necessary information, you don't really need to do all the things you're doing and you will be using BlobServiceClient(String) constructor which expects and accepts the connection string.
You can also delete the following line of code:
_sasCredential = new StorageSharedKeyCredential(options.Value.AccountName, options.Value.Key);
and can probably get rid of AccountName and Key from your configuration settings if they are not used elsewhere.
I am trying to use StartCopyFromUri or StartCopyFromUriAsync to copy a blob from one storage account to another. Even though status.HasCompleted when I try to get the ETag either through
1. var etag = await _siteStorageClient.GetBlobETag(containerPath, asset.BlobName);
//this is the response from WaitForCompletionAsync
2. var etag = complete.GetRawResponse().Headers.Where(x => x.Name == "ETag").FirstOrDefault().Value;
I've tried both methods and both return an Etag that doesn't match what is shown in the properties of the blob when I log in through Azure Portal. It is almost as if the file wasn't done copying(or race condition) when the Etag check was executed. I couldn't find any usage samples on github for the SDK.
Any ideas what could be going awry?
This a similar question but using the older SDK. How to copy a blob from one container to another container using Azure Blob storage SDK
//Storage class
public async Task<CopyFromUriOperation> CopyFile(string containerName, string blobName, Uri sourceUri)
{
var container = _blobServiceClient.GetBlobContainerClient(containerName);
var blockBlobClient = container.GetBlockBlobClient(blobName);
//Made this the synchronous version try and block
//this is the target client
var status = await blockBlobClient.StartCopyFromUriAsync(sourceUri);
while(!status.HasCompleted)
{
//Per Documentation this calls UpdateStatusAsync() periodically
//until status.HasCompleted is true
await status.WaitForCompletionAsync();
}
return status;
}
//Calling Code
var status = await _siteStorageClient.CopyFile(container,BlobName, sasUri);
var etag = await _siteStorageClient.GetBlobETag(container, BlobName);
I was able to get this working after a few tries and troubleshooting. It would only happen in the Azure environment and not when running the web app locally.
Initially the status.WaitForCompletionAsync() was inside the loop and I started getting a socket error. I believe it was getting called too many times and was causing port exhaustion(just speculation at this point).
But this is what is working now.
public async Task<CopyFromUriOperation> CopyFile(string containerName, string blobName,Uri sourceUri)
{
var container = _blobServiceClient.GetBlobContainerClient(containerName);
var blockBlobClient = container.GetBlockBlobClient(blobName);
var status = await blockBlobClient.StartCopyFromUriAsync(sourceUri);
await status.WaitForCompletionAsync();
while(status.HasCompleted == false)
{
await Task.Delay(100);
}
return status;
}
I am trying to Implement the example shared in Learn Path
https://github.com/MicrosoftDocs/mslearn-support-concurrency-blob-storage/blob/master/src/OptimisticNewsEditor/Program.cs
I am trying to use the v12 dll which is Azure.Storage.Blobs
this is the code I have.
public static async Task Main()
{
BlobContainerClient container;
try
{
container = new BlobServiceClient(connectionString).GetBlobContainerClient(containerName);
await container.CreateIfNotExistsAsync(PublicAccessType.None);
}
catch (Exception)
{
var msg = $"Storage account not found. Ensure that the environment variable " +
" is set to a valid Azure Storage connection string and that the storage account exists.";
Console.WriteLine(msg);
return;
}
// First, the newsroom chief writes the story notes to the blob
await SimulateChief();
Console.WriteLine();
await Task.Delay(TimeSpan.FromSeconds(2));
// Next, two reporters begin work on the story at the same time, one starting soon after the other
var reporterA = SimulateReporter("Reporter A", writingTime: TimeSpan.FromSeconds(12));
await Task.Delay(TimeSpan.FromSeconds(4));
var reporterB = SimulateReporter("Reporter B", writingTime: TimeSpan.FromSeconds(4));
await Task.WhenAll(reporterA, reporterB);
await Task.Delay(TimeSpan.FromSeconds(2));
Console.WriteLine();
Console.WriteLine("=============================================");
Console.WriteLine();
Console.WriteLine("Reporters have finished, here's the story saved to the blob:");
BlobDownloadInfo story = await container.GetBlobClient(blobName).DownloadAsync();
Console.WriteLine(new StreamReader(story.Content).ReadToEnd());
}
private static async Task SimulateReporter(string authorName, TimeSpan writingTime)
{
// First, the reporter retrieves the current contents
Console.WriteLine($"{authorName} begins work");
var blob = new BlobContainerClient(connectionString, containerName).GetBlobClient(blobName);
var contents = await blob.DownloadAsync();
Console.WriteLine($"{authorName} loads the file and sees the following content: \"{new StreamReader(contents.Value.Content).ReadToEnd()}\"");
// Store the current ETag
var properties = await blob.GetPropertiesAsync();
var currentETag = properties.Value.ETag;
Console.WriteLine($"\"{contents}\" has this ETag: {properties.Value.ETag}");
// Next, the author writes their story. This takes some time.
Console.WriteLine($"{authorName} begins writing their story...");
await Task.Delay(writingTime);
Console.WriteLine($"{authorName} has finished writing their story");
try
{
// Finally, they save their story back to the blob.
var story = $"[[{authorName.ToUpperInvariant()}'S STORY]]";
await uploadDatatoBlob(blob, story);
Console.WriteLine($"{authorName} has saved their story to Blob storage. New blob contents: \"{story}\"");
}
catch (RequestFailedException e)
{
// Catch error if the ETag has changed it's value since opening the file
Console.WriteLine($"{authorName} sorry cannot save the file as server returned an error: {e.Message}");
}
}
private static async Task SimulateChief()
{
var blob = new BlobContainerClient(connectionString, containerName).GetBlobClient(blobName);
var notes = "[[CHIEF'S STORY NOTES]]";
await uploadDatatoBlob(blob, notes);
Console.WriteLine($"The newsroom chief has saved story notes to the blob {containerName}/{blobName}");
}
private static async Task uploadDatatoBlob(BlobClient blob, string notes)
{
byte[] byteArray = Encoding.UTF8.GetBytes(notes);
MemoryStream stream = new MemoryStream(byteArray);
await blob.UploadAsync(stream, overwrite: true);
}
I need to modify the UploadAsync to check for ETag before Uploading.
In the old version of Azure .Net CLI we had Microsoft.Azure.Storage.Blob dll now this handled Optimistic Concurrency by
await blob.UploadTextAsync(story, null, accessCondition: AccessCondition.GenerateIfMatchCondition(currentETag), null, null);
How do i do this in v12 dll.
Any Help appreciated.
Please use the following override of UploadAsync method:
UploadAsync(Stream, BlobHttpHeaders, IDictionary<String,String>, BlobRequestConditions, IProgress<Int64>, Nullable<AccessTier>, StorageTransferOptions, CancellationToken)
You can define the access conditions as part of BlobRequestConditions parameter.
I am writing a service that uploads / downloads items from Azure Blob storage. When I upload a file I set the ContentType.
public async Task UploadFileStream(Stream filestream, string filename, string contentType)
{
CloudBlockBlob blockBlobImage = this._container.GetBlockBlobReference(filename);
blockBlobImage.Properties.ContentType = contentType;
blockBlobImage.Metadata.Add("DateCreated", DateTime.UtcNow.ToLongDateString());
blockBlobImage.Metadata.Add("TimeCreated", DateTime.UtcNow.ToLongTimeString());
await blockBlobImage.UploadFromStreamAsync(filestream);
}
However when I retrieve the file the ContentType is null.
public async Task<CloudBlockBlob> GetBlobItem(string filename)
{
var doesBlobExist = await this.DoesBlobExist(filename);
return doesBlobExist ? this._container.GetBlockBlobReference(filename) : null;
}
In my code that uses these methods I check the ContentType of the returned Blob but it is null.
var blob = await service.GetBlobItem(blobname);
string contentType = blob.Properties.ContentType; //this is null!
I have tried using the SetProperties() method in my UploadFileStream() method (above) but this doesn't work either.
CloudBlockBlob blockBlobImage = this._container.GetBlockBlobReference(filename);
blockBlobImage.Properties.ContentType = contentType;
blockBlobImage.SetProperties(); //adding this has no effect
blockBlobImage.Metadata.Add("DateCreated", DateTime.UtcNow.ToLongDateString());
blockBlobImage.Metadata.Add("TimeCreated", DateTime.UtcNow.ToLongTimeString());
await blockBlobImage.UploadFromStreamAsync(filestream);
So how do I set the ContentType for a blob item in Azure Blob storage?
The problem is with the following line of code:
this._container.GetBlockBlobReference(filename)
Essentially this creates an instance of CloudBlockBlob on the client side. It does not make any network calls. Because this method simply creates an instance on the client side, all the properties are initialized with default values and that's why you see the ContentType property as null.
What you would need to do is actually make a network call to fetch blob's properties. You can call FetchAttributesAsync() method on CloudBlockBlob object and then you will see the ContentType property filled in properly.
Please keep in mind that FetchAttributesAsync method can throw an error (e.g. 404 in case blob does not exists) so please ensure that the call to this method is wrapped in try/catch block.
You can try code like something below:
public async Task<CloudBlockBlob> GetBlobItem(string filename)
{
try
{
var blob = this._container.GetBlockBlobReference(filename);
await blob.FetchAttributesAsync();
return blob;
}
catch (StorageException exception)
{
return null;
}
}
I've been trying to create a Windows Azure Blob containing an image file. I followed these tutorials: http://www.nickharris.net/2012/11/how-to-upload-an-image-to-windows-azure-storage-using-mobile-services/ and http://www.windowsazure.com/en-us/develop/mobile/tutorials/upload-images-to-storage-dotnet/. Finally the following code represents a merging of them. On the last line, however, an exception is raised:
An exception of type 'System.TypeLoadException' occurred in
mscorlib.ni.dll but was not handled in user code
Additional information: A binding for the specified type name was not
found. (Exception from HRESULT: 0x80132005)
Even the container is created the table, but It doesn't work properly.
private async void SendPicture()
{
StorageFile media = await StorageFile.GetFileFromPathAsync("fanny.jpg");
if (media != null)
{
//add todo item to trigger insert operation which returns item.SAS
var todoItem = new Imagem()
{
ContainerName = "mypics",
ResourceName = "Fanny",
ImageUri = "uri"
};
await imagemTable.InsertAsync(todoItem);
//Upload image direct to blob storage using SAS and the Storage Client library for Windows CTP
//Get a stream of the image just taken
using (var fileStream = await media.OpenStreamForReadAsync())
{
//Our credential for the upload is our SAS token
StorageCredentials cred = new StorageCredentials(todoItem.SasQueryString);
var imageUri = new Uri(todoItem.SasQueryString);
// Instantiate a Blob store container based on the info in the returned item.
CloudBlobContainer container = new CloudBlobContainer(
new Uri(string.Format("https://{0}/{1}",
imageUri.Host, todoItem.ContainerName)), cred);
// Upload the new image as a BLOB from the stream.
CloudBlockBlob blobFromSASCredential =
container.GetBlockBlobReference(todoItem.ResourceName);
await blobFromSASCredential.UploadFromStreamAsync(fileStream.AsInputStream());
}
}
}
Please use Assembly Binding Log Viewer to see which load is failing. As also mentioned in the article, the common language runtime's failure to locate an assembly typically shows up as a TypeLoadException in your application.