Cannot set the ContentType of a blob in Azure Data Lake Gen2 in csharp - azure

I'm using the Azure.Storage.Files.DataLake nuget package to write and append file on my Azure Storage account that is enabled for Data Lake Gen2 (including hierarchy).
However, I need to set the Content Type of a new file I create and I don't succeed in it, although I thought I had things written correctly. Here's my code:
public async Task<bool> WriteBytes(string fileName, byte[] recordContent, string contentType)
{
var directory = await CreateDirectoryIfNotExists();
var fileClient = await directory.CreateFileAsync(fileName,
new PathHttpHeaders
{
ContentType = contentType
})).Value;
long recordSize = recordContent.Length;
var recordStream = new MemoryStream(recordContent);
await fileClient.AppendAsync(recordStream, offset: 0);
await fileClient.FlushAsync(position: recordSize);
return true;
}
The result after the execution of the above code looks like this (the default content-type is kept):
Thanks for any insights

I am able to reproduce the behavior you're seeing. Basically the issue is with fileClient.FlushAsync() method. Before calling this method, I checked the content type of the file and it was set properly. However after execution of this method, the content type was changed to application/octet-stream (which is the default).
Looking at the documentation for this method here, you can also set the headers in this method. I tried doing the same and the content type changed to the desired one.
var headers = new PathHttpHeaders()
{
ContentType = "text/plain"
};
await fileClient.FlushAsync(position: recordSize, httpHeaders: headers);

Related

Azure.Storage.Blobs.BlobServiceClient CopyFromUri() doesn't seem to be finished copying before returning latest ETag

I am trying to use StartCopyFromUri or StartCopyFromUriAsync to copy a blob from one storage account to another. Even though status.HasCompleted when I try to get the ETag either through
1. var etag = await _siteStorageClient.GetBlobETag(containerPath, asset.BlobName);
//this is the response from WaitForCompletionAsync
2. var etag = complete.GetRawResponse().Headers.Where(x => x.Name == "ETag").FirstOrDefault().Value;
I've tried both methods and both return an Etag that doesn't match what is shown in the properties of the blob when I log in through Azure Portal. It is almost as if the file wasn't done copying(or race condition) when the Etag check was executed. I couldn't find any usage samples on github for the SDK.
Any ideas what could be going awry?
This a similar question but using the older SDK. How to copy a blob from one container to another container using Azure Blob storage SDK
//Storage class
public async Task<CopyFromUriOperation> CopyFile(string containerName, string blobName, Uri sourceUri)
{
var container = _blobServiceClient.GetBlobContainerClient(containerName);
var blockBlobClient = container.GetBlockBlobClient(blobName);
//Made this the synchronous version try and block
//this is the target client
var status = await blockBlobClient.StartCopyFromUriAsync(sourceUri);
while(!status.HasCompleted)
{
//Per Documentation this calls UpdateStatusAsync() periodically
//until status.HasCompleted is true
await status.WaitForCompletionAsync();
}
return status;
}
//Calling Code
var status = await _siteStorageClient.CopyFile(container,BlobName, sasUri);
var etag = await _siteStorageClient.GetBlobETag(container, BlobName);
I was able to get this working after a few tries and troubleshooting. It would only happen in the Azure environment and not when running the web app locally.
Initially the status.WaitForCompletionAsync() was inside the loop and I started getting a socket error. I believe it was getting called too many times and was causing port exhaustion(just speculation at this point).
But this is what is working now.
public async Task<CopyFromUriOperation> CopyFile(string containerName, string blobName,Uri sourceUri)
{
var container = _blobServiceClient.GetBlobContainerClient(containerName);
var blockBlobClient = container.GetBlockBlobClient(blobName);
var status = await blockBlobClient.StartCopyFromUriAsync(sourceUri);
await status.WaitForCompletionAsync();
while(status.HasCompleted == false)
{
await Task.Delay(100);
}
return status;
}

Azure Blob Storage - Content Type Setting on Download and Upload

I am trying to upload a word document to blob storage using C#. The code snippet is as below:
var blobServiceClient = new BlobServiceClient(connectionString);
var containerClient = blobServiceClient.GetBlobContainerClient(container);
containerClient.CreateIfNotExists(PublicAccessType.None);
var blobClient = containerClient.GetBlobClient(documentName);
using (var memoryStream = new MemoryStream({Binary of the File}))
{
var headers = new BlobHttpHeaders() { ContentType = "application/vnd.openxmlformats-officedocument.wordprocessingml.document" };
await blobClient.UploadAsync(memoryStream, httpHeaders: headers).ConfigureAwait(false);
}
The document gets uploaded successfully to the Blob Storage. I can see that the content type is also being set properly while looking through Azure Storage Explorer. But, when i try to access this document (using document URL) through browser (chrome), it downloads the file as unknown file.
I tried the same by uploading a word document through Azure Storage Explorer. This file gets downloaded as word document while downloading it through browser.
Any idea what is going wrong?
As you see the figure below, the Content-Type header is stored as a property in the Blob Properties, so it will be set into the headers of the download response and then it will be recognized as a word file while download by browser.
So when you upload a word file, it's necessary to set the ContentType property for a blob via BlobBaseClient.SetHttpHeaders(BlobHttpHeaders, BlobRequestConditions, CancellationToken) Method or BlobBaseClient.SetHttpHeadersAsync(BlobHttpHeaders, BlobRequestConditions, CancellationToken) Method.
And then to write the Content-Typeheader of the headers of a download response with the value of contentType below by yourself on your server app.
Response<BlobProperties> response = await blobClient.GetPropertiesAsync();
var contentType = response.Value.ContentType
Or the download response from a blob url with sas token which headers default include the ContentType property as the Content-Type header.

Set the ContentType on an Azure Blob Storage item

I am writing a service that uploads / downloads items from Azure Blob storage. When I upload a file I set the ContentType.
public async Task UploadFileStream(Stream filestream, string filename, string contentType)
{
CloudBlockBlob blockBlobImage = this._container.GetBlockBlobReference(filename);
blockBlobImage.Properties.ContentType = contentType;
blockBlobImage.Metadata.Add("DateCreated", DateTime.UtcNow.ToLongDateString());
blockBlobImage.Metadata.Add("TimeCreated", DateTime.UtcNow.ToLongTimeString());
await blockBlobImage.UploadFromStreamAsync(filestream);
}
However when I retrieve the file the ContentType is null.
public async Task<CloudBlockBlob> GetBlobItem(string filename)
{
var doesBlobExist = await this.DoesBlobExist(filename);
return doesBlobExist ? this._container.GetBlockBlobReference(filename) : null;
}
In my code that uses these methods I check the ContentType of the returned Blob but it is null.
var blob = await service.GetBlobItem(blobname);
string contentType = blob.Properties.ContentType; //this is null!
I have tried using the SetProperties() method in my UploadFileStream() method (above) but this doesn't work either.
CloudBlockBlob blockBlobImage = this._container.GetBlockBlobReference(filename);
blockBlobImage.Properties.ContentType = contentType;
blockBlobImage.SetProperties(); //adding this has no effect
blockBlobImage.Metadata.Add("DateCreated", DateTime.UtcNow.ToLongDateString());
blockBlobImage.Metadata.Add("TimeCreated", DateTime.UtcNow.ToLongTimeString());
await blockBlobImage.UploadFromStreamAsync(filestream);
So how do I set the ContentType for a blob item in Azure Blob storage?
The problem is with the following line of code:
this._container.GetBlockBlobReference(filename)
Essentially this creates an instance of CloudBlockBlob on the client side. It does not make any network calls. Because this method simply creates an instance on the client side, all the properties are initialized with default values and that's why you see the ContentType property as null.
What you would need to do is actually make a network call to fetch blob's properties. You can call FetchAttributesAsync() method on CloudBlockBlob object and then you will see the ContentType property filled in properly.
Please keep in mind that FetchAttributesAsync method can throw an error (e.g. 404 in case blob does not exists) so please ensure that the call to this method is wrapped in try/catch block.
You can try code like something below:
public async Task<CloudBlockBlob> GetBlobItem(string filename)
{
try
{
var blob = this._container.GetBlockBlobReference(filename);
await blob.FetchAttributesAsync();
return blob;
}
catch (StorageException exception)
{
return null;
}
}

Azure Storage Search Blobs by Metadata

I have CloudBlockBlobs that have metadata.
CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob.jpg");
using (var fileStream = System.IO.File.OpenRead(filePath))
{
blockBlob.UploadFromStream(fileStream);
blockBlob.Properties.ContentType = "image/jpg";
blockBlob.Metadata.Add("Title", "Yellow Pear");
blockBlob.SetProperties();
}
I see the Metadata is there:
Debug.WriteLine(blockBlob.Metadata["Title"]);
Now later if I query from storage I see the blobs but the Metadata is missing:
(in the below I know blobItems[0] had Metadata when uploaded but now blobItems[0].Metadata.Count == 0)
var blobItems = container.ListBlobs(
null, false, BlobListingDetails.Metadata);
I also noticed the Metadata is not available when I obtain the blob by itself:
CloudBlockBlob a = container.GetBlockBlobReference("myblob.jpg");
//Below throws an exception
var b = a.Metadata["Title"];
Thank you!
There are some issues with your code :(.
The blob doesn't have any metadata set actually. After setting the metadata, you're calling blob.SetProperties() method which only sets the blob's properties (ContentType in your example). To set the metadata, you would actually need to call blob.SetMetadata() method.
Your upload code is currently making 2 calls to storage service: 1) upload blob and 2) set properties. If you call SetMetadata then it would be 3 calls. IMHO, these can be combined in just 1 call to storage service by doing something like below:
using (var fileStream = System.IO.File.OpenRead(filePath))
{
blockBlob.Properties.ContentType = "image/jpg";
blockBlob.Metadata.Add("Title", "Yellow Pear");
blockBlob.UploadFromStream(fileStream);
}
This will not only upload the blob but also set it's properties and metadata in a single call to storage service.
Regarding
I also noticed the Metadata is not available when I obtain the blob by
itself:
CloudBlockBlob a = container.GetBlockBlobReference("myblob.jpg");
//Below throws an exception
var b = a.Metadata["Title"];
Basically the code above is just creating an instance of the blob on the client side. It doesn't actually fetch the properties (and metadata) of the blob. To fetch details about the blob, you would need to call FetchAttributes method on the blob. Something like:
CloudBlockBlob a = container.GetBlockBlobReference("myblob.jpg");
a.FetchAttributes();
If after that you retrieve blob's metadata, you should be able to see it (provided metadata was created properly).

Windows Azure Blob

I've been trying to create a Windows Azure Blob containing an image file. I followed these tutorials: http://www.nickharris.net/2012/11/how-to-upload-an-image-to-windows-azure-storage-using-mobile-services/ and http://www.windowsazure.com/en-us/develop/mobile/tutorials/upload-images-to-storage-dotnet/. Finally the following code represents a merging of them. On the last line, however, an exception is raised:
An exception of type 'System.TypeLoadException' occurred in
mscorlib.ni.dll but was not handled in user code
Additional information: A binding for the specified type name was not
found. (Exception from HRESULT: 0x80132005)
Even the container is created the table, but It doesn't work properly.
private async void SendPicture()
{
StorageFile media = await StorageFile.GetFileFromPathAsync("fanny.jpg");
if (media != null)
{
//add todo item to trigger insert operation which returns item.SAS
var todoItem = new Imagem()
{
ContainerName = "mypics",
ResourceName = "Fanny",
ImageUri = "uri"
};
await imagemTable.InsertAsync(todoItem);
//Upload image direct to blob storage using SAS and the Storage Client library for Windows CTP
//Get a stream of the image just taken
using (var fileStream = await media.OpenStreamForReadAsync())
{
//Our credential for the upload is our SAS token
StorageCredentials cred = new StorageCredentials(todoItem.SasQueryString);
var imageUri = new Uri(todoItem.SasQueryString);
// Instantiate a Blob store container based on the info in the returned item.
CloudBlobContainer container = new CloudBlobContainer(
new Uri(string.Format("https://{0}/{1}",
imageUri.Host, todoItem.ContainerName)), cred);
// Upload the new image as a BLOB from the stream.
CloudBlockBlob blobFromSASCredential =
container.GetBlockBlobReference(todoItem.ResourceName);
await blobFromSASCredential.UploadFromStreamAsync(fileStream.AsInputStream());
}
}
}
Please use Assembly Binding Log Viewer to see which load is failing. As also mentioned in the article, the common language runtime's failure to locate an assembly typically shows up as a TypeLoadException in your application.

Resources