Azure CloudBlobContainer.CreateIfNotExists() throws the container already exists error with crashes webpage - azure

I have a asp.net core app that uploads file to azure blob storage. The upload completes successfully but when CloudBlobContainer calls CreateIfNotExistAsync the web page crashes with a "The container already exists error".
var container = BlobClient.GetContainerReference(containerName.ToString().ToLower());
await container.CreateIfNotExistsAsync();
return container;
I have tried using surronding CreateIfNotExistsAsync() with the following if
if (! await container.ExistsAsync())
but that still errors.
The container name and the AccountName= in the connection string is lowercase.
I am using the latest stable Microsoft.Azure.Storage.Blob NuGet package 10.0.2
I have tried to catch the StroageExeception but the exception is not called
try
{
await container.CreateIfNotExistsAsync();
}
catch (StorageException ex)
{
Logger.LogError(ex.ToString());
}
I have been through all the points in this previous question but none of them apply/work in my scenario Azure Blob 400 Bad request on Creation of container
public class CloudBlobStorageProvider : ICloudBlobStorageProvider
{
private CloudBlobClient BlobClient { get; }
private ILogger Logger { get; }
public CloudBlobStorageProvider(IConfiguration configuration, ILogger<CloudBlobStorageProvider> logger)
{
var connectionString = configuration.GetConnectionString("AzureStorageAccount");
if (!CloudStorageAccount.TryParse(connectionString, out CloudStorageAccount storageAccount))
{
logger.LogError($"The supplied connection string for Azure blob storage could not be parsed: {connectionString}");
}
BlobClient = storageAccount.CreateCloudBlobClient();
}
public async Task<CloudBlobContainer> GetContainerAsync(CloudBlobContainerName containerName)
{
var container = BlobClient.GetContainerReference(containerName.ToString().ToLower());
await container.CreateIfNotExistsAsync(BlobContainerPublicAccessType.Off, null, null);
return container;
}
}
public interface ICloudBlobStorageProvider
{
Task<CloudBlobContainer> GetContainerAsync(CloudBlobContainerName containerName);
}
Which is called by
public async Task<CloudBlockBlob> UploadServiceUserAttachmentAsync(IFormFile formFile)
{
var fileExtenion = RegularExpressionHelpers.GetFileExtension(formFile.FileName);
string attachmentFileName = (string.IsNullOrEmpty(fileExtenion)) ? $"{Guid.NewGuid().ToString()}" : $"{Guid.NewGuid().ToString()}{fileExtenion}";
var userAttachmentContainer = await CloudBlobStorageProvider.GetContainerAsync(CloudBlobContainerName.userattachments);
var blobBlockReference = userAttachmentContainer.GetBlockBlobReference(attachmentFileName);
try
{
using (var stream = formFile.OpenReadStream())
{
await blobBlockReference.UploadFromStreamAsync(stream);
await blobBlockReference.FetchAttributesAsync();
var blobProperties = blobBlockReference.Properties;
blobProperties.ContentType = formFile.ContentType;
await blobBlockReference.SetPropertiesAsync();
}
}
catch (Exception e)
{
Logger.LogWarning(e, $"Exception encountered while attempting to write profile photo to blob storage");
}
return blobBlockReference;
}

Related

Blazor server side upload file to Azure Blob Storage

I am using a file upload component that posts the file to an API controller and this works ok but i need to get the progres of the upload.
[HttpPost("upload/single")]
public async Task SingleAsync(IFormFile file)
{
try
{
// Azure connection string and container name passed as an argument to get the Blob reference of the container.
var container = new BlobContainerClient(azureConnectionString, "upload-container");
// Method to create our container if it doesn’t exist.
var createResponse = await container.CreateIfNotExistsAsync();
// If container successfully created, then set public access type to Blob.
if (createResponse != null && createResponse.GetRawResponse().Status == 201)
await container.SetAccessPolicyAsync(Azure.Storage.Blobs.Models.PublicAccessType.Blob);
// Method to create a new Blob client.
var blob = container.GetBlobClient(file.FileName);
// If a blob with the same name exists, then we delete the Blob and its snapshots.
await blob.DeleteIfExistsAsync(Azure.Storage.Blobs.Models.DeleteSnapshotsOption.IncludeSnapshots);
// Create a file stream and use the UploadSync method to upload the Blob.
uploadFileSize = file.Length;
var progressHandler = new Progress<long>();
progressHandler.ProgressChanged += UploadProgressChanged;
using (var fileStream = file.OpenReadStream())
{
await blob.UploadAsync(fileStream, new BlobHttpHeaders { ContentType = file.ContentType },progressHandler:progressHandler);
}
Response.StatusCode = 400;
}
catch (Exception e)
{
Response.Clear();
Response.StatusCode = 204;
Response.HttpContext.Features.Get<IHttpResponseFeature>().ReasonPhrase = "File failed to upload";
Response.HttpContext.Features.Get<IHttpResponseFeature>().ReasonPhrase = e.Message;
}
}
private double GetProgressPercentage(double totalSize, double currentSize)
{
return (currentSize / totalSize) * 100;
}
private void UploadProgressChanged(object sender, long bytesUploaded)
{
uploadPercentage = GetProgressPercentage(uploadFileSize, bytesUploaded);
}
I am posting this file and it does upload but the file upload progress event is inaccurate it says the file upload is complete after a few seconds when in reality the file takes ~90 secs on my connection to appear in the Azure Blob Storage container.
So in the code above i have the progress handler which works (I can put a break point on it and see it increasing) but how do I return this value to the UI?
I found one solution that used Microsoft.AspNetCore.SignalR but I can't manage to integrate this into my own code and I'm not even sure if I'm on the right track.
using BlazorReportProgress.Server.Hubs;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.SignalR;
using System.Threading;
namespace BlazorReportProgress.Server.Controllers
{
[ApiController]
[Route("api/[controller]")]
public class SlowProcessController : ControllerBase
{
private readonly ILogger<SlowProcessController> _logger;
private readonly IHubContext<ProgressHub> _hubController;
public SlowProcessController(
ILogger<SlowProcessController> logger,
IHubContext<ProgressHub> hubContext)
{
_logger = logger;
_hubController = hubContext;
}
[HttpGet("{ClientID}")]
public IEnumerable<int> Get(string ClientID)
{
List<int> retVal = new();
_logger.LogInformation("Incoming call from ClientID : {ClientID}", ClientID);
_hubController.Clients.Client(ClientID).SendAsync("ProgressReport", "Starting...");
Thread.Sleep(1000);
for (int loop = 0; loop < 10; loop++)
{
_hubController.Clients.Client(ClientID).SendAsync("ProgressReport", loop.ToString());
retVal.Add(loop);
Thread.Sleep(500);
}
_hubController.Clients.Client(ClientID).SendAsync("ProgressReport", "Done!");
return retVal;
}
}
}
I read the Steve Sandersen blog but this says not to use the code as its been superceded by inbuilt blazor functionality.
My application is only for a few users and so i'm not too worried about backend APIs etc, If the upload component used a service not a controller I could more easily get the progress, but the compoents all seem to post to controllers.
Can anyone please enlighten me as to the best way to solve this?

IFormFileCollection: Cannot access a Disposed Object

I am trying to upload a document into Azure Blob but it throws an error as below while reading the file stream.
Error,
Cannot access a disposed object.
Object name: 'FileBufferingReadStream'.
The error comes when the code OpenReadStream() called,
public async Task UploadAsync(IFormFileCollection files, string directoryName)
{
var blobContainer = await _azureBlobConnectionFactory.GetBlobContainer();
CloudBlobDirectory directory = blobContainer.GetDirectoryReference(directoryName);
for (int i = 0; i < files.Count; i++)
{
CloudBlockBlob blockblob = directory.GetBlockBlobReference(files[i].FileName);
using (var stream = files[i].OpenReadStream())
{
await blockblob.UploadFromStreamAsync(stream);
}
}
}
I am calling the UploadAsync method from my actual service classs like,
public async Task<bool> UploadToBlob(DocumentModel model,string directorypath)
{
try
{
//string directory = directorypath + model.EmailId + "/" + model.Files[0].FileName;
await _blobService.UploadAsync(model.Files, "Documents/dummy.pdf");
return true;
}
catch(Exception e)
{
throw e;
}
}
Where I did go wrong?
The return method of Action method can be a possible reason for this issue
The error occurs if the return type is not Task<T>.
You can modify your Action Method as follows:
[HttpPost]
public async Task<int> ActionMethod(IFormFile img)
{
// same
return resultValue;
}
References :
When trying to upload file, FileBufferingReadStream.ThrowIfDisposed() error occur
Cannot access a disposed object. Object name: FileBufferingReadStream
IFormFile copy to memorystream ObjectDisposedException

Azure Function: Blob Trigger throws System.OutOfMemoryException

I am getting System.OutOfMemoryException exception for Blob Trigger Azure Function.
When I am executing locally it is working fine.
Blog Trigger Azure Function:
public static class ProcessEvent
{
[FunctionName(nameof(ProcessEvent))]
public static async Task Run([BlobTrigger(BlobStorageContainer.Name + "/{name}",
Connection = "AzureWebJobsStorage")]
Stream eventBlob, string name,
[Inject] ILoggingService loggingService,
[Inject] IEventProcessorService eventProcessor,
[Inject] IBlobClient blobClient)
{
var logger = new Logger(loggingService);
try
{
logger.Info($"Starting blob job tracker for file name {name}",
nameof(ProcessEvent));
var eventContent = eventBlob.ReadAsString();
var result = await eventProcessor.HandleProcessor(eventContent, logger);
if (result)
{
await blobClient.DeleteBlobAsync(BlobStorageContainer.Name, name);
logger.Info($"Blob deleted successfully file name: {name}");
}
else
{
logger.Warning($"Unable to process blob job for file with name: {name}");
}
}
catch (Exception ex)
{
logger.Error($"Unable to process blob job for file with name: {name}", ex,
nameof(ProcessEvent));
}
}
}
My App Service Plan:
You can diagnose the memory usage in portal->your function app->Diagnose and solve problem->Memory Analysis->View Full Report.
It shows the overall percent physical memory usage per instance over the last 24 hours.
https://learn.microsoft.com/en-us/azure/app-service/overview-diagnostics#health-checkup-graphs

Upload image to Azure blob with summernote

public class StorageService
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=removed for this post");
public async Task Upload(string id, Stream data)
{
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve a reference to a container.
CloudBlobContainer container = blobClient.GetContainerReference("images");
await container.CreateIfNotExistsAsync();
container.SetPermissions(
new BlobContainerPermissions
{
PublicAccess = BlobContainerPublicAccessType.Blob
});
// Retrieve reference to a blob named "myblob".
CloudBlockBlob blockBlob = container.GetBlockBlobReference(id);
await blockBlob.UploadFromStreamAsync(data, data.Length);
}
public async Task UploadBlogPhoto(string id, Stream data)
{
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve a reference to a container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
await container.CreateIfNotExistsAsync();
container.SetPermissions(
new BlobContainerPermissions
{
PublicAccess = BlobContainerPublicAccessType.Blob
});
// Retrieve reference to a blob named "myblob".
CloudBlockBlob blockBlob = container.GetBlockBlobReference(id);
await blockBlob.UploadFromStreamAsync(data, data.Length);
}
}
So I have the StorageServices class and I use the first method, Upload, to upload users' profile pics.
here is the markup:
using (Html.BeginForm("UploadPhoto", "Manage", FormMethod.Post, new { enctype = "multipart/form-data" }))
{
<div class="browseimg">
<input type="file" class="display-none" name="file" id="files" onchange="this.form.submit()" />
</div>
}
<button class="btn btn-primary width-100p main-bg round-border-bot" id="falseFiles">
Upload billede
</button>
This is the summernote button for uploading a photo. The post uploads to a database. However, is there a way for the text to be uploaded to the database and the image to Azure blob? Would I need to do it asynchronously by first uploading the picture and adding the azure blob's URL into summernote, can it be done?
As Gaurav said, you could perform both operations in your upload functions. In my opinion, I recommend that you perform both operations in your upload action, in order to ensure data consistency. Here I provide you with a code sample to have a better understanding of it.
ManageController.cs
/// <summary>
/// Upload photo with description
/// </summary>
/// <param name="imageNote">description of the photo</param>
/// <returns></returns>
public async Task<JsonResult> UploadPhoto(string imageNote)
{
string operationResult = string.Empty;
string uploadedImageUrl = string.Empty;
uploadedImageUrl = await UploadImageToBlob();
//make sure the image is uploaded successfully to Azure Blob
if (!string.IsNullOrEmpty(uploadedImageUrl))
{
//insert the image(blob) Url and imageNote to your database
operationResult = "Operation is successful!";
}
else
operationResult = "Image upload failed, please check and submit again!";
return Json(new
{
Message = operationResult
});
}
/// <summary>
/// Upload photo to Azure Blob Storage
/// </summary>
/// <returns>the new Blob(photo) Url</returns>
private async Task<string> UploadImageToBlob()
{
string uploadedImageUrl = string.Empty;
try
{
var files = Request.Files;
if (files != null && files.Count > 0)
{
var file = files[0];
string blobName = Path.GetFileName(file.FileName);
#region upload image to Azure Blob and retrieve the Blob Url
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("brucechenStorage"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve a reference to a container.
CloudBlobContainer container = blobClient.GetContainerReference("images");
await container.CreateIfNotExistsAsync();
// Retrieve reference to a blob named "blobName".
CloudBlockBlob blockBlob = container.GetBlockBlobReference(blobName);
await blockBlob.UploadFromStreamAsync(file.InputStream);
uploadedImageUrl = blockBlob.Uri.ToString();
#endregion
}
}
catch (Exception e)
{
//TODO:log
}
return uploadedImageUrl;
}

Any Example of WebJob using EventHub?

I've tried to come up with something from the example in the WebJobsSDK gitHub
var eventHubConfig = new EventHubConfiguration();
string eventHubName = "MyHubName";
eventHubConfig.AddSender(eventHubName,"Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=SendRule;SharedAccessKey=xxxxxxxx");
eventHubConfig.AddReceiver(eventHubName, "Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=ReceiveRule;SharedAccessKey=yyyyyyy");
config.UseEventHub(eventHubConfig);
JobHost host = new JobHost(config);
But I'm afraid that's not far enough for someone of my limited "skillset"!
I can find no instance of JobHostConfiguration that has a UseEventHub property (using the v1.2.0-alpha-10291 version of the Microsoft.AzureWebJobs package), so I can't pass the EventHubConfiguration to the JobHost.
I've used EventHub before, not within the WebJob context. I don't see if the EventHostProcessor is still required if using the WebJob triggering...or does the WebJob trigger essentially act as the EventHostProcessor?
Anyway, if anyone has a more complete example for a simpleton like me that would be really sweet! Thanks
From the documentation here, you should have all the information you need.
What you are missing is a reference of the Microsoft.Azure.WebJobs.ServiceBus.1.2.0-alpha-10291 nuget package.
The UseEventHub is an extension method that is declared in this package.
Otherwise your configuration seems ok.
Here is an example on how to receive or send messages from/to an EventHub:
public class BasicTest
{
public class Payload
{
public int Counter { get; set; }
}
public static void SendEvents([EventHub("MyHubName")] out Payload x)
{
x = new Payload { Counter = 100 };
}
public static void Trigger(
[EventHubTrigger("MyHubName")] Payload x,
[EventHub("MyHubName")] out Payload y)
{
x.Counter++;
y = x;
}
}
EventProcessorHost is still required, as the WebJob just provides the hosting environment for running it. As far as I know, EventProcessorHost is not integrated so deeply into WebJob, so its triggering mechanism cannot be used for processing EventHub messages. I use WebJob for running EventProcessorHost continuously:
public static void Main()
{
RunAsync().Wait();
}
private static async Task RunAsync()
{
try
{
using (var shutdownWatcher = new WebJobsShutdownWatcher())
{
await Console.Out.WriteLineAsync("Initializing...");
var eventProcessorHostName = "eventProcessorHostName";
var eventHubName = ConfigurationManager.AppSettings["eventHubName"];
var consumerGroupName = ConfigurationManager.AppSettings["eventHubConsumerGroupName"];
var eventHubConnectionString = ConfigurationManager.ConnectionStrings["EventHub"].ConnectionString;
var storageConnectionString = ConfigurationManager.ConnectionStrings["EventHubStorage"].ConnectionString;
var eventProcessorHost = new EventProcessorHost(eventProcessorHostName, eventHubName, consumerGroupName, eventHubConnectionString, storageConnectionString);
await Console.Out.WriteLineAsync("Registering event processors...");
var processorOptions = new EventProcessorOptions();
processorOptions.ExceptionReceived += ProcessorOptions_ExceptionReceived;
await eventProcessorHost.RegisterEventProcessorAsync<CustomEventProcessor>(processorOptions);
await Console.Out.WriteLineAsync("Processing...");
await Task.Delay(Timeout.Infinite, shutdownWatcher.Token);
await Console.Out.WriteLineAsync("Unregistering event processors...");
await eventProcessorHost.UnregisterEventProcessorAsync();
await Console.Out.WriteLineAsync("Finished.");
}
catch (Exception ex)
{
await HandleErrorAsync(ex);
}
}
}
private static async void ProcessorOptions_ExceptionReceived(object sender, ExceptionReceivedEventArgs e)
{
await HandleErrorAsync(e.Exception);
}
private static async Task HandleErrorAsync(Exception ex)
{
await Console.Error.WriteLineAsync($"Critical error occured: {ex.Message}{ex.StackTrace}");
}

Resources