reading content of blob from azure function - azure

I'm trying to read the content of a blob inside an azure function.
Here's the code:
Note:
If I comment out the using block and return the blob i.e.
return new OkObjectResult(blob);
I get back the blob object.
However, if I use the using block, I get 500.
Any idea why I can't get the content?
string storageConnectionString = "myConnectionString";
CloudStorageAccount storageAccount;
CloudStorageAccount.TryParse(storageConnectionString, out storageAccount);
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = cloudBlobClient.GetContainerReference("drawcontainer");
var blob = drawingsContainer.GetBlockBlobReference("notes.txt");
using (StreamReader reader = new StreamReader(blob.OpenRead()))
{
content = reader.ReadToEnd();
}
return new OkObjectResult(content);

HTTP 500 indicates that the code has error. The most probable reason for error is the variable 'content'. Define the variable 'content' outside the using block as the scope of the content variable defined inside it is limited to the block only. Declare it outside the using block, something like below:
try
{
string content = string.Empty;
using (StreamReader reader = new StreamReader(blob.OpenRead()))
{
content = reader.ReadToEnd();
}
}
catch (Exception ex)
{
// Log exception to get the details.
}
Always make use of try catch to get more details about errors in the code.

The OpenRead method didn't exist so I used the async one and it solved it.
I got to this solution after creating an azure function in VS and publishing it and it works.
Here's the code I used:
public static class Function1
{
[FunctionName("Function1")]
public static async Task<ActionResult> Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)]HttpRequest req, TraceWriter log)
{
log.Info("C# HTTP trigger function processed a request.");
string storageConnectionString = "DefaultEndpointsProtocol=https;AccountName=avitest19a1c;AccountKey=<AccessKey>";
CloudStorageAccount storageAccount = null;
CloudStorageAccount.TryParse(storageConnectionString, out storageAccount);
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer drawingsContainer = cloudBlobClient.GetContainerReference("drawcontainer");
var blob = drawingsContainer.GetBlockBlobReference("notes.txt");
string content = string.Empty;
**var contentStream = await blob.OpenReadAsync();**
using (StreamReader reader = new StreamReader(contentStream))
{
content = reader.ReadToEnd();
}
return new OkObjectResult(content);
}
}

Related

Azure Function Blob Storage connection: The format of value '*' is invalid

I am writing a v2 Azure Function in which I will access Azure Blob Storage. Because I was having trouble, I reduced it down to this minimal example.
namespace Test
{
public static class Function1
{
[FunctionName("Function1")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "test")] HttpRequest req,
ILogger log)
{
var azureStorage = CloudStorageAccount.Parse("UseDevelopmentStorage=true");
var blobClient = azureStorage.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("migrated-load-sets-localhost");
var blobReference = container.GetBlockBlobReference("11016093-2f6e-4631-97c1-04f8acfb2370");
var memoryStream = new MemoryStream();
var accessCondition = AccessCondition.GenerateIfExistsCondition();
var blobRequestOptions = new BlobRequestOptions();
await blobReference.DownloadToStreamAsync(memoryStream, accessCondition, blobRequestOptions, null);
var text = System.Text.Encoding.UTF8.GetString(memoryStream.ToArray());
return new OkObjectResult(text);
}
}
}
When I run and hit this, I get the error
System.Private.CoreLib: Exception while executing function: Function1. Microsoft.WindowsAzure.Storage: The format of value '*' is invalid. System.Net.Http: The format of value '*' is invalid.
If I change
var accessCondition = AccessCondition.GenerateIfExistsCondition();
to be
var accessCondition = AccessCondition.GenerateEmptyCondition();
it works.
I have observed in debugging that accessCondition.IfMatchETag equals "*", so it seems like that might be the culprit.
Am I doing something wrong when I use AccessCondition.GenerateIfExistsCondition(), or is there a bug in the library?
In case if you need to check if the blob is present before downloading the file , all you need is
if(blobReference.ExistsAsync())
{
//Download
}

How to upload a file to a storage location through URL using Azure function app

I want to upload the upload a file to a storage location through URL using Azure function app from Azure blob storage. I'm able to pull the file from Azure blob. But not able to upload the file through url.
Below I have attached the code which i have written. Could anyone help me on this?
#r "Newtonsoft.Json"
#r "Microsoft.WindowsAzure.Storage"
#r "System.IO"
using System;
using System.IO;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using Microsoft.WindowsAzure.Storage.Auth;
using System.Xml;
using System.Collections.Generic;
using Newtonsoft.Json;
using System.Net;
public static void Run(string input, TraceWriter log)
{
log.Info($"C# manual trigger function processed\n");
const string StorageAccountName = "";
const string StorageAccountKey = "";
var storageAccount = new CloudStorageAccount(new StorageCredentials(StorageAccountName, StorageAccountKey), true);
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("hannahtest");
var Destcontainer = blobClient.GetContainerReference("hannahtestoutput");
var blobs = container.ListBlobs();
log.Info($"Creating Client and Connecting");
foreach (IListBlobItem item in container.ListBlobs(null, false))
{
if (item is CloudBlockBlob blockBlob)
{
using (StreamReader reader = new StreamReader(blockBlob.OpenRead())
{
//old content string will read the blockblob (xml)till end
string oldContent1 = reader.ReadToEnd();
log.Info(oldContent1);
var content = new FormUrlEncodedContent(oldContent1);
var response = await client.PostAsync("http://www.example.com/recepticle.aspx", content);
var responseString = await response.Content.ReadAsStringAsync();
log.Info($"Success");
}
}
}
}
Have a look at Blob Output Binding - that's how the blobs are intended to be uploaded from Azure Functions, without messing with Azure Storage SDK.
Azure function to upload multiple image file to blob storage.
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
public static class ImageUploadFunction
{
[FunctionName("ImageUploadFunction")]
public static async Task<IActionResult> Run([HttpTrigger(AuthorizationLevel.Function, "post")]HttpRequestMessage req, ILogger log)
{
var provider = new MultipartMemoryStreamProvider();
await req.Content.ReadAsMultipartAsync(provider);
var files = provider.Contents;
List<string> uploadsurls = new List<string>();
foreach (var file in files) {
var fileInfo = file.Headers.ContentDisposition;
Guid guid = Guid.NewGuid();
string oldFileName = fileInfo.FileName;
string newFileName = guid.ToString();
var fileExtension = oldFileName.Split('.').Last().Replace("\"", "").Trim();
var fileData = await file.ReadAsByteArrayAsync();
try {
//Upload file to azure blob storage method
var upload = await UploadFileToStorage(fileData, newFileName + "." + fileExtension);
uploadsurls.Add(upload);
}
catch (Exception ex) {
log.LogError(ex.Message);
return new BadRequestObjectResult("Somthing went wrong.");
}
}
return uploadsurls != null
? (ActionResult)new OkObjectResult(uploadsurls)
: new BadRequestObjectResult("Somthing went wrong.");
}
private static async Task<string> UploadFileToStorage(byte[] fileStream, string fileName)
{
// Create storagecredentials object by reading the values from the configuration (appsettings.json)
StorageCredentials storageCredentials = new StorageCredentials("<AccountName>", "<KeyValue>");
// Create cloudstorage account by passing the storagecredentials
CloudStorageAccount storageAccount = new CloudStorageAccount(storageCredentials, true);
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Get reference to the blob container by passing the name by reading the value from the configuration (appsettings.json)
CloudBlobContainer container = blobClient.GetContainerReference("digital-material-library-images");
// Get the reference to the block blob from the container
CloudBlockBlob blockBlob = container.GetBlockBlobReference(fileName);
// Upload the file
await blockBlob.UploadFromByteArrayAsync(fileStream,0, fileStream.Length);
blockBlob.Properties.ContentType = "image/jpg";
await blockBlob.SetPropertiesAsync();
return blockBlob.Uri.ToString();
//return await Task.FromResult(true);
}
}

How do I upload multiple files to Azure blob storage using an ID parameter as a container?

I've created a storage account on Azure for uploading files to. I am using KenoUI Async upload widget which allows multiple file uploads. What I'd like to do is have the system create a container based on the passing in Id parameter if it doesn't already exist.
I'm struggling with the actual upload part however, I'm not sure how to pass the enumerable files to the storage account. Here is my code so far:
public ActionResult Async_Save(IEnumerable<HttpPostedFileBase> files, string id)
{
//Connect to Azure
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("my_AzureStorageConnectionString"));
//Create Blob Client
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
//Retrieve a reference to a container
CloudBlobContainer blobContainer = blobClient.GetContainerReference("vehicle_" + id);
try
{
// Create the container if it doesn't already exist
blobContainer.CreateIfNotExists();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
Console.WriteLine(ex.InnerException);
}
foreach (var file in files)
{
//This doesn't seem right to me and it's where I'm struggling
var fileName = Path.GetFileName(file.FileName);
var physicalPath = Path.Combine(blobContainer, fileName);
file.SaveAs(physicalPath);
}
// Return an empty string to signify success
return Content("");
}
What I've attempted to do is create a method that connects to my Azure storage account and:
Check for the existence of a container with the same ID as the
the parameter that's passed in, if it doesn't exist, create it.
Upload the files to the existing or newly created directory and give them a prefix of the ID i.e. "_".
As Gaurav Mantri said above, we will need to use Azure Storage SDK.
For better performance, we can do as below:
Parallel.ForEach(files, file =>
{
CloudBlockBlob blob = blobContainer.GetBlockBlobReference(file.FileName);
blob.UploadFromStream(file.InputStream);
file.InputStream.Close();
});
Your code will be better as below:
public ActionResult Async_Save(List<HttpPostedFileBase> files, string id)
{
// Connect to Azure
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("my_AzureStorageConnectionString"));
try
{
//Create Blob Client
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
//Retrieve a reference to a container
CloudBlobContainer blobContainer = blobClient.GetContainerReference("vehicle_" + id);
// Create the container if it doesn't already exist
blobContainer.CreateIfNotExists();
Parallel.ForEach(files, file =>
{
CloudBlockBlob blob = blobContainer.GetBlockBlobReference(file.FileName);
blob.UploadFromStream(file.InputStream);
file.InputStream.Close();
});
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
Console.WriteLine(ex.InnerException);
}
// Return an empty string to signify success
return new ContentResult();
}
More information about how to use Azure Storage SDK, we can refer to:
Get Quick Start with Storage SDK
To upload a blob in Azure Storage, you will need to use Azure Storage SDK.
Please replace the following code:
foreach (var file in files)
{
//This doesn't seem right to me and it's where I'm struggling
var fileName = Path.GetFileName(file.FileName);
var physicalPath = Path.Combine(blobContainer, fileName);
file.SaveAs(physicalPath);
}
With something like:
foreach (var file in files)
{
var blob = blobContainer.GetBlockBlobReference(file.FileName);
using (var s = file.InputStream)
{
blob.UploadFromStream(s);
}
}

Request.Content.ReadAsMultipartAsync Throws System.IO exception

I am uploading an image to an ASP WebAPI service and then uploading it to windows azure. everything was working great but suddenly I started getting the following exception:
System.IO.IOException: The process cannot access the file
'C:\DWASFiles\Sites\Tasteat\Temp\BodyPart_a5c79910-6077-4c24-b814-10fdc0e0b3d4'
because it is being used by another process.
This is the code that throws the exception:
var provider = new BlobStorageProvider(container);
Trace.TraceInformation("Uploading raw image to blob");
await Request.Content.ReadAsMultipartAsync(provider);
Trace.TraceInformation("Uploading finished");
I know its this line await Request.Content.ReadAsMultipartAsync(provider); because I see the line before it in the log but not the line after it.
Any ideas?
Everything was working great up until a few days
So as it seems the code I posted above actually saves a local file and only then uploaded it to the server, which causes the error but also is slow.
after a lot of trying I finally change to the following solution and everything started working and it was even faster!
first create a streamprovider:
public class BlobStorageMultipartStreamProvider : MultipartStreamProvider
{
private readonly string _containerName;
private readonly string _fileName;
public BlobStorageMultipartStreamProvider(string containerName, string fileName)
{
_containerName = containerName;
_fileName = fileName;
}
public override Stream GetStream(HttpContent parent, HttpContentHeaders headers)
{
Stream stream = null;
if (!String.IsNullOrWhiteSpace(_fileName))
{
string connectionString = ConfigurationManager.ConnectionStrings["BlobStorage"].ConnectionString;
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer blobContainer = blobClient.GetContainerReference(_containerName);
CloudBlockBlob blob = blobContainer.GetBlockBlobReference(_fileName);
stream = blob.OpenWrite();
}
return stream;
}
}
The upload code:
string fileName = Guid.NewGuid()+".Png";
MultipartStreamProvider provider = new BlobStorageMultipartStreamProvider("container",fileName);
Trace.TraceInformation("Uploading raw image to blob");
await Request.Content.ReadAsMultipartAsync(provider);
Trace.TraceInformation("Uploading finished");

azure CloudBlobDirectory.ListBlobs() returns "The specified resource does not exist.", but fetchAttributes() works using same data

I am getting a "The specified resource does not exist" exception when I try to iterate the result of a ListBlobs() call. I can get the blob attributes when I access it directly, but I'm trying to get a list of all the blobs in the subdirectory.
I wrote this little test to see exactly where the problem is. I have a test driver and two methods here. The first method, "GetBlockBlobDateTime" runs fine and returns a date time of an existing blob. The second method "GetBlobDirFiles" uses the same inputs and throws the excpetion when I try to iterate the blobItems at.
foreach (IListBlobItem blobItem in blobItems)
Note that the same data is used for both methods. What am I missing?
public static void DoTest(string baseURL, string container, string directory, string fileName)
{
DateTime t = GetBlockBlobDateTime( baseURL, container, directory, fileName);
List<string> fileList = GetBlobDirFiles( baseURL, container, directory);
}
public static DateTime GetBlockBlobDateTime(string baseURL, string container, string directory, string fileName)
{
CloudBlobClient blobClient = new CloudBlobClient(baseURL);
CloudBlobDirectory blobDir = blobClient.GetBlobDirectoryReference(container);
CloudBlobDirectory subDirectory = blobDir.GetSubdirectory(directory);
CloudBlockBlob cloudBlockBlob = subDirectory.GetBlockBlobReference(fileName);
cloudBlockBlob.FetchAttributes();
DateTime cloudTimeStampUTC = cloudBlockBlob.Properties.LastModifiedUtc;
return cloudTimeStampUTC;
}
public static List<string> GetBlobDirFiles(string baseURL, string container, string directory)
{
CloudBlobClient blobClient = new CloudBlobClient(baseURL);
CloudBlobDirectory blobDir = blobClient.GetBlobDirectoryReference(container);
CloudBlobDirectory subDirectory = blobDir.GetSubdirectory(directory);
IEnumerable<IListBlobItem> blobItems = subDirectory.ListBlobs();
List<string> fileList = new List<string>();
foreach (IListBlobItem blobItem in blobItems)
{
fileList.Add(blobItem.Uri.ToString());
}
return fileList;
}
OK, I figured it out:
Apparently, you don't need permissions to get file attributes, but you do to list files in the directory.
CloudBlobClient blobClient = new CloudBlobClient(baseURL);
works when you are going to fetch attributes like this:
cloudBlockBlob.FetchAttributes();
But you need to provide credentials like this:
CloudBlobClient blobClient =
new CloudBlobClient(baseURL,
new StorageCredentialsAccountAndKey(myAccount, myKey));
when you are going to list the blobs like this:
var blobList = subDirectory.ListBlobs();
foreach (var blobInfo in blobList)

Resources