I have some blob access code written in C# that I need to move from using Microsoft.WindowsAzure.Storage.Blob to Azure.Storage.Blobs
In the old code, failover when reading blobs was handled with these lines of code:
using Microsoft.WindowsAzure.Storage;
Blob.BlobRequestOptions myBlobOptions = new Blob.BlobRequestOptions();
myBlobOptions.LocationMode = RetryPolicies.LocationMode.PrimaryThenSecondary;
myBlobClient.DefaultRequestOptions = myBlobOptions;
Blob.CloudBlobContainer myContainer = myBlobClient.GetContainerReference("container");
Blob.CloudBlockBlob blockBlob = myContainer.GetBlockBlobReference("blobfile");
What to do when using Azure.Storage.Blobs instead?
New Code:
BlobServiceClient BlobServiceClient = new BlobServiceClient("connectionString");
BlobContainerClient container = BlobServiceClient.GetBlobContainerClient("container");
BlobClient blob = container.GetBlobClient("blobfile");
I tried in my environment and successfully read files in the azure blob storage through a piece of code.
Code:
using Azure.Storage.Blobs;
namespace blobstr
{
class program {
public static async Task Main()
{
BlobServiceClient blobServiceClient = new BlobServiceClient("< Connection string >");
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("test");
BlobClient blobClient = containerClient.GetBlobClient("sample1.csv");
if (await blobClient.ExistsAsync())
{
var response = await blobClient.DownloadAsync();
using (var streamReader = new StreamReader(response.Value.Content))
{
while (!streamReader.EndOfStream)
{
var line = await streamReader.ReadLineAsync();
Console.WriteLine(line);
}
}
}
}
}
}
Portal:
In Blob storage, I have a csv file with view in below snap.
Console:
After I executed the code in my console, I can read the file from azure blob storage successfully.
Related
Our old ASP.Net application referenced Microsoft.Azure.Management.Storage.Fluent for the code below.
private static void DeleteBlobs(IAzure azure, string sourceContainer, string dbName)
{
var sAcc = GetStorageAccount(azure);
CloudBlobClient bClient = sAcc.CreateCloudBlobClient();
CloudBlobContainer srcCont = bClient.GetContainerReference(sourceContainer);
var srcDir = srcCont.GetDirectoryReference(dbName);
var blobs = srcDir.ListBlobs(useFlatBlobListing: true).ToList();
foreach (CloudBlockBlob blob in blobs)
{
CloudBlockBlob dBlob = srcCont.GetBlockBlobReference(blob.Name);
//Delete the source blob after copying
dBlob.Delete();
}
}
Our new WinUI 3 code, which uses the Azure.Management.Fluent package, is as follows. Similar, but the CloudBlobDirectory.ListBlobs does not exist, and we cannot seem to find an equivalent that will work for the foreach statement.
public static async void DeleteBlobsTest(string dbName)
{
var sAcc = GetStorageAccount(_StorageConnectionString);
CloudBlobClient bClient = sAcc.CreateCloudBlobClient();
CloudBlobContainer srcCont = bClient.GetContainerReference(_ActiveProjectsContainer);
var srcDir = srcCont.GetDirectoryReference(dbName);
var blobs = srcDir.ListBlobs(useFlatBlobListing: true).ToList(); //ListBlobs() does not exist
foreach (CloudBlockBlob blob in blobs)
{
CloudBlockBlob dBlob = srcCont.GetBlockBlobReference(blob.Name);
//Delete the source blob after copying
await dBlob.DeleteAsync();
}
}
We tried replacing var blobs = srcDir.ListBlobs(useFlatBlobListing: true).ToList(); with the two lines of code below, but it did not work, giving the error: Unable to cast object of type 'Microsoft.WindowsAzure.Storage.Blob.CloudBlobDirectory' to type 'Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob. Both our original and replacement return IEnumerable<IListBlobItem> interface enumeration, but different implementations, it seems. In any case, are there any workable replacements for ListBlob() in the WinUI 3 package?
BlobResultSegment blobSegment = await srcDir.ListBlobsSegmentedAsync(new BlobContinuationToken());
IEnumerable<IListBlobItem> blobs = blobSegment.Results;
Per Simon Mourier's comment. The best solution seems to be to use a different package and write the code accordingly. The package I had is now deprecated and doesn't seem to appear anymore in NuGit.
I added the package: Azure.Storage.Blobs
My code now looks like this, similar but not an exact match with the original:
public static void DeleteBlobs(string dbName)
{
BlobServiceClient bSvcCl = GetStorageAccountTest(_StorageConnectionString);
BlobContainerClient contCl = bSvcCl.GetBlobContainerClient(_ActiveProjectsContainer);
var blobs = contCl.GetBlobs();
foreach(BlobItem blob in blobs)
{
BlobClient sourceBlob = contCl.GetBlobClient(blob.Name);
sourceBlob.Delete();
}
}
solution/code how to upload large files more than 100 mb of size to blob storage in azure hosted app service (webapi) using .Net Core, But from local machine it's working not from azure app service.
Error showing file is too large to upload
Tried like one Example below -
[RequestFormLimits(MultipartBodyLengthLimit = 6104857600)]
[RequestSizeLimit(6104857600)]
public async Task<IActionResult> Upload(IFormfile filePosted)
{
string fileName = Path.GetFileName(filePosted.FileName);
string localFilePath = Path.Combine(fileName);
var fileStream = new FileStream(localFilePath, FileMode.Create);
MemoryStream ms = new MemoryStream();
filePosted.CopyTo(ms);
ms.WriteTo(fileStream);
BlobServiceClient blobServiceClient = new BlobServiceClient("ConnectionString");
var containerClient = new blobServiceClient.GetBlobContainerClient("Container");
BlobUploadOptions options = new BlobUploadOptions
{
TransferOptions = new StorageTransferOptions
{
MaximumConcurrency = 8,
MaximumTransferSize = 220 * 1024 * 1024
}
}
Blobsclient bc = containerClient.GetBlobClient("Name");
await bc.UploadAsync(fileStream, options);
ms.Dispose();
return Ok()
}
I tried in my environment and got below results:
To upload large files from local storage to Azure blob storage or file storage, you can use Azure data movement library,It provides high-performance for uploading, downloading larger files.
Code:
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Blob;
using Microsoft.Azure.Storage.DataMovement;
class program
{
public static void Main(string[] args)
{
string storageConnectionString = "<Connection string>";
CloudStorageAccount account = CloudStorageAccount.Parse(storageConnectionString);
CloudBlobClient blobClient = account.CreateCloudBlobClient();
CloudBlobContainer blobContainer = blobClient.GetContainerReference("test");
blobContainer.CreateIfNotExists();
string sourceBlob = #"C:\Users\download\sample.docx";
CloudBlockBlob destPath = blobContainer.GetBlockBlobReference("sample.docx");
TransferManager.Configurations.ParallelOperations = 64;
// Setup the transfer context and track the download progress
SingleTransferContext context = new SingleTransferContext
{
ProgressHandler = new Progress<TransferStatus>(progress =>
{
Console.WriteLine("Bytes Upload: {0}", progress.BytesTransferred);
})
};
// download thee blob
var task = TransferManager.UploadAsync(
sourceBlob, destPath, null, context, CancellationToken.None);
task.Wait();
}
}
Console:
Portal:
After I executed the above code and got successfully uploaded large file to azure blob storage.
I am trying to get all files in a directory of a blob container. I am able to get all files in that whole container. But I am not able to specify a directory.
Here is my Code
public async Task<IDictionary<string,DateTime>> GetBlobFiles(string directory="adf_exports")
{
IDictionary<string, DateTime> files = new Dictionary<string, DateTime>();
try
{
var accountName = _configuration["StorageAccount"];
var blobEndpoint = $"https://{accountName}.blob.core.windows.net";
var credential = new DefaultAzureCredential();
BlobServiceClient blobServiceClient = new BlobServiceClient(new Uri(blobEndpoint), credential);
var containerClient = blobServiceClient.GetBlobContainerClient(_configuration["BlobContainer"] + "/" + directory);
//var blobClient = containerClient.GetBlobClient(directory);
var list = containerClient.GetBlobs();
//var blobs = list.Where(b => Path.GetExtension(b.Name).Equals(".json"));
foreach (var item in list)
{
string name = item.Name;
BlockBlobClient blockBlob = containerClient.GetBlockBlobClient(name);
//using (var fileStream = File.OpenWrite(#"C:\Users\mbcrump\Downloads\test\" + name))
//{
// blockBlob.DownloadTo(fileStream);
//}
}
await foreach(BlobItem blob in containerClient.GetBlobsAsync())
{
files.Add(blob.Name, DateTime.Now);
}
return files;
}
catch (Exception ex)
{
return files;
}
}
It triggers the below error
The requested URI does not represent any resource on the server.
RequestId:b6449bde-d01e-003e-598b-a53f0f000000
Time:2021-09-09T15:02:09.1881302Z Status: 400 (The requested URI does
not represent any resource on the server.) ErrorCode: InvalidUri
But if we wont specify the directory and just the container like this
var containerClient = blobServiceClient.GetBlobContainerClient(_configuration["BlobContainer"]);
it works without any issues. But returns all folders and files in that container.
But How can I specify a folder alone.
FYI I am using Managed Identity to access blob. Connection strings or access keys are restricted in our environment.
You are receiving that error because there are no specified resources in the mentioned subscription.
Here is the code that worked for me
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System;
using System.IO;
using System.Threading.Tasks;
namespace ViewBlobList
{
class Program
{
static async Task Main(string[] args)
{
BlobServiceClient blobServiceClient = new BlobServiceClient("**Connection String**");
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("Container Name");
Console.WriteLine("Listing blobs..."+"\n");
// List all blobs in the container
var blobs = containerClient.GetBlobs();
foreach (BlobItem blobItem in blobs)
{
Console.WriteLine(blobItem.Name);
}
Console.Read();
}
}
}
here is the output
For more information, you can refer Quickstart: Azure Blob Storage library v12 - .NET | Microsoft Docs.
i'm trying to download file from azure blob storage, but it returns only part of file. What i'm doing wrong ? File in storage is not corrupted
public async Task<byte[]> GetFile(string fileName)
{
var blobClient = BlobContainerClient.GetBlobClient(fileName);
var downloadInfo = await blobClient.DownloadAsync();
byte[] b = new byte[downloadInfo.Value.ContentLength];
await downloadInfo.Value.Content.ReadAsync(b, 0, (int)downloadInfo.Value.ContentLength);
return b;
}
I'm using Azure.Storage.Blobs 12.4.2 package. I tried this code and it works for me
public async Task<byte[]> GetFile(string fileName)
{
var blobClient = BlobContainerClient.GetBlobClient(fileName);
using (var memorystream = new MemoryStream())
{
await blobClient.DownloadToAsync(memorystream);
return memorystream.ToArray();
}
}
I am not able to full understand your code as the current BlobClient as of v11.1.1 does not expose any download methods. As #Guarav Mantri-AIS mentioned the readAsync can behave in that manner.
Consider an alternative using the DownloadToByteArrayAsync() which is part of the API. I have include the code required to connect but of course this is just for the purpose of demonstrating a full example.
Your method would be condensed as follows:
public async Task<byte[]> GetFile(string containerName, string fileName)
{
//i am getting the container here, not sure where or how you are doing this
var container = GetContainer("//your connection string", containerName);
//Get the blob first
ICloudBlob blob = container.GetBlockBlobReference(fileName);
//and now download it straight to a byte array
return await blobClient.DownloadAsync();
}
public CloudBlobContainer GetContainer(string connectionString, string containerName)
{
//1. connect to the account
var account = CloudStorageAccount.Parse(connectionString);
//2. create a client
var blobClient = _account.CreateCloudBlobClient();
//3. i am getting the container here, not sure where or how you are doing this
return = _blobClient.GetContainerReference(containerName);
}
I am developing a social network application.
I want to creat ,in azure storage, a container for each user (client) joining a social network, which means user1 has a container named container 1, and inside container 1 there will be user 1 profile in xml format and a profile picture of user 1.
Similarly, for user2 there will be container 2 created in azure blob storage and then user 2 profile is saved in xml format and profile picture of user 2, and it goes like this so for, let say, 10 users there will be 10 containers.
If i want to list all 9 users' information stored in Azure storage in different 9 containers from user client 1, how could i do that?
I am using webservice, but the challenge I am facing is how to collect all the 9 user profile information located in 9 different containers.
The following should do the trick -
CloudStorageAccount account =
CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
// We need to access blobs now, so create a CloudBlobClient
CloudBlobClient blobClient = account.CreateCloudBlobClient();
IEnumerable<CloudBlobContainer> containers = blobClient.ListContainers();
For newer versions of Microsoft.WindowsAzure.Storage, use the right variant of CloudBlobClient.ListContainersSegmentedAsync method. Example usage is as follows:
private static async Task<IEnumerable<CloudBlobContainer>> ListContainersAsync(CloudBlobClient cloudBlobClient)
{
BlobContinuationToken continuationToken = null;
var containers = new List<CloudBlobContainer>();
do
{
ContainerResultSegment response = await cloudBlobClient.ListContainersSegmentedAsync(continuationToken);
continuationToken = response.ContinuationToken;
containers.AddRange(response.Results);
} while (continuationToken != null);
return containers;
}
Usage of the above method will be as follows:
string connectionString = "<your connection string>";
CloudStorageAccount.TryParse(connectionString, out CloudStorageAccount storageAccount);
if (storageAccount == null)
{
Console.WriteLine("Connection string did not work");
}
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
var containers = await ListContainersAsync(cloudBlobClient);
Alternately, you can create an extension method as follows:
public static class CloudBlobClientExtension
{
public static async Task<IEnumerable<CloudBlobContainer>> ListContainersAsync(this CloudBlobClient cloudBlobClient)
{
BlobContinuationToken continuationToken = null;
var containers = new List<CloudBlobContainer>();
do
{
ContainerResultSegment response = await cloudBlobClient.ListContainersSegmentedAsync(continuationToken);
continuationToken = response.ContinuationToken;
containers.AddRange(response.Results);
} while (continuationToken != null);
return containers;
}
}
Usage of the above extension method will be as follows:
string connectionString = "<your connection string>";
CloudStorageAccount.TryParse(connectionString, out CloudStorageAccount storageAccount);
if (storageAccount == null)
{
Console.WriteLine("Connection string did not work");
}
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
IEnumerable<CloudBlobContainer> containers = await cloudBlobClient.ListContainersAsync();
For more details, please refer to MSDN and Balkan's Blog
For older Azure Storage versions, use CloudBlobClient.ListContainers method. Example usage is as follows:
string connectionString = "<your connection string>";
CloudStorageAccount.TryParse(connectionString, out CloudStorageAccount storageAccount);
if (storageAccount == null)
{
Console.WriteLine("Connection string did not work");
}
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
var containers = cloudBlobClient.ListContainers();
Azure Blob Storage client library v12 for .NET
using Azure;
using Azure.Storage.Blobs;
BlobServiceClient blobServiceClient = new BlobServiceClient(azStorageConnString);
var containerList = blobServiceClient.GetBlobContainers();
foreach (var container in containerList)
// doImportantWork
CloudStorageAccount account =
CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
// We need to access blobs now, so create a CloudBlobClient
var blobClient = account.CreateCloudBlobClient();
IEnumerable<CloudBlobContainer> containers = blobClient.ListContainers();
// This will return you list of containers
var containerList = containers.Select(e => e.Name).Tolist();