azure blob client returns file not fully - azure

i'm trying to download file from azure blob storage, but it returns only part of file. What i'm doing wrong ? File in storage is not corrupted
public async Task<byte[]> GetFile(string fileName)
{
var blobClient = BlobContainerClient.GetBlobClient(fileName);
var downloadInfo = await blobClient.DownloadAsync();
byte[] b = new byte[downloadInfo.Value.ContentLength];
await downloadInfo.Value.Content.ReadAsync(b, 0, (int)downloadInfo.Value.ContentLength);
return b;
}

I'm using Azure.Storage.Blobs 12.4.2 package. I tried this code and it works for me
public async Task<byte[]> GetFile(string fileName)
{
var blobClient = BlobContainerClient.GetBlobClient(fileName);
using (var memorystream = new MemoryStream())
{
await blobClient.DownloadToAsync(memorystream);
return memorystream.ToArray();
}
}

I am not able to full understand your code as the current BlobClient as of v11.1.1 does not expose any download methods. As #Guarav Mantri-AIS mentioned the readAsync can behave in that manner.
Consider an alternative using the DownloadToByteArrayAsync() which is part of the API. I have include the code required to connect but of course this is just for the purpose of demonstrating a full example.
Your method would be condensed as follows:
public async Task<byte[]> GetFile(string containerName, string fileName)
{
//i am getting the container here, not sure where or how you are doing this
var container = GetContainer("//your connection string", containerName);
//Get the blob first
ICloudBlob blob = container.GetBlockBlobReference(fileName);
//and now download it straight to a byte array
return await blobClient.DownloadAsync();
}
public CloudBlobContainer GetContainer(string connectionString, string containerName)
{
//1. connect to the account
var account = CloudStorageAccount.Parse(connectionString);
//2. create a client
var blobClient = _account.CreateCloudBlobClient();
//3. i am getting the container here, not sure where or how you are doing this
return = _blobClient.GetContainerReference(containerName);
}

Related

Searching for CloudBlobDirectory.ListBlobs Method Equivalent in WinUI 3 Using Azure.Management.Fluent Package

Our old ASP.Net application referenced Microsoft.Azure.Management.Storage.Fluent for the code below.
private static void DeleteBlobs(IAzure azure, string sourceContainer, string dbName)
{
var sAcc = GetStorageAccount(azure);
CloudBlobClient bClient = sAcc.CreateCloudBlobClient();
CloudBlobContainer srcCont = bClient.GetContainerReference(sourceContainer);
var srcDir = srcCont.GetDirectoryReference(dbName);
var blobs = srcDir.ListBlobs(useFlatBlobListing: true).ToList();
foreach (CloudBlockBlob blob in blobs)
{
CloudBlockBlob dBlob = srcCont.GetBlockBlobReference(blob.Name);
//Delete the source blob after copying
dBlob.Delete();
}
}
Our new WinUI 3 code, which uses the Azure.Management.Fluent package, is as follows. Similar, but the CloudBlobDirectory.ListBlobs does not exist, and we cannot seem to find an equivalent that will work for the foreach statement.
public static async void DeleteBlobsTest(string dbName)
{
var sAcc = GetStorageAccount(_StorageConnectionString);
CloudBlobClient bClient = sAcc.CreateCloudBlobClient();
CloudBlobContainer srcCont = bClient.GetContainerReference(_ActiveProjectsContainer);
var srcDir = srcCont.GetDirectoryReference(dbName);
var blobs = srcDir.ListBlobs(useFlatBlobListing: true).ToList(); //ListBlobs() does not exist
foreach (CloudBlockBlob blob in blobs)
{
CloudBlockBlob dBlob = srcCont.GetBlockBlobReference(blob.Name);
//Delete the source blob after copying
await dBlob.DeleteAsync();
}
}
We tried replacing var blobs = srcDir.ListBlobs(useFlatBlobListing: true).ToList(); with the two lines of code below, but it did not work, giving the error: Unable to cast object of type 'Microsoft.WindowsAzure.Storage.Blob.CloudBlobDirectory' to type 'Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob. Both our original and replacement return IEnumerable<IListBlobItem> interface enumeration, but different implementations, it seems. In any case, are there any workable replacements for ListBlob() in the WinUI 3 package?
BlobResultSegment blobSegment = await srcDir.ListBlobsSegmentedAsync(new BlobContinuationToken());
IEnumerable<IListBlobItem> blobs = blobSegment.Results;
Per Simon Mourier's comment. The best solution seems to be to use a different package and write the code accordingly. The package I had is now deprecated and doesn't seem to appear anymore in NuGit.
I added the package: Azure.Storage.Blobs
My code now looks like this, similar but not an exact match with the original:
public static void DeleteBlobs(string dbName)
{
BlobServiceClient bSvcCl = GetStorageAccountTest(_StorageConnectionString);
BlobContainerClient contCl = bSvcCl.GetBlobContainerClient(_ActiveProjectsContainer);
var blobs = contCl.GetBlobs();
foreach(BlobItem blob in blobs)
{
BlobClient sourceBlob = contCl.GetBlobClient(blob.Name);
sourceBlob.Delete();
}
}

How to upload large files to blob storage in hosted azure app service

solution/code how to upload large files more than 100 mb of size to blob storage in azure hosted app service (webapi) using .Net Core, But from local machine it's working not from azure app service.
Error showing file is too large to upload
Tried like one Example below -
[RequestFormLimits(MultipartBodyLengthLimit = 6104857600)]
[RequestSizeLimit(6104857600)]
public async Task<IActionResult> Upload(IFormfile filePosted)
{
string fileName = Path.GetFileName(filePosted.FileName);
string localFilePath = Path.Combine(fileName);
var fileStream = new FileStream(localFilePath, FileMode.Create);
MemoryStream ms = new MemoryStream();
filePosted.CopyTo(ms);
ms.WriteTo(fileStream);
BlobServiceClient blobServiceClient = new BlobServiceClient("ConnectionString");
var containerClient = new blobServiceClient.GetBlobContainerClient("Container");
BlobUploadOptions options = new BlobUploadOptions
{
TransferOptions = new StorageTransferOptions
{
MaximumConcurrency = 8,
MaximumTransferSize = 220 * 1024 * 1024
}
}
Blobsclient bc = containerClient.GetBlobClient("Name");
await bc.UploadAsync(fileStream, options);
ms.Dispose();
return Ok()
}
I tried in my environment and got below results:
To upload large files from local storage to Azure blob storage or file storage, you can use Azure data movement library,It provides high-performance for uploading, downloading larger files.
Code:
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Blob;
using Microsoft.Azure.Storage.DataMovement;
class program
{
public static void Main(string[] args)
{
string storageConnectionString = "<Connection string>";
CloudStorageAccount account = CloudStorageAccount.Parse(storageConnectionString);
CloudBlobClient blobClient = account.CreateCloudBlobClient();
CloudBlobContainer blobContainer = blobClient.GetContainerReference("test");
blobContainer.CreateIfNotExists();
string sourceBlob = #"C:\Users\download\sample.docx";
CloudBlockBlob destPath = blobContainer.GetBlockBlobReference("sample.docx");
TransferManager.Configurations.ParallelOperations = 64;
// Setup the transfer context and track the download progress
SingleTransferContext context = new SingleTransferContext
{
ProgressHandler = new Progress<TransferStatus>(progress =>
{
Console.WriteLine("Bytes Upload: {0}", progress.BytesTransferred);
})
};
// download thee blob
var task = TransferManager.UploadAsync(
sourceBlob, destPath, null, context, CancellationToken.None);
task.Wait();
}
}
Console:
Portal:
After I executed the above code and got successfully uploaded large file to azure blob storage.

Get list of all files in azure blob folder

I am trying to get all files in a directory of a blob container. I am able to get all files in that whole container. But I am not able to specify a directory.
Here is my Code
public async Task<IDictionary<string,DateTime>> GetBlobFiles(string directory="adf_exports")
{
IDictionary<string, DateTime> files = new Dictionary<string, DateTime>();
try
{
var accountName = _configuration["StorageAccount"];
var blobEndpoint = $"https://{accountName}.blob.core.windows.net";
var credential = new DefaultAzureCredential();
BlobServiceClient blobServiceClient = new BlobServiceClient(new Uri(blobEndpoint), credential);
var containerClient = blobServiceClient.GetBlobContainerClient(_configuration["BlobContainer"] + "/" + directory);
//var blobClient = containerClient.GetBlobClient(directory);
var list = containerClient.GetBlobs();
//var blobs = list.Where(b => Path.GetExtension(b.Name).Equals(".json"));
foreach (var item in list)
{
string name = item.Name;
BlockBlobClient blockBlob = containerClient.GetBlockBlobClient(name);
//using (var fileStream = File.OpenWrite(#"C:\Users\mbcrump\Downloads\test\" + name))
//{
// blockBlob.DownloadTo(fileStream);
//}
}
await foreach(BlobItem blob in containerClient.GetBlobsAsync())
{
files.Add(blob.Name, DateTime.Now);
}
return files;
}
catch (Exception ex)
{
return files;
}
}
It triggers the below error
The requested URI does not represent any resource on the server.
RequestId:b6449bde-d01e-003e-598b-a53f0f000000
Time:2021-09-09T15:02:09.1881302Z Status: 400 (The requested URI does
not represent any resource on the server.) ErrorCode: InvalidUri
But if we wont specify the directory and just the container like this
var containerClient = blobServiceClient.GetBlobContainerClient(_configuration["BlobContainer"]);
it works without any issues. But returns all folders and files in that container.
But How can I specify a folder alone.
FYI I am using Managed Identity to access blob. Connection strings or access keys are restricted in our environment.
You are receiving that error because there are no specified resources in the mentioned subscription.
Here is the code that worked for me
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System;
using System.IO;
using System.Threading.Tasks;
namespace ViewBlobList
{
class Program
{
static async Task Main(string[] args)
{
BlobServiceClient blobServiceClient = new BlobServiceClient("**Connection String**");
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("Container Name");
Console.WriteLine("Listing blobs..."+"\n");
// List all blobs in the container
var blobs = containerClient.GetBlobs();
foreach (BlobItem blobItem in blobs)
{
Console.WriteLine(blobItem.Name);
}
Console.Read();
}
}
}
here is the output
For more information, you can refer Quickstart: Azure Blob Storage library v12 - .NET | Microsoft Docs.

Uploading ID to same Azure Blob Index that a file is being uploaded to using .Net SDK

I am uploading documents to my an Azure Blob Storage, which works perfect, but I want to be able to link and ID to this specifically uploaded document.
Below is my code for uploading the file:
[HttpPost]
public ActionResult Upload(HttpPostedFileBase file)
{
try
{
var path = Path.Combine(Server.MapPath("~/App_Data/Uploads"), file.FileName);
string searchServiceName = ConfigurationManager.AppSettings["SearchServiceName"];
string blobStorageKey = ConfigurationManager.AppSettings["BlobStorageKey"];
string blobStorageName = ConfigurationManager.AppSettings["BlobStorageName"];
string blobStorageURL = ConfigurationManager.AppSettings["BlobStorageURL"];
file.SaveAs(path);
var credentials = new StorageCredentials(searchServiceName, blobStorageKey);
var client = new CloudBlobClient(new Uri(blobStorageURL), credentials);
// Retrieve a reference to a container. (You need to create one using the mangement portal, or call container.CreateIfNotExists())
var container = client.GetContainerReference(blobStorageName);
// Retrieve reference to a blob named "myfile.gif".
var blockBlob = container.GetBlockBlobReference(file.FileName);
// Create or overwrite the "myblob" blob with contents from a local file.
using (var fileStream = System.IO.File.OpenRead(path))
{
blockBlob.UploadFromStream(fileStream);
}
System.IO.File.Delete(path);
return new JsonResult
{
JsonRequestBehavior = JsonRequestBehavior.AllowGet,
Data = "Success"
};
}
catch (Exception ex)
{
throw;
}
}
I have added the ClientID field to the Index(It is at the bottom), but have no idea how I am able to add this to this index. This is still al nerw to me and just need a little guidance if someone can help :
Thanks in advance.

Azure block-blob storage uploading same file in succession fails

I' m using Azure Block Blob Storage to keep my files. Here is my code to upload file.
I m calling the method twice as below for the same file in the same request;
The first call of the method saves the file as expected but the second call saves file as length of 0 so i cant display the image and no error occurs.
[HttpPost]
public ActionResult Index(HttpPostedFileBase file)
{
UploadFile(file);
UploadFile(file);
return View();
}
public static string UploadFile(HttpPostedFileBase file){
var credentials = new StorageCredentials("accountName", "key");
var storageAccount = new CloudStorageAccount(credentials, true);
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("images");
container.CreateIfNotExists();
var containerPermissions = new BlobContainerPermissions
{
PublicAccess = BlobContainerPublicAccessType.Blob
};
container.SetPermissions(containerPermissions);
var blockBlob = container.GetBlockBlobReference(Guid.NewGuid().ToString());
blockBlob.Properties.ContentType = file.ContentType;
var azureFileUrl = string.Format("{0}", blockBlob.Uri.AbsoluteUri);
try
{
blockBlob.UploadFromStream(file.InputStream);
}
catch (StorageException ex)
{
throw;
}
return azureFileUrl ;
}
I just find the below solution which is strange like mine, but it does not help.
Strange Sudden Error "The number of bytes to be written is greater than the specified ContentLength"
Any idea?
Thanks
You need to reset the position of the stream back to the beginning. Put this line at the top of your UploadFile method.
file.InputStream.Seek(0, SeekOrigin.Begin);

Resources