My code below uploads an Excel file from a user's form submission. However, I'd like to work with the Excel file using EPplus to write it to the database. For that, I need the file's address on the disk, rather than the web address. How can I get that?
Relevant code:
{
public class ExcelService
{
public async Task<string> UploadExcelAsync(HttpPostedFileBase upload)
{
string excelFullPath = null;
if (upload == null || upload.ContentLength == 0)
{
return null;
}
try
{
CloudStorageAccount cloudStorageAccount = ConnectionString.GetConnectionString();
CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference("excel");
if (await cloudBlobContainer.CreateIfNotExistsAsync())
{
await cloudBlobContainer.SetPermissionsAsync(
new BlobContainerPermissions
{
PublicAccess = BlobContainerPublicAccessType.Blob
}
);
}
string excelName = Guid.NewGuid().ToString() + "-" + Path.GetExtension(upload.FileName);
CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(excelName);
cloudBlockBlob.Properties.ContentType = upload.ContentType;
await cloudBlockBlob.UploadFromStreamAsync(upload.InputStream);
excelFullPath = cloudBlockBlob.Uri.ToString();
}
catch (Exception ex)
{
}
return excelFullPath;
}
}
}
You can't. Azure Storage Blobs is a cloud service that only exposes web access to the stored blobs, so there is no physical location on a disk you can access. So unless you download the file, edit it and upload it again there is not way to edit it directly.
An alternative could be to use Azure Storage Files. Still an Azure Cloud based service for storage but it allows you to map folders to your machine so you can access it like any network share. See the docs. This, of course, only works if you are able to create a file share on the web server.
Unfortunately you cannot access Azure Storage Files shares using Azure Web Apps (source) so if that it how your web app is hosted than you are out of luck for that.
Related
I created a web API which allows users to send files and upload to Azure Storage. The way it works is, the client app will connect to API to send one or more files to the file upload controller and controller will take care of rest such as
Upload file to Azure storage
Update database
Works great but I don't think it is the right way to do this because now I can see there are two different processes
Upload file from the client's file system to my web API (server)
Upload file to the Azure storage from API (server)
It gives me the feeling that I am duplicating the upload process as the same file first travels to API (server) and then Azure (destination) from the client (file system). I feel the need of showing two progress-bars to the client for file upload progress (from client to server and then the server to Azure) - That just doesn't make sense to me and I feel that my approach is incorrect.
My API accepts up to 250MBs so you can imagine the overload.
What do you guys think?
//// API Controller
if (!Request.Content.IsMimeMultipartContent("form-data"))
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
var provider = new RestrictiveMultipartMemoryStreamProvider();
var contents = await Request.Content.ReadAsMultipartAsync(provider);
int Total_Files = contents.Contents.Count();
foreach (HttpContent ctnt in contents.Contents)
{
await storageManager.AddBlob(ctnt)
}
////// Stream
#region SteamHelper
public class RestrictiveMultipartMemoryStreamProvider : MultipartMemoryStreamProvider
{
public override Stream GetStream(HttpContent parent, HttpContentHeaders headers)
{
var extensions = new[] { "pdf", "doc", "docx", "cab", "zip" };
var filename = headers.ContentDisposition.FileName.Replace("\"", string.Empty);
if (filename.IndexOf('.') < 0)
return Stream.Null;
var extension = filename.Split('.').Last();
return extensions.Any(i => i.Equals(extension, StringComparison.InvariantCultureIgnoreCase))
? base.GetStream(parent, headers)
: Stream.Null;
}
}
#endregion SteamHelper
///// AddBlob
public async Task<string> AddBlob(HttpContent _Payload)
{
CloudStorageAccount cloudStorageAccount = KeyVault.AzureStorage.GetConnectionString();
CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference("SomeContainer");
cloudBlobContainer.CreateIfNotExists();
try
{
byte[] fileContentBytes = _Payload.ReadAsByteArrayAsync().Result;
CloudBlockBlob blob = cloudBlobContainer.GetBlockBlobReference("SomeBlob");
blob.Properties.ContentType = _Payload.Headers.ContentType.MediaType;
blob.UploadFromByteArray(fileContentBytes, 0, fileContentBytes.Length);
var B = await blob.CreateSnapshotAsync();
B.FetchAttributes();
return "Snapshot ETAG: " + B.Properties.ETag.Replace("\"", "");
}
catch (Exception X)
{
return ($"Error : " + X.Message);
}
}
It gives me the feeling that I am duplicating the upload process as
the same file first travels to API (server) and then Azure
(destination) from the client (file system).
I think you're correct. One possible solution would be have your API generate a Shared Access Signature (SAS) token and return that SAS token/URI to the client whenever a client wishes to upload a file.
Using this SAS URI your client can directly upload the file to Azure Storage without sending it to your API first. Once the file is uploaded successfully by the client, it can send a message to the API to update the database.
You can read more about SAS here: https://learn.microsoft.com/en-us/azure/storage/common/storage-dotnet-shared-access-signature-part-1.
I have also written a blog post long time back on using SAS that you may find useful: https://gauravmantri.com/2013/02/13/revisiting-windows-azure-shared-access-signature/.
I am using the Azure object store in the following manner:
I have one container, and beneath it many blobs in a directory structure.
I am using Azure Blob Storage api to manage it.
Is there a way to delete an entire directory?
Do I really need to list all the blobs under it and then delete them one by one?
Is there a workaround like deleting all blobs with the same uri prefix (again, without listing them and then deleting them one by one)?
I don't know if there is a new solution, but we did that using https://msdn.microsoft.com/library/microsoft.windowsazure.storage.blob.cloudblobcontainer.listblobs.aspx - if we see what is going on with Fiddler, there are only prefix-ed blobs returned. Please see if that will work for you:
static void GetBlobsByPrefix(string Container, string Prefix)
{
if (!string.IsNullOrEmpty(Prefix))
{
var _Container = GetBlobContainer(Container);
var _Blobs = _Container.ListBlobs(Prefix, true);
foreach (IListBlobItem blob in _Blobs)
{
....
}
}
}
static CloudBlobContainer GetBlobContainer(string container)
{
CloudStorageAccount _StorageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("rus_AzureStorageConnectionString"));
CloudBlobClient _BlobClient = _StorageAccount.CreateCloudBlobClient();
CloudBlobContainer _Container = _BlobClient.GetContainerReference(container);
return _Container;
}
You can delete the container and all your blobs will be deleted. Containers on Azure Storage act as a "folder" for your blobs.
I know Azure doesn't have actual subpaths, but if I have for example container/projectID/iterationNumber/filename.jpg and I delete a project, how can I delete from ProjectID? Is it possible through coding?
I don't want to use the azure application as I am creating a web app.
Thanks in Advance
EDIT:
This is the code provided by Microsoft to target on specific item:
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Retrieve reference to a blob named "myblob.txt".
CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob.txt");
// Delete the blob.
blockBlob.Delete();
SystemDesignModel
public static SystemDesign returnImageURL(IListBlobItem item)
{
if (item is CloudBlockBlob)
{
var blob = (CloudBlockBlob)item;
return new SystemDesign
{
URL = blob.Uri.ToString(),
};
}
return null;
}
}
As you know, blob storage does not have the concept of subfolders. It has just 2 level hierarchy - container & blobs. So in essence, a subfolder is just a prefix that you attach to blob name. In your example, the actual file you uploaded is filename.jpg but its name from blob storage perspective is projectID/iterationNumber/filename.jpg.
Since there is no concept of subfolder, you just can't delete it like we do on our local computer. However there's a way. Blob storage provides a way to search for blobs starting with a certain blob prefix. So what you have to do is first list all blobs that start with certain prefix (projectID in your case) and then delete the blobs one at a time returned as a result of listing operations.
Take a look at sample code below:
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
var container = storageAccount.CreateCloudBlobClient().GetContainerReference("container");
BlobContinuationToken token = null;
do
{
var listingResult = container.ListBlobsSegmented("blob-prefix (projectID in your case)", true, BlobListingDetails.None, 5000, token, null, null);
token = listingResult.ContinuationToken;
var blobs = listingResult.Results;
foreach (var blob in blobs)
{
(blob as ICloudBlob).DeleteIfExists();
Console.WriteLine(blob.Uri.AbsoluteUri + " deleted.");
}
}
while (token != null);
I have created a private blob in a container on Azure.
Unfortunately it changes to public when I upload files. I have tried finding a way to set files as private when uploading, since that might be the problem, but I can't find anything.
Any ideas as to why this is?
Should private files be treated different when uploading?
My upload code:
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
if (file != null)
{
CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);
blockBlob.UploadFromStream(file.InputStream);
blockBlob.Properties.ContentEncoding = MimeTypes.GetContentType(filename);
if (ht != null)
{
foreach (DictionaryEntry item in ht)
{
blockBlob.Metadata[item.Key.ToString()] = item.Value.ToString();
}
blockBlob.SetMetadata();
}
blockBlob.Metadata["Created"] = DateTime.UtcNow.ToString();
blockBlob.SetProperties();
}
The code seems allright, please take a look here:
How to use Blobs in Windows Azure
Your blob will get the same access permissions as your container.
Some extra information: It seems you using StorageClient Library v1.7. This one is deprecated, v3.0 is the recommended version (StorageClient Library v3.0.0).
There is an issue in the StorageClient Library v.3.0.0 using containers on your local (test) machine though.
I'm converting a website from a standard ASP.NET website over to use Azure. The website had previously taken an Excel file uploaded by an administrative user and saved it on the file system. As part of the migration, I'm saving this file to Azure Storage. It works fine when running against my local storage through the Azure SDK. (I'm using version 1.3 since I didn't want to upgrade during the development process.)
When I point the code to run against Azure Storage itself, though, the process usually fails. The error I get is:
System.IO.IOException occurred
Message=Unable to read data from the transport connection: The connection was closed.
Source=Microsoft.WindowsAzure.StorageClient
StackTrace:
at Microsoft.WindowsAzure.StorageClient.Tasks.Task`1.get_Result()
at Microsoft.WindowsAzure.StorageClient.Tasks.Task`1.ExecuteAndWait()
at Microsoft.WindowsAzure.StorageClient.CloudBlob.UploadFromStream(Stream source, BlobRequestOptions options)
at Framework.Common.AzureBlobInteraction.UploadToBlob(Stream stream, String BlobContainerName, String fileName, String contentType) in C:\Development\RateSolution2010\Framework.Common\AzureBlobInteraction.cs:line 95
InnerException:
The code is as follows:
public void UploadToBlob(Stream stream, string BlobContainerName, string fileName,
string contentType)
{
// Setup the connection to Windows Azure Storage
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(GetConnStr());
DiagnosticMonitorConfiguration dmc = DiagnosticMonitor.GetDefaultInitialConfiguration();
dmc.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
dmc.Logs.ScheduledTransferLogLevelFilter = LogLevel.Verbose;
DiagnosticMonitor.Start(storageAccount, dmc);
CloudBlobClient BlobClient = null;
CloudBlobContainer BlobContainer = null;
BlobClient = storageAccount.CreateCloudBlobClient();
// For large file copies you need to set up a custom timeout period
// and using parallel settings appears to spread the copy across multiple threads
// if you have big bandwidth you can increase the thread number below
// because Azure accepts blobs broken into blocks in any order of arrival.
BlobClient.Timeout = new System.TimeSpan(1, 0, 0);
Role serviceRole = RoleEnvironment.Roles.Where(s => s.Value.Name == "OnlineRates.Web").First().Value;
BlobClient.ParallelOperationThreadCount = serviceRole.Instances.Count;
// Get and create the container
BlobContainer = BlobClient.GetContainerReference(BlobContainerName);
BlobContainer.CreateIfNotExist();
//delete prior version if one exists
BlobRequestOptions options = new BlobRequestOptions();
options.DeleteSnapshotsOption = DeleteSnapshotsOption.None;
CloudBlob blobToDelete = BlobContainer.GetBlobReference(fileName);
Trace.WriteLine("Blob " + fileName + " deleted to be replaced by newer version.");
blobToDelete.DeleteIfExists(options);
//set stream to starting position
stream.Position = 0;
long totalBytes = 0;
//Open the stream and read it back.
using (stream)
{
// Create the Blob and upload the file
CloudBlockBlob blob = BlobContainer.GetBlockBlobReference(fileName);
try
{
BlobClient.ResponseReceived += new EventHandler<ResponseReceivedEventArgs>((obj, responseReceivedEventArgs)
=>
{
if (responseReceivedEventArgs.RequestUri.ToString().Contains("comp=block&blockid"))
{
totalBytes += Int64.Parse(responseReceivedEventArgs.RequestHeaders["Content-Length"]);
}
});
blob.UploadFromStream(stream);
// Set the metadata into the blob
blob.Metadata["FileName"] = fileName;
blob.SetMetadata();
// Set the properties
blob.Properties.ContentType = contentType;
blob.SetProperties();
}
catch (Exception exc)
{
Logging.ExceptionLogger.LogEx(exc);
}
}
}
I've tried a number of different alterations to the code: deleting a blob before replacing it (although the problem exists on new blobs as well), setting container permissions, not setting permissions, etc.
Your code looks like it should work, but it has lots of extra functionality that is not strictly required. I would cut it down to an absolute minimum and go from there. It's really only a gut feeling, but I think it might be the using statement giving you grief. This enture function could be written (presuming the container already exists) as:
public void UploadToBlob(Stream stream, string BlobContainerName, string fileName,
string contentType)
{
// Setup the connection to Windows Azure Storage
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(GetConnStr());
CloudBlobClient BlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer BlobContainer = BlobClient.GetContainerReference(BlobContainerName);
CloudBlockBlob blob = BlobContainer.GetBlockBlobReference(fileName);
stream.Position = 0;
blob.UploadFromStream(stream);
}
Notes on the stuff that I've removed:
You should set up diagnostics just once when you're app starts, not every time a method is called. Usually in the RoleEntryPoint.OnStart()
I'm not sure why you're trying to set ParallelOperationThreadCount higher if you have more instances. Those two things seem unrelated.
It's not good form to check for the existence of a container/table every time you save something to it. It's more usual to do that check once when your app starts or to have a process external to the website to make sure all the required containers/tables/queues exist. Of course if you're trying to dynamically create containers this is not true.
The problem turned out to be firewall settings on my laptop. It's my personal laptop originally set up at home and so the firewall rules weren't set up for a corporate environment resulting in slow performance on uploads and downloads.