I have created a private blob in a container on Azure.
Unfortunately it changes to public when I upload files. I have tried finding a way to set files as private when uploading, since that might be the problem, but I can't find anything.
Any ideas as to why this is?
Should private files be treated different when uploading?
My upload code:
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
if (file != null)
{
CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);
blockBlob.UploadFromStream(file.InputStream);
blockBlob.Properties.ContentEncoding = MimeTypes.GetContentType(filename);
if (ht != null)
{
foreach (DictionaryEntry item in ht)
{
blockBlob.Metadata[item.Key.ToString()] = item.Value.ToString();
}
blockBlob.SetMetadata();
}
blockBlob.Metadata["Created"] = DateTime.UtcNow.ToString();
blockBlob.SetProperties();
}
The code seems allright, please take a look here:
How to use Blobs in Windows Azure
Your blob will get the same access permissions as your container.
Some extra information: It seems you using StorageClient Library v1.7. This one is deprecated, v3.0 is the recommended version (StorageClient Library v3.0.0).
There is an issue in the StorageClient Library v.3.0.0 using containers on your local (test) machine though.
Related
My code below uploads an Excel file from a user's form submission. However, I'd like to work with the Excel file using EPplus to write it to the database. For that, I need the file's address on the disk, rather than the web address. How can I get that?
Relevant code:
{
public class ExcelService
{
public async Task<string> UploadExcelAsync(HttpPostedFileBase upload)
{
string excelFullPath = null;
if (upload == null || upload.ContentLength == 0)
{
return null;
}
try
{
CloudStorageAccount cloudStorageAccount = ConnectionString.GetConnectionString();
CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference("excel");
if (await cloudBlobContainer.CreateIfNotExistsAsync())
{
await cloudBlobContainer.SetPermissionsAsync(
new BlobContainerPermissions
{
PublicAccess = BlobContainerPublicAccessType.Blob
}
);
}
string excelName = Guid.NewGuid().ToString() + "-" + Path.GetExtension(upload.FileName);
CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(excelName);
cloudBlockBlob.Properties.ContentType = upload.ContentType;
await cloudBlockBlob.UploadFromStreamAsync(upload.InputStream);
excelFullPath = cloudBlockBlob.Uri.ToString();
}
catch (Exception ex)
{
}
return excelFullPath;
}
}
}
You can't. Azure Storage Blobs is a cloud service that only exposes web access to the stored blobs, so there is no physical location on a disk you can access. So unless you download the file, edit it and upload it again there is not way to edit it directly.
An alternative could be to use Azure Storage Files. Still an Azure Cloud based service for storage but it allows you to map folders to your machine so you can access it like any network share. See the docs. This, of course, only works if you are able to create a file share on the web server.
Unfortunately you cannot access Azure Storage Files shares using Azure Web Apps (source) so if that it how your web app is hosted than you are out of luck for that.
Here is my code:
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System;
using Microsoft.WindowsAzure;
using System.Net.Http;
namespace Test
{
class Program
{
static void Main(string[] args)
{
//get the storage account from the connection string
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=https;AccountName=[account name];AccountKey=[account key];EndpointSuffix=core.windows.net");
//instantiate the client
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
//set the container
CloudBlobContainer container = blobClient.GetContainerReference("images");
//get the blob reference
CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob.jpg");
//get image from stream and upload
using (var client = new HttpClient())
{
using (var stream = client.GetStreamAsync(some_url).GetAwaiter().GetResult())
{
if (stream != null)
{
blockBlob.UploadFromStreamAsync(stream);
}
}
client.Dispose();
}
}
}
}
The storage account instantiation works fine.
The container referencing works fine (it actually exists).
The block blob referencing works, as well, with no errors.
The stream has the image I am getting from the URL referenced.
Finally, the upload returns no errors.
Except, there is no image when I navigate to the Blob URI.
I get the following error:
The specified blob does not exist. RequestId:7df0aadc-0001-007c-6b90-f95158000000 Time:2017-07-10T15:21:25.2984015Z
I have also uploaded an image via the Azure Portal and that exists and can be navigated to through a browser.
Am I missing something?
Update below line in your code as you're calling async method.
blockBlob.UploadFromStreamAsync(stream).GetAwaiter().GetResult();
This should resolve your problem.
I am using the Azure object store in the following manner:
I have one container, and beneath it many blobs in a directory structure.
I am using Azure Blob Storage api to manage it.
Is there a way to delete an entire directory?
Do I really need to list all the blobs under it and then delete them one by one?
Is there a workaround like deleting all blobs with the same uri prefix (again, without listing them and then deleting them one by one)?
I don't know if there is a new solution, but we did that using https://msdn.microsoft.com/library/microsoft.windowsazure.storage.blob.cloudblobcontainer.listblobs.aspx - if we see what is going on with Fiddler, there are only prefix-ed blobs returned. Please see if that will work for you:
static void GetBlobsByPrefix(string Container, string Prefix)
{
if (!string.IsNullOrEmpty(Prefix))
{
var _Container = GetBlobContainer(Container);
var _Blobs = _Container.ListBlobs(Prefix, true);
foreach (IListBlobItem blob in _Blobs)
{
....
}
}
}
static CloudBlobContainer GetBlobContainer(string container)
{
CloudStorageAccount _StorageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("rus_AzureStorageConnectionString"));
CloudBlobClient _BlobClient = _StorageAccount.CreateCloudBlobClient();
CloudBlobContainer _Container = _BlobClient.GetContainerReference(container);
return _Container;
}
You can delete the container and all your blobs will be deleted. Containers on Azure Storage act as a "folder" for your blobs.
I have created container on azure storage and with code below made it work to upload my files(blobs) in container. Now it says upload was successful however I can't figure out how to reach those files to download it. There is no documentation on the internet about it.
Any help would be appreciated to solve it.
// create Azure Storage
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=https;AccountName=bireddy;AccountKey=8OMbjBeJR+SIYaSt0YtBUzivLKPX5ZbsGJeEY9vsX0BPbX3uy9KxOckK7LuLeH3ZbOh+NoEaiEIV/NWvZbFOrA==");
// create a blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// create a container
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// make it public
container.SetPermissions(
new BlobContainerPermissions {
PublicAccess = BlobContainerPublicAccessType.Container
});
// create a block blob
CloudBlockBlob blockBlob = container.GetBlockBlobReference(FileUpload1.FileName);
// upload to Azure Storage
// this has to be changed sometime later
blockBlob.Properties.ContentType = FileUpload1.PostedFile.ContentType;
blockBlob.UploadFromStream(FileUpload1.FileContent);
Grateful to your discussion.
There are a number of options to download your blobs:
Using PowerShell: See the Azure Storage Powershell guide here:
http://azure.microsoft.com/en-us/documentation/articles/storage-powershell-guide-full/
Using Command Line: See the documentation for the AzCopy tool here:
http://aka.ms/azcopy
Using your C# code: There is a guide here, that includes a section on how to download blobs:
http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-blobs/
Using GUI-based explorers: There are a number of third-party explorers, some of them are listed here:
http://blogs.msdn.com/b/windowsazurestorage/archive/2014/03/11/windows-azure-storage-explorers-2014.aspx
// Parse the connection string and return a reference to the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
//Create the Blob service client
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve a reference to a container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Retrieve reference to a blob named
CloudBlockBlob blockBlob2 = container.GetBlockBlobReference(document);
//using (var np = File.Open(#"C:\mydocuments\"+ document, FileMode.Create))
// blockBlob2.DownloadToStream(np, null, options, null);
byte[] fileBytes;
using (var fileStream = new MemoryStream())
{
blockBlob2.DownloadToStream(fileStream, null, options, null);
fileBytes = fileStream.ToArray();
}
return fileBytes;
I'm converting a website from a standard ASP.NET website over to use Azure. The website had previously taken an Excel file uploaded by an administrative user and saved it on the file system. As part of the migration, I'm saving this file to Azure Storage. It works fine when running against my local storage through the Azure SDK. (I'm using version 1.3 since I didn't want to upgrade during the development process.)
When I point the code to run against Azure Storage itself, though, the process usually fails. The error I get is:
System.IO.IOException occurred
Message=Unable to read data from the transport connection: The connection was closed.
Source=Microsoft.WindowsAzure.StorageClient
StackTrace:
at Microsoft.WindowsAzure.StorageClient.Tasks.Task`1.get_Result()
at Microsoft.WindowsAzure.StorageClient.Tasks.Task`1.ExecuteAndWait()
at Microsoft.WindowsAzure.StorageClient.CloudBlob.UploadFromStream(Stream source, BlobRequestOptions options)
at Framework.Common.AzureBlobInteraction.UploadToBlob(Stream stream, String BlobContainerName, String fileName, String contentType) in C:\Development\RateSolution2010\Framework.Common\AzureBlobInteraction.cs:line 95
InnerException:
The code is as follows:
public void UploadToBlob(Stream stream, string BlobContainerName, string fileName,
string contentType)
{
// Setup the connection to Windows Azure Storage
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(GetConnStr());
DiagnosticMonitorConfiguration dmc = DiagnosticMonitor.GetDefaultInitialConfiguration();
dmc.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
dmc.Logs.ScheduledTransferLogLevelFilter = LogLevel.Verbose;
DiagnosticMonitor.Start(storageAccount, dmc);
CloudBlobClient BlobClient = null;
CloudBlobContainer BlobContainer = null;
BlobClient = storageAccount.CreateCloudBlobClient();
// For large file copies you need to set up a custom timeout period
// and using parallel settings appears to spread the copy across multiple threads
// if you have big bandwidth you can increase the thread number below
// because Azure accepts blobs broken into blocks in any order of arrival.
BlobClient.Timeout = new System.TimeSpan(1, 0, 0);
Role serviceRole = RoleEnvironment.Roles.Where(s => s.Value.Name == "OnlineRates.Web").First().Value;
BlobClient.ParallelOperationThreadCount = serviceRole.Instances.Count;
// Get and create the container
BlobContainer = BlobClient.GetContainerReference(BlobContainerName);
BlobContainer.CreateIfNotExist();
//delete prior version if one exists
BlobRequestOptions options = new BlobRequestOptions();
options.DeleteSnapshotsOption = DeleteSnapshotsOption.None;
CloudBlob blobToDelete = BlobContainer.GetBlobReference(fileName);
Trace.WriteLine("Blob " + fileName + " deleted to be replaced by newer version.");
blobToDelete.DeleteIfExists(options);
//set stream to starting position
stream.Position = 0;
long totalBytes = 0;
//Open the stream and read it back.
using (stream)
{
// Create the Blob and upload the file
CloudBlockBlob blob = BlobContainer.GetBlockBlobReference(fileName);
try
{
BlobClient.ResponseReceived += new EventHandler<ResponseReceivedEventArgs>((obj, responseReceivedEventArgs)
=>
{
if (responseReceivedEventArgs.RequestUri.ToString().Contains("comp=block&blockid"))
{
totalBytes += Int64.Parse(responseReceivedEventArgs.RequestHeaders["Content-Length"]);
}
});
blob.UploadFromStream(stream);
// Set the metadata into the blob
blob.Metadata["FileName"] = fileName;
blob.SetMetadata();
// Set the properties
blob.Properties.ContentType = contentType;
blob.SetProperties();
}
catch (Exception exc)
{
Logging.ExceptionLogger.LogEx(exc);
}
}
}
I've tried a number of different alterations to the code: deleting a blob before replacing it (although the problem exists on new blobs as well), setting container permissions, not setting permissions, etc.
Your code looks like it should work, but it has lots of extra functionality that is not strictly required. I would cut it down to an absolute minimum and go from there. It's really only a gut feeling, but I think it might be the using statement giving you grief. This enture function could be written (presuming the container already exists) as:
public void UploadToBlob(Stream stream, string BlobContainerName, string fileName,
string contentType)
{
// Setup the connection to Windows Azure Storage
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(GetConnStr());
CloudBlobClient BlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer BlobContainer = BlobClient.GetContainerReference(BlobContainerName);
CloudBlockBlob blob = BlobContainer.GetBlockBlobReference(fileName);
stream.Position = 0;
blob.UploadFromStream(stream);
}
Notes on the stuff that I've removed:
You should set up diagnostics just once when you're app starts, not every time a method is called. Usually in the RoleEntryPoint.OnStart()
I'm not sure why you're trying to set ParallelOperationThreadCount higher if you have more instances. Those two things seem unrelated.
It's not good form to check for the existence of a container/table every time you save something to it. It's more usual to do that check once when your app starts or to have a process external to the website to make sure all the required containers/tables/queues exist. Of course if you're trying to dynamically create containers this is not true.
The problem turned out to be firewall settings on my laptop. It's my personal laptop originally set up at home and so the firewall rules weren't set up for a corporate environment resulting in slow performance on uploads and downloads.