Unable to download the files which were uploaded to the blob storage - azure

I have created container on azure storage and with code below made it work to upload my files(blobs) in container. Now it says upload was successful however I can't figure out how to reach those files to download it. There is no documentation on the internet about it.
Any help would be appreciated to solve it.
// create Azure Storage
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=https;AccountName=bireddy;AccountKey=8OMbjBeJR+SIYaSt0YtBUzivLKPX5ZbsGJeEY9vsX0BPbX3uy9KxOckK7LuLeH3ZbOh+NoEaiEIV/NWvZbFOrA==");
// create a blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// create a container
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// make it public
container.SetPermissions(
new BlobContainerPermissions {
PublicAccess = BlobContainerPublicAccessType.Container
});
// create a block blob
CloudBlockBlob blockBlob = container.GetBlockBlobReference(FileUpload1.FileName);
// upload to Azure Storage
// this has to be changed sometime later
blockBlob.Properties.ContentType = FileUpload1.PostedFile.ContentType;
blockBlob.UploadFromStream(FileUpload1.FileContent);
Grateful to your discussion.

There are a number of options to download your blobs:
Using PowerShell: See the Azure Storage Powershell guide here:
http://azure.microsoft.com/en-us/documentation/articles/storage-powershell-guide-full/
Using Command Line: See the documentation for the AzCopy tool here:
http://aka.ms/azcopy
Using your C# code: There is a guide here, that includes a section on how to download blobs:
http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-blobs/
Using GUI-based explorers: There are a number of third-party explorers, some of them are listed here:
http://blogs.msdn.com/b/windowsazurestorage/archive/2014/03/11/windows-azure-storage-explorers-2014.aspx

// Parse the connection string and return a reference to the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
//Create the Blob service client
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve a reference to a container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Retrieve reference to a blob named
CloudBlockBlob blockBlob2 = container.GetBlockBlobReference(document);
//using (var np = File.Open(#"C:\mydocuments\"+ document, FileMode.Create))
// blockBlob2.DownloadToStream(np, null, options, null);
byte[] fileBytes;
using (var fileStream = new MemoryStream())
{
blockBlob2.DownloadToStream(fileStream, null, options, null);
fileBytes = fileStream.ToArray();
}
return fileBytes;

Related

Write file to blob storage and save the SaS URL using C#

I am trying to create an Azure Function that create files in blob storage and then save a pre-signed blob file url that is generated dynamically in an azure table so that we can return blob file url to the client program to open.
I am able to create the files in blob storage and save the urls. Right now, the code makes the file urls public, I am not sure how can I make the current code generate SaS url instead of public url and save it to the azure table.
I didn't see any example that shows the usage of CloudBlobClient and SaS. Appreciate any help.
[FunctionName("CreateFiles")]
public static async void Run([QueueTrigger("JobQueue", Connection = "")]string myQueueItem,
[Table("SubJobTable", Connection = "AzureWebJobsStorage")] CloudTable subJobTable,
ILogger log)
{
Job job = JsonConvert.DeserializeObject<Job>(myQueueItem);
var storageAccount = CloudStorageAccount.Parse("UseDevelopmentStorage=true");
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
string containerName = $"{job.Name.ToLowerInvariant()}{Guid.NewGuid().ToString()}";
CloudBlobContainer cloudBlobContainer =
cloudBlobClient.GetContainerReference(containerName);
cloudBlobContainer.CreateIfNotExists();
BlobContainerPermissions permissions = new BlobContainerPermissions
{
PublicAccess = BlobContainerPublicAccessType.Blob
};
cloudBlobContainer.SetPermissions(permissions);
string localPath = "./data/";
string localFileName = $"{job.Id}.json";
string localFilePath = Path.Combine(localPath, localFileName);
File.WriteAllText(localFilePath, myQueueItem);
CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(localFileName);
log.LogInformation("Uploading to Blob storage as blob:\n\t {0}\n", cloudBlockBlob.Uri.AbsoluteUri);
cloudBlockBlob.UploadFromFile(localFilePath);
// update the table with file uri
DynamicTableEntity entity = new DynamicTableEntity(job.Id, job.PracticeId);
entity.Properties.Add("FileUri", new EntityProperty(cloudBlockBlob.Uri.AbsoluteUri));
entity.Properties.Add("Status", new EntityProperty("Complete"));
TableOperation mergeOperation = TableOperation.InsertOrMerge(entity);
subJobTable.Execute(mergeOperation);
}
It looks like that your code is making use of the older version of the SDK (Microsoft.Azure.Storage.Blob). If that's the case, then you would need to use GetSharedAccessSignature method in CloudBlob to generate a shared access signature token.
Your code would be something like:
...
cloudBlockBlob.UploadFromFile(localFilePath);
var sasToken = cloudBlockBlob. GetSharedAccessSignature(sas-token-parameters);
var sasUrl = "${cloudBlockBlob.Uri.AbsoluteUri}?${sasToken}";//Add question mark only if sas token does not have it.
...

Copy Azure block blob to Azure file share?

How do I copy a block (or page) blob in azure storage to an Azure file share?
My example code works fine if I download the block blob to a local file, but there does not appear to be a method to download to an Azure file share. I've looked at the Azure data movements library but there's no example of how to do this.
void Main()
{
string myfile =#"Image267.png";
CredentialEntity backupCredentials = Utils.GetBackupsCredentials();
CloudStorageAccount backupAccount = new CloudStorageAccount(new StorageCredentials(backupCredentials.Name, backupCredentials.Key), true);
CloudBlobClient backupClient = backupAccount.CreateCloudBlobClient();
CloudBlobContainer backupContainer = backupClient.GetContainerReference(#"archive");
CloudBlockBlob blob = backupContainer.GetBlockBlobReference(myfile);
CredentialEntity fileCredentails = Utils.GetFileCredentials();
CloudStorageAccount fileAccount = new CloudStorageAccount(new StorageCredentials(fileCredentails.Name,fileCredentails.Key), true);
CloudFileClient fileClient = fileAccount.CreateCloudFileClient();
CloudFileShare share = fileClient.GetShareReference(#"xfer");
if (share.Exists())
{
CloudFileDirectory rootDir = share.GetRootDirectoryReference();
CloudFileDirectory sampleDir = rootDir.GetDirectoryReference("hello");
if (sampleDir.Exists())
{
CloudFile file = sampleDir.GetFileReference(myfile);
// blob.DownloadToFile(file.ToString());
}
}
}
The part that does not work is the commented out line blob.DownloadToFile
Any ideas on how I can do this?
There is an example in the official docs here: it's used to copy files from file share to blob storage, but you can make a little change to copy from blob storage to file share. I also write a sample code which is used copy from blob storage to file share, you can take a look at it as below.
You can use SAS token(for the source blob or source file) to copy files to blob / or copy blob to files, in the same storage account or different storage account.
A sample code as below(copy blob to file in the same storage account, and you can make a little change if they are in different storage account):
static void Main(string[] args)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("connection string");
var blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = blobClient.GetContainerReference("t0201");
CloudBlockBlob sourceCloudBlockBlob = cloudBlobContainer.GetBlockBlobReference("test.txt");
//Note that if the file share is in a different storage account, you should use CloudStorageAccount storageAccount2 = CloudStorageAccount.Parse("the other storage connection string"), then use storageAccount2 for the file share.
CloudFileClient fileClient = storageAccount.CreateCloudFileClient();
CloudFileShare share = fileClient.GetShareReference("testfolder");
CloudFile destFile = share.GetRootDirectoryReference().GetFileReference("test.txt");
//Create a SAS for the source blob
string blobSas = sourceCloudBlockBlob.GetSharedAccessSignature(new SharedAccessBlobPolicy()
{
Permissions = SharedAccessBlobPermissions.Read,
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(24)
});
Uri blobSasUri = new Uri(sourceCloudBlockBlob.StorageUri.PrimaryUri.ToString()+blobSas);
destFile.StartCopy(blobSasUri);
Console.WriteLine("done now");
Console.ReadLine();
}
it works well at my side, hope it helps.
There are a few samples in here but in dot net. A non code related alternative would be to use Azcopy it would help transfer data from blob to File Shares and vice versa.
External Suggestions:
The following tool called Blobxfer seems to support the transfer but in python.

Azure access blob in directory

I'm trying to download a pdf blob that is in a directory in a container: folder1/test.pdf.
I've tried the following with no luck and searched for examples, but can't find anything.
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("testcontainer/folder1");
// Retrieve reference to a blob
//CloudBlockBlob blockBlob2 = container.GetBlockBlobReference("test.pdf");
Also tried this:
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("testcontainer");
// Retrieve reference to a blob
CloudBlockBlob blockBlob2 = container.GetBlockBlobReference("folder1/test.pdf");
thanks in advance!

How to copy an Azure File to an Azure Blob?

I've got some files sitting in an Azure File storage.
I'm trying to programatically archive them to Azure Blobs and I'm not sure how to do this efficiently.
I keep seeing code samples about copying from one blob container to another blob container .. but not from a File to a Blob.
Is it possible to do without downloading the entire File content locally and then uploading this content? Maybe use Uri's or something?
More Info:
The File and Blob containers are in the same Storage account.
The storage account is RA-GRS
Here was some sample code I was thinking of doing .. but it just doesn't feel right :( (pseudo code also .. with validation and checks omitted).
var file = await ShareRootDirectory.GetFileReference(fileName);
using (var stream = new MemoryStream())
{
await file.DownloadToStreamAsync(stream);
// Custom method that basically does:
// 1. GetBlockBlobReference
// 2. UploadFromStreamAsync
await cloudBlob.AddItemAsync("some-container", stream);
}
How to copy an Azure File to an Azure Blob?
We also can use CloudBlockBlob.StartCopy(CloudFile). CloudFile type is also can be accepted by the CloudBlockBlob.StartCopy function.
How to copy CloudFile to blob please refer to document. The following demo code is snippet from the document.
// Parse the connection string for the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
Microsoft.Azure.CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create a CloudFileClient object for credentialed access to File storage.
CloudFileClient fileClient = storageAccount.CreateCloudFileClient();
// Create a new file share, if it does not already exist.
CloudFileShare share = fileClient.GetShareReference("sample-share");
share.CreateIfNotExists();
// Create a new file in the root directory.
CloudFile sourceFile = share.GetRootDirectoryReference().GetFileReference("sample-file.txt");
sourceFile.UploadText("A sample file in the root directory.");
// Get a reference to the blob to which the file will be copied.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("sample-container");
container.CreateIfNotExists();
CloudBlockBlob destBlob = container.GetBlockBlobReference("sample-blob.txt");
// Create a SAS for the file that's valid for 24 hours.
// Note that when you are copying a file to a blob, or a blob to a file, you must use a SAS
// to authenticate access to the source object, even if you are copying within the same
// storage account.
string fileSas = sourceFile.GetSharedAccessSignature(new SharedAccessFilePolicy()
{
// Only read permissions are required for the source file.
Permissions = SharedAccessFilePermissions.Read,
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(24)
});
// Construct the URI to the source file, including the SAS token.
Uri fileSasUri = new Uri(sourceFile.StorageUri.PrimaryUri.ToString() + fileSas);
// Copy the file to the blob.
destBlob.StartCopy(fileSasUri);
Note:
If you are copying a blob to a file, or a file to a blob, you must use a shared access signature (SAS) to authenticate the source object, even if you are copying within the same storage account.
Use the Transfer Manager:
https://msdn.microsoft.com/en-us/library/azure/microsoft.windowsazure.storage.datamovement.transfermanager_methods.aspx
There are methods to copy from CloudFile to CloudBlob.
Add the "Microsoft.Azure.Storage.DataMovement" nuget package
using Microsoft.WindowsAzure.Storage.Blob;
using Microsoft.WindowsAzure.Storage.File;
using Microsoft.WindowsAzure.Storage.DataMovement;
private string _storageConnectionString = "your_connection_string_here";
public async Task CopyFileToBlob(string blobContainer, string blobPath, string fileShare, string fileName)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(_connectionString);
CloudFileShare cloudFileShare = storageAccount.CreateCloudFileClient().GetShareReference(fileShare);
CloudFile source = cloudFileShare.GetRootDirectoryReference().GetFileReference(fileName);
CloudBlobContainer blobContainer = storageAccount.CreateCloudBlobClient().GetContainerReference(blobContainer);
CloudBlockBlob target = blobContainer.GetBlockBlobReference(blobPath);
await TransferManager.CopyAsync(source, target, true);
}

Delete "subpath" from Azure Storage

I know Azure doesn't have actual subpaths, but if I have for example container/projectID/iterationNumber/filename.jpg and I delete a project, how can I delete from ProjectID? Is it possible through coding?
I don't want to use the azure application as I am creating a web app.
Thanks in Advance
EDIT:
This is the code provided by Microsoft to target on specific item:
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Retrieve reference to a blob named "myblob.txt".
CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob.txt");
// Delete the blob.
blockBlob.Delete();
SystemDesignModel
public static SystemDesign returnImageURL(IListBlobItem item)
{
if (item is CloudBlockBlob)
{
var blob = (CloudBlockBlob)item;
return new SystemDesign
{
URL = blob.Uri.ToString(),
};
}
return null;
}
}
As you know, blob storage does not have the concept of subfolders. It has just 2 level hierarchy - container & blobs. So in essence, a subfolder is just a prefix that you attach to blob name. In your example, the actual file you uploaded is filename.jpg but its name from blob storage perspective is projectID/iterationNumber/filename.jpg.
Since there is no concept of subfolder, you just can't delete it like we do on our local computer. However there's a way. Blob storage provides a way to search for blobs starting with a certain blob prefix. So what you have to do is first list all blobs that start with certain prefix (projectID in your case) and then delete the blobs one at a time returned as a result of listing operations.
Take a look at sample code below:
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
var container = storageAccount.CreateCloudBlobClient().GetContainerReference("container");
BlobContinuationToken token = null;
do
{
var listingResult = container.ListBlobsSegmented("blob-prefix (projectID in your case)", true, BlobListingDetails.None, 5000, token, null, null);
token = listingResult.ContinuationToken;
var blobs = listingResult.Results;
foreach (var blob in blobs)
{
(blob as ICloudBlob).DeleteIfExists();
Console.WriteLine(blob.Uri.AbsoluteUri + " deleted.");
}
}
while (token != null);

Resources