I have a file at a remote url such as http://www.site.com/docs/doc1.xls and i would like to copy that file onto my BLOB storage account.
I am aware and know of uploading files to a BLOB storage but wasn't sure how this can be done for a file from remote URL.
Try looking at CloudBlockBlob.StartCopyFromBlob that takes a URI if you are using the .NET client library.
string accountName = "accountname";
string accountKey = "key";
string newFileName = "newfile2.png";
string destinationContainer = "destinationcontainer";
string sourceUrl = "http://www.site.com/docs/doc1.xls";
CloudStorageAccount csa = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
CloudBlobClient blobClient = csa.CreateCloudBlobClient();
var blobContainer = blobClient.GetContainerReference(destinationContainer);
blobContainer.CreateIfNotExists();
var newBlockBlob = blobContainer.GetBlockBlobReference(newFileName);
newBlockBlob.StartCopyFromBlob(new Uri(sourceUrl), null, null, null);
Gaurav posted about this when it first came out. Handy, and his post shows how to watch for completion since the operation is Async.
Related
I am trying to create an Azure Function that create files in blob storage and then save a pre-signed blob file url that is generated dynamically in an azure table so that we can return blob file url to the client program to open.
I am able to create the files in blob storage and save the urls. Right now, the code makes the file urls public, I am not sure how can I make the current code generate SaS url instead of public url and save it to the azure table.
I didn't see any example that shows the usage of CloudBlobClient and SaS. Appreciate any help.
[FunctionName("CreateFiles")]
public static async void Run([QueueTrigger("JobQueue", Connection = "")]string myQueueItem,
[Table("SubJobTable", Connection = "AzureWebJobsStorage")] CloudTable subJobTable,
ILogger log)
{
Job job = JsonConvert.DeserializeObject<Job>(myQueueItem);
var storageAccount = CloudStorageAccount.Parse("UseDevelopmentStorage=true");
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
string containerName = $"{job.Name.ToLowerInvariant()}{Guid.NewGuid().ToString()}";
CloudBlobContainer cloudBlobContainer =
cloudBlobClient.GetContainerReference(containerName);
cloudBlobContainer.CreateIfNotExists();
BlobContainerPermissions permissions = new BlobContainerPermissions
{
PublicAccess = BlobContainerPublicAccessType.Blob
};
cloudBlobContainer.SetPermissions(permissions);
string localPath = "./data/";
string localFileName = $"{job.Id}.json";
string localFilePath = Path.Combine(localPath, localFileName);
File.WriteAllText(localFilePath, myQueueItem);
CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(localFileName);
log.LogInformation("Uploading to Blob storage as blob:\n\t {0}\n", cloudBlockBlob.Uri.AbsoluteUri);
cloudBlockBlob.UploadFromFile(localFilePath);
// update the table with file uri
DynamicTableEntity entity = new DynamicTableEntity(job.Id, job.PracticeId);
entity.Properties.Add("FileUri", new EntityProperty(cloudBlockBlob.Uri.AbsoluteUri));
entity.Properties.Add("Status", new EntityProperty("Complete"));
TableOperation mergeOperation = TableOperation.InsertOrMerge(entity);
subJobTable.Execute(mergeOperation);
}
It looks like that your code is making use of the older version of the SDK (Microsoft.Azure.Storage.Blob). If that's the case, then you would need to use GetSharedAccessSignature method in CloudBlob to generate a shared access signature token.
Your code would be something like:
...
cloudBlockBlob.UploadFromFile(localFilePath);
var sasToken = cloudBlockBlob. GetSharedAccessSignature(sas-token-parameters);
var sasUrl = "${cloudBlockBlob.Uri.AbsoluteUri}?${sasToken}";//Add question mark only if sas token does not have it.
...
I'm trying to upload images to Azure Blob Storage. I'm using .Net Core and Azure.Storage.Blobs v12.8.0.
The following code is what I have so far.
try
{
var documentByteArray = // a valid byte array of a png file
var blobUri = new Uri("https://mystorageaccount.blob.core.windows.net/images/myfile.png");
BlobClient blobClient = new BlobClient(blobUri);
using (MemoryStream stream = new MemoryStream(documentByteArray))
{
await blobClient.UploadAsync(stream, true, default);
await blobClient.SetHttpHeadersAsync(new BlobHttpHeaders
{
ContentType = "image/png"
});
}
}
catch (Exception ex)
{
//
}
...but somewhat predictably it fails with the exception Server failed to authenticate the request. Please refer to the information in the www-authenticate header.. I say predictable because I've not added any authentication...
And this is the problem/question. How do I add authentication so it will upload?
I know there are Access Keys that I can use - but how? I can't find any examples in MS documentation.
Any insight is appreciated.
If you have access to the Azure Portal, you can get the connection string of the storage account (under "Access Keys" section).
Once you have the connection string, you can use the following code:
var connectionString = "your-storage-account-connection-string";
var containerName = "images";
var blobName = "myfile.png";
var blobClient = new BlobClient(connectionString, containerName, blobName);
//do the upload here...
Other option is to use storage account name and access key (again you can get it from Azure Portal). You would do something like:
var accountName = "account-name";
var accountKey = "account-key";
var blobUri = new Uri("https://mystorageaccount.blob.core.windows.net/images/myfile.png");
var credentials = new StorageSharedKeyCredential(accountName, accountKey);
var blobClient = new BlobClient(blobUri, credentials);
//do the upload here...
You can find more information about BlobClient constructors here: https://learn.microsoft.com/en-us/dotnet/api/azure.storage.blobs.blobclient?view=azure-dotnet.
You should upload your blob through a CloudStorageAccount instance, like this:
var storageAccount = new CloudStorageAccount(
new StorageCredentials("<your account name>", "<your key>"),
"core.windows.net",
true);
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference(containerName);
var blob = container.GetBlockBlobReference(fileName);
await blob.UploadFromStreamAsync(stream);
I am trying to download a Blob from an Azure storage account container. When I run the application locally, I get the correct "Download" folder C:\Users\xxxx\Downloads. When I publish the application to Azure and try to download the file, I get an error. I have tried various "Knownfolders", and some return empty strings, others return the folders on the Azure server. I am able to upload files fine, list the files in a container, but am struggling with downloading a file.
string conn =
configuration.GetValue<string>"AppSettings:AzureContainerConn");
CloudStorageAccount storageAcct = CloudStorageAccount.Parse(conn);
CloudBlobClient blobClient = storageAcct.CreateCloudBlobClient();
CloudBlobContainer container =
blobClient.GetContainerReference(containerName);
Uri uriObj = new Uri(uri);
string filename = Path.GetFileName(uriObj.LocalPath);
// get block blob reference
CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);
Stream blobStream = await blockBlob.OpenReadAsync();
string _filepath = _knownfolder.Path + "\\projectfiles\\";
Directory.CreateDirectory(_filepath);
_filepath = _filepath + filename;
Stream _file = new MemoryStream();
try
{
_file = File.Open(_filepath, FileMode.Create, FileAccess.Write);
await blobStream.CopyToAsync(_file);
}
finally
{
_file.Dispose();
}
The expected end result is the file ends up in the folder within the users "Downloads" folder.
Since you're talking about publishing to Azure, the code is probably from a web application, right? And the code for the web application runs on the server. Which means the code is trying to download the blob to the server running the web application.
To present a downloadlink to the user to enable them to download the file, use the FileStreamResult which
Represents an ActionResult that when executed will write a file from a stream to the response.
A (pseudo code) example:
[HttpGet]
public FileStreamResult GetFile()
{
var stream = new MemoryStream();
CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);
blockBlob.DownloadToStream(stream);
blockBlob.Seek(0, SeekOrigin.Begin);
return new FileStreamResult(stream, new MediaTypeHeaderValue("text/plain"))
{
FileDownloadName = "someFile.txt"
};
}
I am trying to download a block blob from Azure storage explorer. I am able to download all the block blobs that exist in the root directory of my container. I am unable to download blobs that are nested in subfolders inside the container.
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessExpiryTime = DateTimeOffset.UtcNow.AddHours(1);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
string sasBlobToken = blob.GetSharedAccessSignature(sasConstraints);
return blob.Uri.AbsoluteUri + sasBlobToken;
I couldn't get the absolute path of blockBlob using GetBlockBlobReference(fileName). The below code solved my issue. I got the listing and then used LINQ to get the blockBlob with the absolute path details.
This post helped as well
do
{
var listingResult = await blobDirectory.ListBlobsSegmentedAsync(useFlatBlobListing, blobListingDetails, maxBlobsPerRequest, continuationToken, null, null);
//The below lined fetched the blockBlob with the correct directory details.
var blockBlob = listingResult.Results.Where(x => x.Uri.AbsolutePath.Contains(fileName)).Count()>0 ? (CloudBlockBlob)listingResult.Results.Where(x=>x.Uri.AbsolutePath.Contains(fileName)).FirstOrDefault():null;
if (blockBlob != null)
{
sasConstraints.SharedAccessExpiryTime = expiryTimeSAS;
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
string sasBlobToken = blockBlob.GetSharedAccessSignature(sasConstraints);
return blockBlob.Uri.AbsoluteUri + sasBlobToken;
}
continuationToken = listingResult.ContinuationToken;
} while (continuationToken != null);
Correct me if there is any other efficient way of pulling the blob information from a list of directories in a container.
Below Solution helps to access single File Absolute path residing under Directory( or Folder Path).
public static String GetBlobUri(string dirPath, string fileName)
{
//Get a reference to a blob within the container.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("Blob Key");
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("Blob Container");
CloudBlockBlob blockBlob = container.GetBlockBlobReference(dirPath+fileName);
return blockBlob.Uri.AbsoluteUri;
}
Hope this helps someone trying to access Blob File path based on multi level Directory(Level1/Level2/Level3) path.
Just use the ListBlobs mentioned in the answer from Gaurav Mantri to retrieve all files (blobs) within your desired subfolder. Then iterate over it and download it:
var storageAccount = CloudStorageAccount.Parse("yourConnectionString");
var client = storageAccount.CreateCloudBlobClient();
var container = client.GetContainerReference("yourContainer");
var blobs = container.ListBlobs(prefix: "subdirectory1/subdirectory2", useFlatBlobListing: true);
foreach (var blob in blobs)
{
blob.DownloadToFileAsync("yourFilePath");
}
I have created container on azure storage and with code below made it work to upload my files(blobs) in container. Now it says upload was successful however I can't figure out how to reach those files to download it. There is no documentation on the internet about it.
Any help would be appreciated to solve it.
// create Azure Storage
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=https;AccountName=bireddy;AccountKey=8OMbjBeJR+SIYaSt0YtBUzivLKPX5ZbsGJeEY9vsX0BPbX3uy9KxOckK7LuLeH3ZbOh+NoEaiEIV/NWvZbFOrA==");
// create a blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// create a container
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// make it public
container.SetPermissions(
new BlobContainerPermissions {
PublicAccess = BlobContainerPublicAccessType.Container
});
// create a block blob
CloudBlockBlob blockBlob = container.GetBlockBlobReference(FileUpload1.FileName);
// upload to Azure Storage
// this has to be changed sometime later
blockBlob.Properties.ContentType = FileUpload1.PostedFile.ContentType;
blockBlob.UploadFromStream(FileUpload1.FileContent);
Grateful to your discussion.
There are a number of options to download your blobs:
Using PowerShell: See the Azure Storage Powershell guide here:
http://azure.microsoft.com/en-us/documentation/articles/storage-powershell-guide-full/
Using Command Line: See the documentation for the AzCopy tool here:
http://aka.ms/azcopy
Using your C# code: There is a guide here, that includes a section on how to download blobs:
http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-blobs/
Using GUI-based explorers: There are a number of third-party explorers, some of them are listed here:
http://blogs.msdn.com/b/windowsazurestorage/archive/2014/03/11/windows-azure-storage-explorers-2014.aspx
// Parse the connection string and return a reference to the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
//Create the Blob service client
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve a reference to a container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Retrieve reference to a blob named
CloudBlockBlob blockBlob2 = container.GetBlockBlobReference(document);
//using (var np = File.Open(#"C:\mydocuments\"+ document, FileMode.Create))
// blockBlob2.DownloadToStream(np, null, options, null);
byte[] fileBytes;
using (var fileStream = new MemoryStream())
{
blockBlob2.DownloadToStream(fileStream, null, options, null);
fileBytes = fileStream.ToArray();
}
return fileBytes;