I'm trying to implement a piece of code in Java using Azure Java SDK that allows me to upload a file (jpg, in mycase) to a Azure container. I have written this, but it fails:
String ENDPOINT= "https://myaccount.blob.core.windows.net";
BlobContainerClient blobContainerClient = new BlobContainerClientBuilder().
endpoint(ENDPOINT).
credential(CREDENTIAL).
containerName(CONTAINER_NAME).
buildClient();
// Get a reference to a local image
BlobClient blobClient = blobContainerClient.getBlobClient(myJpgLocalPath + myJpgFile);
// Set headers
blobClient.setHttpHeaders(new BlobHttpHeaders().setContentType("image/jpeg"));
// Upload the blob
blobClient.uploadFromFile(myJpgLocalPath + myJpgFile, false);
Another approach I've done is this:
BlobServiceClient storageClient = new BlobServiceClientBuilder().
endpoint(ENDPOINT).
credential(CREDENTIAL).
buildClient();
BlobContainerClient blobContainerClient = storageClient.getBlobContainerClient(CONTAINER_NAME);
// Get a reference to a local image
BlobClient blobClient = blobContainerClient.getBlobClient(myJpgLocalPath + myJpgFile);
// Set headers
blobClient.setHttpHeaders(new BlobHttpHeaders().setContentType("image/jpeg"));
// Upload the blob
blobClient.uploadFromFile(myJpgLocalPath + myJpgFile, false);
None of these two works at all and fails even building the client.
The errors shown are these two:
java.lang.ClassNotFoundException: com.fasterxml.jackson.dataformat.xml.PackageVersion
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1412)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1220)
at com.azure.core.implementation.jackson.JacksonVersion.<init>(JacksonVersion.java:46)
at com.azure.core.implementation.jackson.JacksonVersion.getInstance(JacksonVersion.java:72)
at com.azure.core.implementation.jackson.ObjectMapperShim.<clinit>(ObjectMapperShim.java:40)
at com.azure.core.util.serializer.JacksonAdapter.<init>(JacksonAdapter.java:81)
at com.azure.core.util.serializer.JacksonAdapter.<init>(JacksonAdapter.java:59)
at com.azure.core.util.serializer.JacksonAdapter$SerializerAdapterHolder.<clinit>(JacksonAdapter.java:114)
at com.azure.core.util.serializer.JacksonAdapter.createDefaultSerializerAdapter(JacksonAdapter.java:123)
at com.azure.storage.blob.implementation.util.ModelHelper.<clinit>(ModelHelper.java:62)
at com.azure.storage.blob.BlobUrlParts.parse(BlobUrlParts.java:371)
at com.azure.storage.blob.BlobContainerClientBuilder.endpoint(BlobContainerClientBuilder.java:181)
...
java.lang.NoClassDefFoundError: Could not initialize class com.azure.storage.blob.implementation.util.ModelHelper
at com.azure.storage.blob.BlobUrlParts.parse(BlobUrlParts.java:371)
at com.azure.storage.blob.BlobContainerClientBuilder.endpoint(BlobContainerClientBuilder.java:181)
...
What I am doing bad?
Thanks!
Related
I'm trying to follow the example to upload a file to Azure Data Storage as mentioned in the documentation : https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-dotnet?tabs=visual-studio%2Cmanaged-identity%2Croles-azure-portal%2Csign-in-azure-cli%2Cidentity-visual-studio
Following is my code:
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System;
using System.IO;
using Azure.Identity;
// TODO: Replace <storage-account-name> with your actual storage account name
var blobServiceClient = new BlobServiceClient(
new Uri("https://[some azure storage]"),
new DefaultAzureCredential());
// Set container name
string containerName = "data";
// Get container
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
// Create a local file in the ./data/ directory for uploading and downloading
string localPath = "data";
Directory.CreateDirectory(localPath);
string fileName = "testupload" + Guid.NewGuid().ToString() + ".txt";
string localFilePath = Path.Combine(localPath, fileName);
// Write text to the file
await File.WriteAllTextAsync(localFilePath, "Hello, World!");
// Get a reference to a blob
BlobClient blobClient = containerClient.GetBlobClient(fileName);
Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);
// Upload data from the local file
await blobClient.UploadAsync(localFilePath, true);
But I'm getting an error message that the request is not authorized.
Error message:
Azure.RequestFailedException: 'This request is not authorized to perform this operation using this permission.
I have Contributor role (which based on description is Grant full access to manage all resources ....), is this role still not enough to perform the operation?
I tried in my environment and got below results:
Initially I tried same code in my environment and got same error
Console:
Azure.RequestFailedException: 'This request is not authorized to perform this operation using this permission.
The above error occurs when your principal doesn't has access to azure blob storage.
For accessing blob storage through identity, Gaurav Mantri comment it is correct, you need a role to access blob storage,
The roles are
Storage-blob-contributor(or)
Storage-blob-owner
Go to portal -> storage accounts -> Access Control (IAM) ->Add -> Add role assignments -> storage-blob-contributor or storage-blob-owner role to the storage account.
Portal:
After assigning role to my storage account, I executed same code and it successfully uploaded file in azure blob storage.
Code:
using Azure.Storage.Blobs;
using System;
using System.IO;
using Azure.Identity;
// TODO: Replace <storage-account-name> with your actual storage account name
var blobServiceClient = new BlobServiceClient(
new Uri("https://[some azure storage]"),
new DefaultAzureCredential());
// Set container name
string containerName = "test";
// Get container
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
// Create a local file in the ./data/ directory for uploading and downloading
string localPath = "data";
Directory.CreateDirectory(localPath);
string fileName = "testupload" + Guid.NewGuid().ToString() + ".txt";
string localFilePath = Path.Combine(localPath, fileName);
// Write text to the file
await File.WriteAllTextAsync(localFilePath, "Hello, World!");
// Get a reference to a blob
BlobClient blobClient = containerClient.GetBlobClient(fileName);
Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);
// Upload data from the local file
await blobClient.UploadAsync(localFilePath, true);
Console:
Portal:
Make sure you have to change the Network Access to Enable Public to All, if you're not using VPN or dedicated Network to access Azure Environment.
I'm trying to move blob from one container to another with in the same storage account. I'm using Java SDK for it.
My code:
StorageSharedKeyCredential credential = new StorageSharedKeyCredential("accountname", "accountkey");
BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().endpoint("storageaccountendpoint").credential(credential).buildClient();
BlobContainerClient blobContainerClient = blobServiceClient.getBlobContainerClient("failed");
BlobClient dst = blobContainerClient.getBlobClient("https://xxxstorage.blob.core.windows.net/success/");
BlobClient src = blobContainerClient.getBlobClient("https://xxxstorage.blob.core.windows.net/failed/Graphs.jpeg");
dst.beginCopy(src.getBlobUrl(), null);
I have to move the blob from failed container to success container. But I'm facing 500 internal server error.
What can I try next?
I believe the reason for getting the error is that you are specifying blob URL instead of blob name when creating BlobClient. Also, you were not creating a BlobContainerClient for the source container.
Please try by changing your code to:
StorageSharedKeyCredential credential = new StorageSharedKeyCredential("accountname", "accountkey");
BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().endpoint("storageaccountendpoint").credential(credential).buildClient();
BlobContainerClient sourceBlobContainerClient = blobServiceClient.getBlobContainerClient("success");
BlobContainerClient destinationBlobContainerClient = blobServiceClient.getBlobContainerClient("failed");
BlobClient dst = sourceBlobContainerClient.getBlobClient("Graphs.jpeg");
BlobClient src = destinationBlobContainerClient.getBlobClient("Graphs.jpeg");
dst.beginCopy(src.getBlobUrl(), null);
Do note that I have created a new BlobContainerClient for the source blob container.
Here is my code:
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System;
using Microsoft.WindowsAzure;
using System.Net.Http;
namespace Test
{
class Program
{
static void Main(string[] args)
{
//get the storage account from the connection string
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=https;AccountName=[account name];AccountKey=[account key];EndpointSuffix=core.windows.net");
//instantiate the client
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
//set the container
CloudBlobContainer container = blobClient.GetContainerReference("images");
//get the blob reference
CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob.jpg");
//get image from stream and upload
using (var client = new HttpClient())
{
using (var stream = client.GetStreamAsync(some_url).GetAwaiter().GetResult())
{
if (stream != null)
{
blockBlob.UploadFromStreamAsync(stream);
}
}
client.Dispose();
}
}
}
}
The storage account instantiation works fine.
The container referencing works fine (it actually exists).
The block blob referencing works, as well, with no errors.
The stream has the image I am getting from the URL referenced.
Finally, the upload returns no errors.
Except, there is no image when I navigate to the Blob URI.
I get the following error:
The specified blob does not exist. RequestId:7df0aadc-0001-007c-6b90-f95158000000 Time:2017-07-10T15:21:25.2984015Z
I have also uploaded an image via the Azure Portal and that exists and can be navigated to through a browser.
Am I missing something?
Update below line in your code as you're calling async method.
blockBlob.UploadFromStreamAsync(stream).GetAwaiter().GetResult();
This should resolve your problem.
I've got some files sitting in an Azure File storage.
I'm trying to programatically archive them to Azure Blobs and I'm not sure how to do this efficiently.
I keep seeing code samples about copying from one blob container to another blob container .. but not from a File to a Blob.
Is it possible to do without downloading the entire File content locally and then uploading this content? Maybe use Uri's or something?
More Info:
The File and Blob containers are in the same Storage account.
The storage account is RA-GRS
Here was some sample code I was thinking of doing .. but it just doesn't feel right :( (pseudo code also .. with validation and checks omitted).
var file = await ShareRootDirectory.GetFileReference(fileName);
using (var stream = new MemoryStream())
{
await file.DownloadToStreamAsync(stream);
// Custom method that basically does:
// 1. GetBlockBlobReference
// 2. UploadFromStreamAsync
await cloudBlob.AddItemAsync("some-container", stream);
}
How to copy an Azure File to an Azure Blob?
We also can use CloudBlockBlob.StartCopy(CloudFile). CloudFile type is also can be accepted by the CloudBlockBlob.StartCopy function.
How to copy CloudFile to blob please refer to document. The following demo code is snippet from the document.
// Parse the connection string for the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
Microsoft.Azure.CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create a CloudFileClient object for credentialed access to File storage.
CloudFileClient fileClient = storageAccount.CreateCloudFileClient();
// Create a new file share, if it does not already exist.
CloudFileShare share = fileClient.GetShareReference("sample-share");
share.CreateIfNotExists();
// Create a new file in the root directory.
CloudFile sourceFile = share.GetRootDirectoryReference().GetFileReference("sample-file.txt");
sourceFile.UploadText("A sample file in the root directory.");
// Get a reference to the blob to which the file will be copied.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("sample-container");
container.CreateIfNotExists();
CloudBlockBlob destBlob = container.GetBlockBlobReference("sample-blob.txt");
// Create a SAS for the file that's valid for 24 hours.
// Note that when you are copying a file to a blob, or a blob to a file, you must use a SAS
// to authenticate access to the source object, even if you are copying within the same
// storage account.
string fileSas = sourceFile.GetSharedAccessSignature(new SharedAccessFilePolicy()
{
// Only read permissions are required for the source file.
Permissions = SharedAccessFilePermissions.Read,
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(24)
});
// Construct the URI to the source file, including the SAS token.
Uri fileSasUri = new Uri(sourceFile.StorageUri.PrimaryUri.ToString() + fileSas);
// Copy the file to the blob.
destBlob.StartCopy(fileSasUri);
Note:
If you are copying a blob to a file, or a file to a blob, you must use a shared access signature (SAS) to authenticate the source object, even if you are copying within the same storage account.
Use the Transfer Manager:
https://msdn.microsoft.com/en-us/library/azure/microsoft.windowsazure.storage.datamovement.transfermanager_methods.aspx
There are methods to copy from CloudFile to CloudBlob.
Add the "Microsoft.Azure.Storage.DataMovement" nuget package
using Microsoft.WindowsAzure.Storage.Blob;
using Microsoft.WindowsAzure.Storage.File;
using Microsoft.WindowsAzure.Storage.DataMovement;
private string _storageConnectionString = "your_connection_string_here";
public async Task CopyFileToBlob(string blobContainer, string blobPath, string fileShare, string fileName)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(_connectionString);
CloudFileShare cloudFileShare = storageAccount.CreateCloudFileClient().GetShareReference(fileShare);
CloudFile source = cloudFileShare.GetRootDirectoryReference().GetFileReference(fileName);
CloudBlobContainer blobContainer = storageAccount.CreateCloudBlobClient().GetContainerReference(blobContainer);
CloudBlockBlob target = blobContainer.GetBlockBlobReference(blobPath);
await TransferManager.CopyAsync(source, target, true);
}
I have created container on azure storage and with code below made it work to upload my files(blobs) in container. Now it says upload was successful however I can't figure out how to reach those files to download it. There is no documentation on the internet about it.
Any help would be appreciated to solve it.
// create Azure Storage
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=https;AccountName=bireddy;AccountKey=8OMbjBeJR+SIYaSt0YtBUzivLKPX5ZbsGJeEY9vsX0BPbX3uy9KxOckK7LuLeH3ZbOh+NoEaiEIV/NWvZbFOrA==");
// create a blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// create a container
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// make it public
container.SetPermissions(
new BlobContainerPermissions {
PublicAccess = BlobContainerPublicAccessType.Container
});
// create a block blob
CloudBlockBlob blockBlob = container.GetBlockBlobReference(FileUpload1.FileName);
// upload to Azure Storage
// this has to be changed sometime later
blockBlob.Properties.ContentType = FileUpload1.PostedFile.ContentType;
blockBlob.UploadFromStream(FileUpload1.FileContent);
Grateful to your discussion.
There are a number of options to download your blobs:
Using PowerShell: See the Azure Storage Powershell guide here:
http://azure.microsoft.com/en-us/documentation/articles/storage-powershell-guide-full/
Using Command Line: See the documentation for the AzCopy tool here:
http://aka.ms/azcopy
Using your C# code: There is a guide here, that includes a section on how to download blobs:
http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-blobs/
Using GUI-based explorers: There are a number of third-party explorers, some of them are listed here:
http://blogs.msdn.com/b/windowsazurestorage/archive/2014/03/11/windows-azure-storage-explorers-2014.aspx
// Parse the connection string and return a reference to the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
//Create the Blob service client
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve a reference to a container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Retrieve reference to a blob named
CloudBlockBlob blockBlob2 = container.GetBlockBlobReference(document);
//using (var np = File.Open(#"C:\mydocuments\"+ document, FileMode.Create))
// blockBlob2.DownloadToStream(np, null, options, null);
byte[] fileBytes;
using (var fileStream = new MemoryStream())
{
blockBlob2.DownloadToStream(fileStream, null, options, null);
fileBytes = fileStream.ToArray();
}
return fileBytes;