Azure SAS connection is not working with Azure.Storage.Blobs - azure

I am using Azure.Storage.Blobs, Version=12.1.0.0.
Blobclient is working fine with AccessKey ,but we wanted to use SAS connectionstring
It is throwing exception here.
var blobClient = new BlobServiceClient(**Connectionstring**);
"No valid combination of account information found." this is the exception am getting at the above line.
I am using the below SAS connection format
BlobEndpoint=xxxxxxx;QueueEndpoint=xxxxxxx;FileEndpoint=xxxxxxx;TableEndpoint=xxxxxxx;SharedAccessSignature=xxxxxxx

For SAS connection, you should follow the steps below to generate sas-url: Nav to azure portal -> your storage account -> Shared access signature:
Then copy the "Blob Service SAS URL"(if you want to operate file share / queue, you should use the respective SAS URL).
Then in the code with library Azure.Storage.Blobs, Version=12.1.0.0.:
using Azure.Storage.Blobs;
using System;
namespace ConsoleApp16
{
class Program
{
static void Main(string[] args)
{
//replace the sas_url with the one you copied in the above steps.
string sas_url = "https://xxx.blob.core.windows.net/?sv=2019-02-02&ss=bfqt&srt=sco&sp=rwdlacup&se=2020-01-07T17:04:27Z&st=2020-01-07T09:04:27Z&spr=https&sig=xxx";
Uri uri = new Uri(sas_url);
BlobServiceClient blobServiceClient = new BlobServiceClient(uri);
var blobContainer = blobServiceClient.GetBlobContainerClient("test1");
var blobclient = blobContainer.GetBlobClient("yy1.txt");
blobclient.Upload(#"d:\aa.txt");
Console.WriteLine("**completed**");
Console.ReadLine();
}
}
}

Related

How to resolve `Value for one of the query parameters specified in the request URI is invalid` error?

I am trying to create a parquet file in an ALDS gen2 container but it is failing with below error
Status code 400, "{"error":{"code":"InvalidQueryParameterValue","message":"Value for one of the query parameters specified in the request URI is invalid.\nRequestId:0dec0224-c01f-0048-5227-36ecfc000000\nTime:2023-02-01T10:23:08.8603292Z"}}"
Below is the code snippet I am using to create a file:
public void uploadFile(File fileToUpload) {
StorageSharedKeyCredential sharedKeyCredential = new StorageSharedKeyCredential("adlssynapse123","accountKey");
DataLakeServiceClientBuilder builder = new DataLakeServiceClientBuilder();
DataLakeServiceClient dataLakeServiceClient =
builder.credential(sharedKeyCredential).endpoint(endpoint).buildClient();
DataLakeFileSystemClient fileSystemClient = dataLakeServiceClient.getFileSystemClient("hdfs");
DataLakeDirectoryClient directoryClient =fileSystemClient.getDirectoryClient("synapse/workspaces/adls-synapse/warehouse/adlstesting");
DataLakeFileClient fileClient = directoryClient.createFile(fileToUpload.getName()); // This is where the execution fails
fileClient.uploadFromFile(fileToUpload.getPath());
}
can some please help resolve this issue?
Status code 400, "{"error":{"code":"InvalidQueryParameterValue","message":"Value for one of the query parameters specified in the request URI is invalid.\nRequestId:0dec0224-c01f-0048-5227 36ecfc000000\nTime:2023-02-01T10:23:08.8603292Z"}}"
The above error indicates one of the query parameters specified on the request URI is Invalid.
The issue is likely with the "fileToUpload.getName()" argument provided to the "createFile" method, as that is where the execution fails. Verify if the value of fileToUpload.getName() is a valid file name and identifies the API's requirements.
To create parquet file and upload in ADLS Gen 2 you can refer the below sample code:
Code:
public static void main( String[] args )
{
StorageSharedKeyCredential sharedKeyCredential = new StorageSharedKeyCredential("Straccountname","<Account key>");
DataLakeServiceClientBuilder builder = new DataLakeServiceClientBuilder();
DataLakeServiceClient dataLakeServiceClient =builder
.credential(sharedKeyCredential)
.endpoint("https://Straccountname.dfs.core.windows.net")
.buildClient();
DataLakeFileSystemClient fileSystemClient = dataLakeServiceClient.getFileSystemClient("test");
DataLakeDirectoryClient directoryClient =fileSystemClient.getDirectoryClient("synapse/workspaces");
DataLakeFileClient fileClient = directoryClient.getFileClient("demo.parquet");
fileClient.uploadFromFile("<filepath>");
}
Console:
Portal:
Reference:
Use Java to manage data in Azure Data Lake Storage Gen2 - Azure Storage | Microsoft Learn

Azure Data Storage: Unable to upload file (Not authorized to perform this operation using this permission)

I'm trying to follow the example to upload a file to Azure Data Storage as mentioned in the documentation : https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-dotnet?tabs=visual-studio%2Cmanaged-identity%2Croles-azure-portal%2Csign-in-azure-cli%2Cidentity-visual-studio
Following is my code:
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System;
using System.IO;
using Azure.Identity;
// TODO: Replace <storage-account-name> with your actual storage account name
var blobServiceClient = new BlobServiceClient(
new Uri("https://[some azure storage]"),
new DefaultAzureCredential());
// Set container name
string containerName = "data";
// Get container
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
// Create a local file in the ./data/ directory for uploading and downloading
string localPath = "data";
Directory.CreateDirectory(localPath);
string fileName = "testupload" + Guid.NewGuid().ToString() + ".txt";
string localFilePath = Path.Combine(localPath, fileName);
// Write text to the file
await File.WriteAllTextAsync(localFilePath, "Hello, World!");
// Get a reference to a blob
BlobClient blobClient = containerClient.GetBlobClient(fileName);
Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);
// Upload data from the local file
await blobClient.UploadAsync(localFilePath, true);
But I'm getting an error message that the request is not authorized.
Error message:
Azure.RequestFailedException: 'This request is not authorized to perform this operation using this permission.
I have Contributor role (which based on description is Grant full access to manage all resources ....), is this role still not enough to perform the operation?
I tried in my environment and got below results:
Initially I tried same code in my environment and got same error
Console:
Azure.RequestFailedException: 'This request is not authorized to perform this operation using this permission.
The above error occurs when your principal doesn't has access to azure blob storage.
For accessing blob storage through identity, Gaurav Mantri comment it is correct, you need a role to access blob storage,
The roles are
Storage-blob-contributor(or)
Storage-blob-owner
Go to portal -> storage accounts -> Access Control (IAM) ->Add -> Add role assignments -> storage-blob-contributor or storage-blob-owner role to the storage account.
Portal:
After assigning role to my storage account, I executed same code and it successfully uploaded file in azure blob storage.
Code:
using Azure.Storage.Blobs;
using System;
using System.IO;
using Azure.Identity;
// TODO: Replace <storage-account-name> with your actual storage account name
var blobServiceClient = new BlobServiceClient(
new Uri("https://[some azure storage]"),
new DefaultAzureCredential());
// Set container name
string containerName = "test";
// Get container
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
// Create a local file in the ./data/ directory for uploading and downloading
string localPath = "data";
Directory.CreateDirectory(localPath);
string fileName = "testupload" + Guid.NewGuid().ToString() + ".txt";
string localFilePath = Path.Combine(localPath, fileName);
// Write text to the file
await File.WriteAllTextAsync(localFilePath, "Hello, World!");
// Get a reference to a blob
BlobClient blobClient = containerClient.GetBlobClient(fileName);
Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);
// Upload data from the local file
await blobClient.UploadAsync(localFilePath, true);
Console:
Portal:
Make sure you have to change the Network Access to Enable Public to All, if you're not using VPN or dedicated Network to access Azure Environment.

Write file to blob storage and save the SaS URL using C#

I am trying to create an Azure Function that create files in blob storage and then save a pre-signed blob file url that is generated dynamically in an azure table so that we can return blob file url to the client program to open.
I am able to create the files in blob storage and save the urls. Right now, the code makes the file urls public, I am not sure how can I make the current code generate SaS url instead of public url and save it to the azure table.
I didn't see any example that shows the usage of CloudBlobClient and SaS. Appreciate any help.
[FunctionName("CreateFiles")]
public static async void Run([QueueTrigger("JobQueue", Connection = "")]string myQueueItem,
[Table("SubJobTable", Connection = "AzureWebJobsStorage")] CloudTable subJobTable,
ILogger log)
{
Job job = JsonConvert.DeserializeObject<Job>(myQueueItem);
var storageAccount = CloudStorageAccount.Parse("UseDevelopmentStorage=true");
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
string containerName = $"{job.Name.ToLowerInvariant()}{Guid.NewGuid().ToString()}";
CloudBlobContainer cloudBlobContainer =
cloudBlobClient.GetContainerReference(containerName);
cloudBlobContainer.CreateIfNotExists();
BlobContainerPermissions permissions = new BlobContainerPermissions
{
PublicAccess = BlobContainerPublicAccessType.Blob
};
cloudBlobContainer.SetPermissions(permissions);
string localPath = "./data/";
string localFileName = $"{job.Id}.json";
string localFilePath = Path.Combine(localPath, localFileName);
File.WriteAllText(localFilePath, myQueueItem);
CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(localFileName);
log.LogInformation("Uploading to Blob storage as blob:\n\t {0}\n", cloudBlockBlob.Uri.AbsoluteUri);
cloudBlockBlob.UploadFromFile(localFilePath);
// update the table with file uri
DynamicTableEntity entity = new DynamicTableEntity(job.Id, job.PracticeId);
entity.Properties.Add("FileUri", new EntityProperty(cloudBlockBlob.Uri.AbsoluteUri));
entity.Properties.Add("Status", new EntityProperty("Complete"));
TableOperation mergeOperation = TableOperation.InsertOrMerge(entity);
subJobTable.Execute(mergeOperation);
}
It looks like that your code is making use of the older version of the SDK (Microsoft.Azure.Storage.Blob). If that's the case, then you would need to use GetSharedAccessSignature method in CloudBlob to generate a shared access signature token.
Your code would be something like:
...
cloudBlockBlob.UploadFromFile(localFilePath);
var sasToken = cloudBlockBlob. GetSharedAccessSignature(sas-token-parameters);
var sasUrl = "${cloudBlockBlob.Uri.AbsoluteUri}?${sasToken}";//Add question mark only if sas token does not have it.
...

ASP.Net Core 2.2 - Azure SAS Tokens?

I am trying to figure out how to generate a time based token for users to access data that is stored in an Azure Storage Account Blob container. Users upload various data (PDFs, images) but I don't want links to this data to be public. The recommended strategy was to use a SAS token which I was able to get this working under .Net using the following function which I found on the MS site about a year ago:
//Function for getting a temporary Azure SAS
public static string GetAzureSASToken(string userhashid, int minutes)
{
//Get Azure SAS Token so we can allow them to temporarily view the photos
UploadedFileInfo uploadedfileinfo = new UploadedFileInfo();
//Azure User containter must be all lowercase!!
uploadedfileinfo.usercontainer = "user-" + userhashid;
uploadedfileinfo.azureurl = ConfigurationManager.AppSettings["AzureURL"].ToString() + uploadedfileinfo.usercontainer + "/";
// Retrieve storage account from connection string.
string azureconnection = CloudConfigurationManager.GetSetting("StorageConnectionString");
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(azureconnection);
CloudBlobClient client = storageAccount.CreateCloudBlobClient();
CloudBlobContainer blobContainer = client.GetContainerReference(uploadedfileinfo.usercontainer);
//Set the expiry time and permissions for the container.
//In this case no start time is specified, so the shared access signature becomes valid immediately.
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessExpiryTime = DateTimeOffset.UtcNow.AddMinutes(minutes);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
//Generate the shared access signature on the container, setting the constraints directly on the signature.
string sasContainerToken = blobContainer.GetSharedAccessSignature(sasConstraints);
return sasContainerToken;
}
The problem is that I now need to access these files from an Asp.Net Core 2.2 app and I can't seem to figure out how to replicate the code to get the token (the Core libraries are different)
Any suggestions on how to accomplish this in .Net Core 2.2?
Thanks!
Just install the latest nuget package Microsoft.Azure.Storage.Blob -Version 11.1.0.
Then your .net core code( same as .net framework code in your post) can work well as .net framework code.
Here is an sample code of .net core 2.2. I didn't read the settings from configure file, so it's a little different:
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Auth;
using Microsoft.Azure.Storage.Blob;
using System;
namespace ConsoleApp5
{
class Program
{
static void Main(string[] args)
{
string sas = GetAzureSASToken();
Console.WriteLine(sas);
Console.ReadLine();
}
public static string GetAzureSASToken()
{
string accountName = "xxx";
string accountKey = "xxx";
CloudStorageAccount storageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
CloudBlobClient client = storageAccount.CreateCloudBlobClient();
CloudBlobContainer blobContainer = client.GetContainerReference("test1");
//Set the expiry time and permissions for the container.
//In this case no start time is specified, so the shared access signature becomes valid immediately.
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessExpiryTime = DateTimeOffset.UtcNow.AddMinutes(5);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
//Generate the shared access signature on the container, setting the constraints directly on the signature.
string sasContainerToken = blobContainer.GetSharedAccessSignature(sasConstraints);
return sasContainerToken;
}
}
}
And the test result:

Azure Blob Storage Uploads Fine, but Blob Doesn't Exist

Here is my code:
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System;
using Microsoft.WindowsAzure;
using System.Net.Http;
namespace Test
{
class Program
{
static void Main(string[] args)
{
//get the storage account from the connection string
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=https;AccountName=[account name];AccountKey=[account key];EndpointSuffix=core.windows.net");
//instantiate the client
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
//set the container
CloudBlobContainer container = blobClient.GetContainerReference("images");
//get the blob reference
CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob.jpg");
//get image from stream and upload
using (var client = new HttpClient())
{
using (var stream = client.GetStreamAsync(some_url).GetAwaiter().GetResult())
{
if (stream != null)
{
blockBlob.UploadFromStreamAsync(stream);
}
}
client.Dispose();
}
}
}
}
The storage account instantiation works fine.
The container referencing works fine (it actually exists).
The block blob referencing works, as well, with no errors.
The stream has the image I am getting from the URL referenced.
Finally, the upload returns no errors.
Except, there is no image when I navigate to the Blob URI.
I get the following error:
The specified blob does not exist. RequestId:7df0aadc-0001-007c-6b90-f95158000000 Time:2017-07-10T15:21:25.2984015Z
I have also uploaded an image via the Azure Portal and that exists and can be navigated to through a browser.
Am I missing something?
Update below line in your code as you're calling async method.
blockBlob.UploadFromStreamAsync(stream).GetAwaiter().GetResult();
This should resolve your problem.

Resources