I'm creating solution based on this [documentation][1]. I have it almost working as I want to but it works only when deployed to Azure. App Service has Managed Identity configured and it is assigned Storage Blob Data Contributor role. Is there any way to make it run on my local machine? Currently I need to publish code from VS to Azure and then use Remote debugging to verify how it works.
This is the problematic line:
userDelegationKey key = await blobClient.GetUserDelegationKeyAsync(DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow.AddDays(7));
I get exception:
Status: 400 (The value for one of the XML nodes is not in the correct format.)
ErrorCode: InvalidXmlNodeValue
I use DefaultAzureCredentials and in debug I see it has 3 different sources. First of them is EnvironmentCredential (then ManagedIdentityCredential and SharedTokenCacheCredential). So I tried registering application in Azure AD and configured those 3 env variables but it didn't help. Maybe I need to add some specific permissions to this app?
"AZURE_CLIENT_ID": "",
"AZURE_CLIENT_SECRET": "",
"AZURE_TENANT_ID": ""
Or maybe this could somehow work with my account in Azure? If I'm also assigned Storage Blob Data Contributor role?
EDIT: I captured request and response with Fiddler
Request:
POST https://myaccount.blob.core.windows.net/?restype=service&comp=userdelegationkey HTTP/1.1
Host: myaccount.blob.core.windows.net
x-ms-version: 2019-07-07
x-ms-client-request-id: 23071825-dcf0-4803-a8a9-c44ec38695d5
x-ms-return-client-request-id: true
User-Agent: azsdk-net-Storage.Blobs/12.4.4 (.NET Core 3.1.2; Microsoft Windows 10.0.16299)
Authorization: Bearer Hidden
traceparent: 00-7e23f80250325742853c846211a83d02-6739d4cbc8ef0a46-00
Content-Type: application/xml
Content-Length: 91
<KeyInfo><Start>2020-07-08T06:02:35Z</Start><Expiry>2020-07-15T06:17:35Z</Expiry></KeyInfo>
Response:
HTTP/1.1 400 The value for one of the XML nodes is not in the correct format.
Content-Length: 348
Content-Type: application/xml
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: ba2fe641-f01e-0065-3eef-54acae000000
x-ms-client-request-id: 23071825-dcf0-4803-a8a9-c44ec38695d5
x-ms-version: 2019-07-07
x-ms-error-code: InvalidXmlNodeValue
Date: Wed, 08 Jul 2020 06:16:14 GMT
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidXmlNodeValue</Code><Message>The value for one of the XML nodes is not in the correct format.
RequestId:ba2fe641-f01e-0065-3eef-54acae000000
Time:2020-07-08T06:16:15.7553428Z</Message><XmlNodeName>2020-07-15T06:17:35Z</XmlNodeName><XmlNodeValue>2020-07-15T06:17:35Z</XmlNodeValue></Error>
EDIT 2:
It works with help from #JimXu. In addition I was able to make it work with Azure account configured in Visual Studio so I could remove application registration that I created just for this purpose in Azure AD.
[1]: https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-user-delegation-sas-create-dotnet#example-get-a-user-delegation-sas
If you want to get Azure storage account User Delegation Key, you need to assign Storage Blob Data Contributor, Storage Blob Data Owner or Storage Blob Delegator to the AD application or account. For more details, please refer to the document
Besides, please note that when we get Azure storage account User Delegation Key, we need to provide the expire time. Its value must be a valid date and time within 7 days of the current time. For more details, please refer to here.
For example
string accountName = "jimtestdiag417";
string blobEndpoint = $"https://{accountName}.blob.core.windows.net/";
// Create a new Blob service client with Azure AD credentials.
BlobServiceClient blobClient = new BlobServiceClient(new Uri(blobEndpoint),
new DefaultAzureCredential();
// Get a user delegation key for the Blob service that's valid for seven days.
// to avoid clock skew between the requesting pc and azure servers, please set expire time in future six days
UserDelegationKey key = await blobClient.GetUserDelegationKeyAsync(DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow.AddDays(6));
// Read the key's properties.
Console.WriteLine("User delegation key properties:");
Console.WriteLine("Key signed start: {0}", key.SignedStartsOn);
Console.WriteLine("Key signed expiry: {0}", key.SignedExpiresOn);
Console.WriteLine("Key signed object ID: {0}", key.SignedObjectId);
Console.WriteLine("Key signed tenant ID: {0}", key.SignedTenantId);
Console.WriteLine("Key signed service: {0}", key.SignedService);
Console.WriteLine("Key signed version: {0}", key.SignedVersion);
Console.WriteLine();
Related
I am using Premium/Hot, LRS, StorageV2 Azure storage and trying to write a simple string but I keep getting an authentication error.
To generate the SAS URI of the container in the portal, I went to:
storage resource -> containers -> my container -> shared access token -> generate SAS token and URL
// SAS URI of blob container
var sasUriStr = "https://storageaccountname.blob.core.windows.net/containername?sp=r&st=2021-08-10T00:34:00Z&se=2021-08-15T08:34:00Z&spr=https&sv=2020-08-04&sr=c&sig=ABCDEFGH/YJKLMNOP=";
var uri = new Uri(sasUriStr);
var pageBlobClient = new PageBlobClient(uri);
pageBlobClient.UploadPages(new MemoryStream(Encoding.UTF8.GetBytes("hello world")), 0);
Unhandled exception. Azure.RequestFailedException: Server failed to
authenticate the request. Make sure the value of Authorization header
is formed correctly including the signature.
Time:2021-08-12T20:22:44.0905117Z Status: 403 (Server failed to
authenticate the request. Make sure the value of Authorization header
is formed correctly including the signature.) ErrorCode:
AuthenticationFailed
Additional Information: AuthenticationErrorDetail: Signature did not
match. String to sign used was r
I appreciate any help or hint. Thank you
UPDATE:
After adding /blob name to SAS URI I get this error:
Unhandled exception. Azure.RequestFailedException: The value for one of the HTTP headers is not in the correct format.
RequestId:7a741951-401c-00c9-3ce3-8f5076000000
Time:2021-08-13T01:38:13.4585107Z
Status: 400 (The value for one of the HTTP headers is not in the correct format.)
ErrorCode: InvalidHeaderValue
Additional Information:
HeaderName: x-ms-range
HeaderValue: bytes=0-10
Content:
<?xml version="1.0" encoding="utf-8"?>
<Error><Code>InvalidHeaderValue</Code><Message>The value for one of the HTTP headers is not in the correct format.
RequestId:7a741951-401c-00c9-3ce3-8f5076000000
Time:2021-08-13T01:38:13.4585107Z</Message><HeaderName>x-ms-range</HeaderName><HeaderValue>bytes=0-10</HeaderValue></Error>
Headers:
Server: Windows-Azure-Blob/1.0,Microsoft-HTTPAPI/2.0
x-ms-error-code: InvalidHeaderValue
x-ms-request-id: 7a741951-401c-00c9-3ce3-8f5076000000
x-ms-version: 2020-08-04
x-ms-client-request-id: 4366d771-7f70-4bbf-9677-6e9fcf3cb7a1
Date: Fri, 13 Aug 2021 01:38:12 GMT
Content-Length: 327
Content-Type: application/xml
Solution
Please change your code to something like:
var sasUriStr = "https://storageaccountname.blob.core.windows.net/containername?sp=r&st=2021-08-10T00:34:00Z&se=2021-08-15T08:34:00Z&spr=https&sv=2020-08-04&sr=c&sig=ABCDEFGH/YJKLMNOP=";
var uri = new Uri(sasUriStr);
BlobContainerClient containerClient = new BlobContainerClient(uri);
var pageBlobClient = containerClient.GetPageBlobClientCore("page-blob-name");
pageBlobClient.UploadPages(new MemoryStream(Encoding.UTF8.GetBytes("hello world")), 0);
Please ensure that the blob already exists before you use this code.
Problem
The reason you were running into the problem is because you were trying to create a PageBlobClient using a URI representing a blob container SAS. Because of this, Azure Storage service assumed that your blob name is containername and the container is $root. Since the SAS token was obtained for containername blob container and the service used $root blob container for validating the SAS token, you are getting authorization failed error.
By creating a BlobContainerClient using the SAS URL and then creating a PageBlobClient using BlobContainerClient.GetPageBlobClientCore(String) solves the problem.
I created an Azure KeyVault that I want my App Service to be able to access. From what I can tell, the principal of my App Service should have access to the KeyVault, but I always get the following error when trying to retrieve from it. This happens whether I am running locally in Visual Studio or in Azure.
Service request failed. Status: 403 (Forbidden) Content: {"error":{"code":"Forbidden","message":"The policy requires the caller 'appid=MY_APP_ID;oid=MY_PRINCIPAL_ID_STARTING_WITH_1A5 ;iss=https://sts.windows.net/REDACTED_GUID/' to use on-behalf-of (OBO) flow. For more information on OBO, please see https://go.microsoft.com/fwlink/?linkid=2152310","innererror":{"code":"ForbiddenByPolicy"}}} Headers: Pragma: no-cache x-ms-keyvault-region: westus x-ms-client-request-id: REDACTED x-ms-request-id: REDACTED x-ms-keyvault-service-version: 1.2.236.0 x-ms-keyvault-network-info: conn_type=Ipv4;addr=52.155.40.204;act_addr_fam=InterNetwork; Strict-Transport-Security: REDACTED X-Content-Type-Options: REDACTED Content-Length: 387 Cache-Control: no-cache Content-Type: application/json; charset=utf-8 Date: Wed, 21 Apr 2021 20:10:33 GMT Expires: -1 X-Powered-By: REDACTED
I have looked at the linked KB article and it talks about OAUTH`. I am not trying to use OAUTH, so I don't understand why that is relevant.
Here is the access policy in my KeyVault:
Here is the screenshot from the App Service where I configured a principal.
It clearly says at the bottom that it can be configured to access other resources, so I don't understand why it cannot access KeyVault.
Code used to attempt to access.
var kvUri = "https://" + Properties.Settings.Default.KeyVaultName + ".vault.azure.net";
var credential = new DefaultAzureCredential();
var client = new SecretClient(new Uri(kvUri),credential );
var result = client.GetSecret(secret);
When you set access policy, you need to select service principal with only object id (without app id).
See this similar post answer.
I have a nodejs application using the express framework. The user must sign in via Oauth2 protocol against their MS Azure Active Directory credentials. This is done using the passport-azure-ad-oauth2 npm package
I have successfully got the application working so that I can upload files in blob form to an azure storage container. The container access level is set to private. I have assigned user roles for the container so that certain users within the AD have 'reader and data access'. So my understanding is that when these users are authenticated via Oauth2, they should be able to access the files when retrieving the file's URL. However, after authentication, I am not able to access the files. I get the following error.
ResourceNotFound
The specified resource does not exist. RequestId:6341ef80-f01e-0011-6442-08f7c2000000 Time:2021-02-21T11:14:26.3475641Z
I have also followed the steps to grant the application permissions to azure storage here
What do I need to do so that authenticated users can go to the specific url for each blob and get access? Do I need to pass a token in the request? If so, how do you do this?
Any help would be much appreciated.
The 404 error(The specified resource does not exist) is always related to your request URL, but not the access token.
For example to get blob, you need to use GET Method, and some of the Request Headers are required.
GET https://myaccount.blob.core.windows.net/mycontainer/myblob
headers:
Authorization: Bear <access-token>
x-ms-version: 2020-04-08
x-ms-date: Fri, 26 Jun 2015 23:39:12 GMT
Get access token by application permission without user:
POST https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token
client_id={client_id}
&client_secret={client_secret}
&scope=http://storage.azure.com/.default // change to http://storage.azure.com/ for resource
&grant_type=client_credentials
Get access token by delegated permission with a signed-in user:
GET https://login.microsoftonline.com/{tenant}/oauth2/v2.0/authorize?
client_id={client_id}
&response_type=token
&redirect_uri=https://localhost:44300/
&scope=https://storage.azure.com/user_impersonation
&response_mode=fragment
&state=12345
&nonce=678910
Note: Navigate to your storage account -> Access Control (IAM) -> Add role assignment -> select Storage Blob Data XXX role and your login account. I add Storage Blob Data Contributor in my side, it might take up to 5 minutes to propagate the RBAC rule.
I am trying to sync data across Azure containers in different accounts using a Multitetant App and azcopy tool.
The syncing happens via "azcopy sync" and using separate SAS tokens for both source storage account and destination storage account.
I am generating short lived sas tokens using the Java SDK following the user delegation key method.
Here is the scenario:
Account1 (destination) has App1 registered. i.e. Account1 is home tenant for App1.
Account1 has StorageAccount1 and Container1 configured
App1 is given "Storage Blob Data Contributor" role on StorageAccount1
Account2 (source) has StorageAccount2 and Container2 configured. It is the data source for us. Here, App1 is added as a ServicePrincipal via:
az ad sp create --id client-id-of-App1-in-Account1
In Account2, for this SP, we also gave the Storage Blob Data Reader Role as:
az role assignment create \
--assignee-object-id <object-id-for-this-sp> \
--role 2a2b9908-6ea1-4ae2-8e65-a410df84e7d1 \
--scope /subscriptions/<subsid-account2>/resourceGroups/<resgrpname>/providers/Microsoft.Storage/storageAccounts/<storagename>
This completes the setup.
Now using Java SDK, I generated a user delegation key for both source and destination.
The snippet looks something like below.
genSasToken(String storageAccountName, String containerName,
String tenantId,
String azureAppClientId,
String azureAppClientSecret,
boolean isDestinationAccount) {
BlobContainerSasPermission blobContainerSasPermission =
new BlobContainerSasPermission().setReadPermission(true).setListPermission(true);
if (isDestinationAccount) {
blobContainerSasPermission.setCreatePermission(true)
.setAddPermission(true)
.setWritePermission(true)
.setExecutePermission(true);
}
BlobServiceSasSignatureValues builder =
new BlobServiceSasSignatureValues(OffsetDateTime.now().plusHours(1), blobContainerSasPermission)
.setProtocol(SasProtocol.HTTPS_ONLY);
// Create a BlobServiceClient object which will be used to create a container client
String endpoint = String.format(Locale.ROOT, "https://%s.blob.core.windows.net",
storageAccountName);
ClientSecretCredential clientSecretCredential = new ClientSecretCredentialBuilder()
.clientId(azureAppClientId)
.clientSecret(azureAppClientSecret)
.tenantId(tenantId)
.build();
BlobServiceClient blobServiceClient =
new BlobServiceClientBuilder().endpoint(endpoint).credential(clientSecretCredential).buildClient();
BlobContainerClient blobContainerClient =
blobServiceClient.getBlobContainerClient(containerName);
// Get a user delegation key for the Blob service that's valid for one hour.
// You can use the key to generate any number of shared access signatures over the lifetime of the key.
OffsetDateTime keyStart = OffsetDateTime.now();
OffsetDateTime keyExpiry = OffsetDateTime.now().plusHours(1);
UserDelegationKey userDelegationKey = blobServiceClient.getUserDelegationKey(keyStart, keyExpiry);
String sas = blobContainerClient.generateUserDelegationSas(builder, userDelegationKey);
return sas;
}
Above method is called for both source and destination and gives us SAS tokens generated programmatically.
Interesting thing happening is this:
azcopy sync https://storageaccount2/container2/?sas-token-for2 https://storageaccount1/container1/?sas-token-for1
above sync errors out as
INFO: Authentication failed, it is either not correct, or expired, or does not have the correct permission -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /Users/runner/go/pkg/mod/github.com/!azure/azure-storage-blob-go#v0.10.1-0.20201022074806-8d8fc11be726/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) =====
Description=This request is not authorized to perform this operation.
RequestId:xxx
Time:2021-01-27T10:26:34.9282634Z, Details:
Code: AuthorizationFailure
GET https://storageaccount1.blob.core.windows.net/container1/?comp=properties&restype=account&se=2021-01-27t11%3A10%3A12z&sig=-REDACTED-&ske=2021-01-27t11%3A10%3A12z&skoid=xxx&sks=b&skt=2021-01-27t10%3A10%3A12z&sktid=xxx&skv=2020-02-10&sp=racwle&spr=https&sr=c&sv=2020-02-10&timeout=901
User-Agent: [AzCopy/10.8.0 Azure-Storage/0.10 (go1.13; darwin)]
X-Ms-Client-Request-Id: [xxx]
X-Ms-Version: [2019-12-12]
--------------------------------------------------------------------------------
RESPONSE Status: 403 This request is not authorized to perform this operation.
Content-Length: [246]
Content-Type: [application/xml]
Date: [Wed, 27 Jan 2021 10:26:34 GMT]
Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Client-Request-Id: [xxx]
X-Ms-Error-Code: [AuthorizationFailure]
X-Ms-Request-Id: [xxx]
X-Ms-Version: [2019-12-12]
But, when I try to copy from source to localhost using same sas token 2, it works.
azcopy sync https://storageaccount2/container2/sas-token-for2 /tmp
and
when I try to copy a localhost folder to destination using same sas token it also works.
azcopy sync /tmp https://storageaccount1/container1/sas-token-for1
So the tokens work individually like above.
But azcopy sync https://storageaccount2/container2/sas-token-for2 https://storageaccount1/container1/sas-token-for1
Fails.
Any pointers what might be the issue here?
For syncing you don't need execution permission (which is still in preview in any case). Just remove .setExecutePermission(true) you should be good. In fact syncing should work with only read, write and list permission on destination.
I have come across with the Azure App Configuration service, with the ability to link secret from Azure KeyVault, by creating a new record with an option of Key Vault reference.
I have used Microsoft extension for App Configuration as described in Microsoft Doc
The Steps that have been done
Creating a service principle via CMD - ```az ad sp create-for-rbac -n "http://mySP" --sdk-auth
Given permission to the created service provider also via CMD - az keyvault set-policy -n <your-unique-keyvault-name> --spn <clientId-of-your-service-principal> --secret-permissions delete get list set --key-permissions create decrypt delete encrypt get list unwrapKey wrapKey
Set the client id & secret in environment variables
The method implementation
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder =>
webBuilder.ConfigureAppConfiguration((hostingContext, config) =>
{
var settings = config.Build();
config.AddAzureAppConfiguration(options =>
{
options.Connect(settings["ConnectionStrings:AppConfig"])
.ConfigureKeyVault(kv =>
{
kv.SetCredential(new DefaultAzureCredential());
});
});
})
.UseStartup<Startup>());
}
The issue is started when I trying to fetch data from App Configuration that have at least one KV reference.
I'm getting the following error(only in case of that, one KV reference is linked to the App Configuration)
Service request failed.
Status: 401 (Unauthorized)
Content:
{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret is provided.\r\nTrace ID: \r\nCorrelation ID: \r\nTimestamp: 2020-05-27 22:59:52Z","error_codes":[7000215],"timestamp":"2020-05-27 22:59:52Z","trace_id":"","correlation_id":"","error_uri":"https://login.microsoftonline.com/error?code=7000215"}
Headers:
Cache-Control: no-store, no-cache
Pragma: no-cache
Strict-Transport-Security: REDACTED
X-Content-Type-Options: REDACTED
x-ms-request-id: REDACTED
x-ms-ests-server: REDACTED
P3P: REDACTED
Set-Cookie: REDACTED
Date: Wed, 27 May 2020 22:59:51 GMT
Content-Type: application/json; charset=utf-8
Expires: -1
Content-Length: 471
Any help will much appreciate :)
Thanks!
When using the DefaultAzureCredential, it will first try Managed Identity (recommended for services on Azure), and eventually a service principal that requires the following environment variables to be set for the process (both on your application service, as well as for local development - can be different, so long as the service principal ID has appropriate permissions):
AZURE_TENANT_ID : the tenant ID
AZURE_CLIENT_ID : the service principal ID
AZURE_CLIENT_SECRET : the service principal secret (password) you were shown only after creating the service principal
You could also use the new preview of Azure.Identity which supports other authentication schemes more common and easier to use on development machines, such as the Azure CLI (az login), Visual Studio, and Visual Studio Code.
The issue was, that Visual Studio wasn't able to get the Environment Variable from some reason so it does not send with the request, once I ran the Visual Studio as Admin it works