Azure Function - Managed IDs to write to storage table - failing with 403 AuthorizationPermissionMismatch - azure

I have an Azure function application (HTTP trigger) that writes to the storage queue and table. Both fail when I try to change to managed Id. This post / question is about just the storage table part.
Here's the code that does the actual writing to the table:
GetStorageAccountConnectionData();
try
{
WorkspaceProvisioningRecord provisioningRecord = new PBIWorkspaceProvisioningRecord();
provisioningRecord.status = requestType;
provisioningRecord.requestId = requestId;
provisioningRecord.workspace = request;
#if DEBUG
Console.WriteLine(Environment.GetEnvironmentVariable("AZURE_TENANT_ID"));
Console.WriteLine(Environment.GetEnvironmentVariable("AZURE_CLIENT_ID"));
DefaultAzureCredentialOptions options = new DefaultAzureCredentialOptions()
{
Diagnostics =
{
LoggedHeaderNames = { "x-ms-request-id" },
LoggedQueryParameters = { "api-version" },
IsLoggingContentEnabled = true
},
ExcludeVisualStudioCodeCredential = true,
ExcludeAzureCliCredential = true,
ExcludeManagedIdentityCredential = true,
ExcludeAzurePowerShellCredential = true,
ExcludeSharedTokenCacheCredential = true,
ExcludeInteractiveBrowserCredential = true,
ExcludeVisualStudioCredential = true
};
#endif
DefaultAzureCredential credential = new DefaultAzureCredential();
Console.WriteLine(connection.storageTableUri);
Console.WriteLine(credential);
var serviceClient = new TableServiceClient(new Uri(connection.storageTableUri), credential);
var tableClient = serviceClient.GetTableClient(connection.tableName);
await tableClient.CreateIfNotExistsAsync();
var entity = new TableEntity();
entity.PartitionKey = provisioningRecord.status;
entity.RowKey = provisioningRecord.requestId;
entity["requestId"] = provisioningRecord.requestId.ToString();
entity["status"] = provisioningRecord.status.ToString();
entity["workspace"] = JsonConvert.SerializeObject(provisioningRecord.workspace);
//this is where I get the 403
await tableClient.UpsertEntityAsync(entity);
//other stuff...
catch(AuthenticationFailedException e)
{
Console.WriteLine($"Authentication Failed. {e.Message}");
WorkspaceResponse response = new PBIWorkspaceResponse();
response.requestId = null;
response.status = "failure";
return response;
}
catch (Exception ex)
{
Console.WriteLine($"whoops! Failed to create storage record:{ex.Message}");
WorkspaceResponse response = new WorkspaceResponse();
response.requestId = null;
response.status = "failure";
return response;
}
I have the client id/ client secret for this security principal defined in my local.settings.json as AZURE_TENANT_ID/AZURE_CLIENT_ID/AZURE_CLIENT_SECRET.
The code dies trying to do the upsert. And it never hits the AuthenticationFailedException - just the general exception.
The security principal defined in the AZURE* variables was used to created this entire application including the storage account.

To manage data inside a storage account (like creating table etc.), you will need to assign different sets of permissions. Owner role is a control-plane role that enables you to manage storage accounts themselves and not the data inside them.
From this link:
Only roles explicitly defined for data access permit a security
principal to access blob data. Built-in roles such as Owner,
Contributor, and Storage Account Contributor permit a security
principal to manage a storage account, but do not provide access to
the blob data within that account via Azure AD.
Even though the text above is for Blobs, same thing applies for Tables as well.
Please assign Storage Table Data Contributor to your Managed Identity and then you should not get this error.

Related

Azure blob read SAS token throws AuthorizationPermissionMismatch exception

I'm trying to generate a SAS token for a blob, so that any user with the token can read the blob. Below is the code I have. I get an exception when I try to read the blob. If I grant "Storage Blob Data Reader" access to the user, then it works. My understanding is that user with SAS token should be able to read the blob without granting specific permission. what am I missing here ?
BlobServiceClient blobServiceClient = new BlobServiceClient(new Uri("https://accountname.blob.core.windows.net/"), new DefaultAzureCredential());
UserDelegationKey key = await blobServiceClient.GetUserDelegationKeyAsync(DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow.AddDays(1));
BlobSasBuilder sasBuilder = new BlobSasBuilder()
{
BlobContainerName = "containerName",
BlobName = "file.json",
Resource = "b",
StartsOn = DateTimeOffset.UtcNow,
ExpiresOn = DateTimeOffset.UtcNow.AddHours(1)
};
sasBuilder.SetPermissions(BlobSasPermissions.Read);
string sasToken = sasBuilder.ToSasQueryParameters(key, "accountname").ToString();
UriBuilder fullUri = new UriBuilder()
{
Scheme = "https",
Host = string.Format("{0}.blob.core.windows.net", "accountname"),
Path = string.Format("{0}/{1}", "containerName", "file.json"),
Query = sasToken
};
var blobClient = new Azure.Storage.Blobs.BlobClient(fullUri.Uri);
using (var stream = await blobClient.OpenReadAsync()) // throws exception
{ }
Exception : Service request failed.
Status: 403 (This request is not authorized to perform this operation using this permission.)
ErrorCode: AuthorizationPermissionMismatch
I believe you are getting this error is because the user for which you are getting the user delegation key does not have permissions to access the data in the storage account.
Assigning Owner permission enables the user to manage the storage account itself, it does not give them permissions to manage the data.
Please try by assigning the user one of the data roles described here: https://learn.microsoft.com/en-us/azure/storage/blobs/authorize-access-azure-active-directory#azure-built-in-roles-for-blobs.
To learn more about RBAC roles to manage data, please see this link: https://learn.microsoft.com/en-us/azure/storage/blobs/assign-azure-role-data-access?tabs=portal.

Can't Access Storage Queue receiveMessages due AuthorizationPermissionMismatch

I have accessed Azure Storage Queue methods using ClientSecretCredential but on accessing
queue receiveMessages, queue peekMessages and deleteMessages it is giving me error
RestError: This request is not authorized to perform this operation using this permission.
RequestId:c92577923e-a603-0004-61c0-f70a19000
here is my node js code
const { QueueServiceClient } = require("#azure/storage-queue");
const { ClientSecretCredential } = require("#azure/identity");
async function getQueueMessages() {
try {
let myStorageAccount = "hellostorage";
const credential = new ClientSecretCredential(tenantId, app_id, SecretKey);
const queueServiceClient = new QueueServiceClient(
`https://${myStorageAccount}.queue.core.windows.net`,
credential
);
const queueName = "hello-queue";
const queueClient = queueServiceClient.getQueueClient(queueName);
const response = await queueClient.receiveMessages(10);
console.log("response: ", response);
} catch (error) {
console.log("error: ", error);
}
}
getQueueMessages();
Here is my App permission
The screenshot you shared essentially allows your Service Principal to acquire token for your Storage Accounts. It does not give you permissions to perform operations on a Storage Account and this is why you are getting this error.
What you would need to do is give appropriate data related permissions to your Service Principal on a Storage Account. Please see this link for the appropriate RBAC roles that you must assign to your Service Principal to perform data related operations: https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-azure-active-directory#manage-access-rights-with-rbac.
You can try with Storage Queue Data Message Processor or Storage Queue Data Contributor roles.
After you apply appropriate roles, you should be able to perform the operations.

Getting Error while Creating SAS Token for Azure Storage Blob with MSI

I'm trying to create a SAS token for a storage blob. I use a StorageCredentials which was created with MSI (Managed Service Identity) to create the CloudBlobClient. When creating the SAS I'm getting "Cannot create Shared Access Signature unless Account Key credentials are used". Is there support to SAS with MSI?
var container = blobClient.GetContainerReference(containerName);
var blockBlob = container.GetBlockBlobReference(snapname);
var sas = string.Concat(blockBlob.Uri.ToString(), blockBlob.GetSharedAccessSignature(sasConstraints));
This is how I create the StorageCredentials:
tokenCallback = CreateMsiCallback();
var initToken = await tokenCallback(audience);
return new StorageCredentials(
new TokenCredential(initToken, async (state, token) =>
{
var accessToken = await _tokenCallback(audience);
return new NewTokenAndFrequency(accessToken, TimeSpan.FromMinutes(1));
}, null, TimeSpan.FromMinutes(1))
);
To create the token callback I use HttpClient
public Func<string, Task<string>> CreateMsiCallback()
{
var handler = new HttpClientHandler
{
ServerCertificateCustomValidationCallback =
(httpRequestMessage, cert, certChain, policyErrors) =>
{
if (policyErrors == SslPolicyErrors.None)
{
return true;
}
return 0 == string.Compare(cert.GetCertHashString(), FabricThumbprint, StringComparison.OrdinalIgnoreCase);
}
};
var client = new HttpClient(handler)
{
DefaultRequestHeaders =
{
{"secret", FabricAuthenticationCode }
}
};
return async (resource) =>
{
var requestUri = $"{FabricMsiEndpoint}?api-version={FabricApiVersion}&resource={HttpUtility.UrlEncode(resource)}";
var requestMessage = new HttpRequestMessage(HttpMethod.Get, requestUri);
var response = await client.SendAsync(requestMessage);
response.EnsureSuccessStatusCode();
var tokenResponseString = await response.Content.ReadAsStringAsync();
var tokenResponseObject =
JsonConvert.DeserializeObject<ManagedIdentityTokenResponse>(tokenResponseString);
return tokenResponseObject.AccessToken;
};
}
}
Based on this Github issue, you will need to assign Storage data roles to your MSI in order to generate SAS token. From this thread:
The error is because your oauth account don't have permission to
generateUserDelegationKey. To get SAS with Oauth storage context
(New-AzStorageContext -UseConnectedAuth), we need first generate
UserDelegationKey from server , then use the key to generate the SAS
token.
Please check have you assigned correct roles to the Oauth login user
(with Connect-AzAccount). like at least one of the following 4 roles
on the specific storage account:
Storage Blob Data Owner
Storage Blob Data Contributor
Storage Blob Data Reader
Storage Blob Delegator

Create azure cdn endpoint for azure container

I need to create Azure CDN Endpoint for Azure Container. I am using below code to do so.
Endpoint endpoint = new Endpoint() {
IsHttpAllowed = true,
IsHttpsAllowed = true,
Location = this.config.ResourceLocation,
Origins = new List<DeepCreatedOrigin> { new DeepCreatedOrigin(containerName, string.Format(STORAGE_URL, storageAccountName)) },
OriginPath = "/" + containerName,
};
await this.cdnManagementClient.Endpoints.CreateAsync(this.config.ResourceGroupName, storageAccountName, containerName, endpoint);
All the information I provide is correct and Endpoint is getting created successfully. But when I try to access any blob inside it. It is giving an InvalidUrl error.
However the weird thing is If I create the same endpoint using same values through portal, I am able to access and download blobs.
Anyone please let me know what am I doing wrong in my code? Do I need to pass any extra parameters?
As far as I know, if you want to create a storage CDN in code, you need set the OriginHostHeader value as your storage account URL.
More details, you could refer to below codes:
// Create CDN client
CdnManagementClient cdn = new CdnManagementClient(new TokenCredentials(token))
{ SubscriptionId = subscriptionId };
//ListProfilesAndEndpoints(cdn);
Endpoint e1 = new Endpoint()
{
// OptimizationType = "storage",
Origins = new List<DeepCreatedOrigin>() { new DeepCreatedOrigin("{yourstoragename}-blob-core-windows-net", "{yourstoragename}.blob.core.windows.net") },
OriginHostHeader = "{yourstoragename}.blob.core.windows.net",
IsHttpAllowed = true,
IsHttpsAllowed = true,
OriginPath=#"/foo2",
Location = "EastAsia"
};
cdn.Endpoints.Create(resourcegroup, profilename, enpointname, e1);
Besides, I suggest you could generate SAS token to directly access the blob file by URL.

Create Shared Access Token with Microsoft.WindowsAzure.Storage returns 403

I have a fairly simple method that uses the NEW Storage API to create a SAS and copy a blob from one container to another.
I am trying to use this to Copy blob BETWEEN STORAGE ACCOUNTS. So I have TWo Storage accounts, with the exact same Containers, and I am trying to copy a blob from the Storage Account's Container to another Storage Account's Container.
I don't know if the SDK is built for that, but it seems like it would be a common scenario.
Some additional information:
I create the token on the Destination Container.
Does that token need to be created on both the source and destination? Does it take time to register the token? Do I need to create it for each request, or only once per token "lifetime"?
I should mention a 403 is an Unauthorized Result http error code.
private static string CreateSharedAccessToken(CloudBlobClient blobClient, string containerName)
{
var container = blobClient.GetContainerReference(containerName);
var blobPermissions = new BlobContainerPermissions();
// The shared access policy provides read/write access to the container for 10 hours:
blobPermissions.SharedAccessPolicies.Add("SolutionPolicy", new SharedAccessBlobPolicy()
{
// To ensure SAS is valid immediately we don’t set start time
// so we can avoid failures caused by small clock differences:
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(1),
Permissions = SharedAccessBlobPermissions.Write |
SharedAccessBlobPermissions.Read
});
blobPermissions.PublicAccess = BlobContainerPublicAccessType.Blob;
container.SetPermissions(blobPermissions);
return container.GetSharedAccessSignature(new SharedAccessBlobPolicy(), "SolutionPolicy");
}
Down the line I use this token to call a copy operation, which returns a 403:
var uri = new Uri(srcBlob.Uri.AbsoluteUri + blobToken);
destBlob.StartCopyFromBlob(uri);
My version of Azure.Storage is 2.1.0.2.
Here is the full copy method in case that helps:
private static void CopyBlobs(
CloudBlobContainer srcContainer, string blobToken,
CloudBlobContainer destContainer)
{
var srcBlobList
= srcContainer.ListBlobs(string.Empty, true, BlobListingDetails.All); // set to none in prod (4perf)
//// get the SAS token to use for all blobs
//string token = srcContainer.GetSharedAccessSignature(
// new SharedAccessBlobPolicy(), "SolutionPolicy");
bool pendingCopy = true;
foreach (var src in srcBlobList)
{
var srcBlob = src as ICloudBlob;
// Determine BlobType:
ICloudBlob destBlob;
if (srcBlob.Properties.BlobType == BlobType.BlockBlob)
{
destBlob = destContainer.GetBlockBlobReference(srcBlob.Name);
}
else
{
destBlob = destContainer.GetPageBlobReference(srcBlob.Name);
}
// Determine Copy State:
if (destBlob.CopyState != null)
{
switch (destBlob.CopyState.Status)
{
case CopyStatus.Failed:
log.Info(destBlob.CopyState);
break;
case CopyStatus.Aborted:
log.Info(destBlob.CopyState);
pendingCopy = true;
destBlob.StartCopyFromBlob(destBlob.CopyState.Source);
return;
case CopyStatus.Pending:
log.Info(destBlob.CopyState);
pendingCopy = true;
break;
}
}
// copy using only Policy ID:
var uri = new Uri(srcBlob.Uri.AbsoluteUri + blobToken);
destBlob.StartCopyFromBlob(uri);
//// copy using src blob as SAS
//var source = new Uri(srcBlob.Uri.AbsoluteUri + token);
//destBlob.StartCopyFromBlob(source);
}
}
And finally the account and client (vetted) code:
var credentials = new StorageCredentials("BAR", "FOO");
var account = new CloudStorageAccount(credentials, true);
var blobClient = account.CreateCloudBlobClient();
var sasToken = CreateSharedAccessToken(blobClient, "content");
When I use a REST client this seems to work... any ideas?
Consider also this problem:
var uri = new Uri(srcBlob.Uri.AbsoluteUri + blobToken);
Probably you are calling the "ToString" method of Uri that produce a "Human redable" version of the url. If the blobToken contain special caracters like for example "+" this will cause a token malformed error on the storage server that will refuse to give you the access.
Use this instead:
String uri = srcBlob.Uri.AbsoluteUri + blobToken;
Shared Access Tokens are not required for this task. I ended up with two accounts and it works fine:
var accountSrc = new CloudStorageAccount(credsSrc, true);
var accountDest = new CloudStorageAccount(credsSrc, true);
var blobClientSrc = accountSrc.CreateCloudBlobClient();
var blobClientDest = accountDest.CreateCloudBlobClient();
// Set permissions on the container.
var permissions = new BlobContainerPermissions {PublicAccess = BlobContainerPublicAccessType.Blob};
srcContainer.SetPermissions(permissions);
destContainer.SetPermissions(permissions);
//grab the blob
var sourceBlob = srcContainer.GetBlockBlobReference("FOO");
var destinationBlob = destContainer.GetBlockBlobReference("BAR");
//create a new blob
destinationBlob.StartCopyFromBlob(sourceBlob);
Since both CloudStorageAccount objects point to the same account, copying without a SAS token would work just fine as you also mentioned.
On the other hand, you need either a publicly accessible blob or a SAS token to copy from another account. So what you tried was correct, but you established a container-level access policy, which can take up to 30 seconds to take effect as also documented in MSDN. During this interval, a SAS token that is associated with the stored access policy will fail with status code 403 (Forbidden), until the access policy becomes active.
One more thing that I would like to point is; when you call Get*BlobReference to create a new blob object, the CopyState property will not be populated until you do a GET/HEAD operation such as FetchAttributes.

Resources