Authorization failed using Azure Service Bus API to manage queues - azure

I'm trying to use the Azure Service Bus api to purge messages from certain queues.
According to the documentation (https://learn.microsoft.com/en-us/rest/api/storageservices/clear-messages) I would do this using the following request:
https://myaccount.queue.core.windows.net/myqueue/messages
This seems simple enough, but the problem is I just cannot get this request to authorize correctly.
(result = 401 Unauthorized)
Apparently the signing process is rather elaborate and described here (https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key)
I had many attempts, like the following code:
var httpClient = new HttpClient();
using (HMACSHA256 hmac = new HMACSHA256())
{
var StringToSign = $"GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-date:{DateTime.UtcNow.ToString("o")}\nx-ms-version:2015-02-21\n\n\n\n";
var stringToSignHash = hmac.ComputeHash(Encoding.UTF8.GetBytes(StringToSign));
var base64StringToSign = Convert.ToBase64String(stringToSignHash);
var signature = $"{base64StringToSign}, {MySecretKey}";
using (HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Delete, "https://myaccount.servicebus.windows.net/myqueue/messages"))
{
//requestMessage.Headers.Add("x-ms-date", DateTime.UtcNow.ToString());
requestMessage.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("SharedKey", signature);
var result = httpClient.SendAsync(requestMessage).GetAwaiter().GetResult();
}
}
For MySecretKey I used my shared access policy key. I tried using as is, and also decoded base64 to ascii.
Does anyone have more success with this?
Are there more simple ways to access the api?
Thanks.

You can leverage the code here in different languages to generate the SAS token.
Updated:
Can you please confirm which API you are trying to execute? As per the code, you are calling the Delete method on myaccount.servicebus.windows.net (i.e. service bus resource). Delete API operation completes the processing of a locked message and deletes it from the queue or subscription. The samples/reference article that you have shared is for the delete messages from the storage queue. If your requirement is to purge all messages from the service bus queue then you need to use Recieve and Delete API. Alternative you can also use service bus explorer to purge the messages.

As an alternative you can use Service Bus Cloud Explorer which runs in the browser with logged-in user Azure account by default and have a purge functionality.
Keep in mind that it will take time to clear up queues or subscriptions. More messages it needs to purge; the more time it would take.

Related

Service Bus Connection String using the Azure.ServiceBus.Messaging C# SDK for SAS token

I am trying to migrate from the Legacy Azure service bus SDK to the new one "Azure.Messaging.ServiceBus". But it seems many of the functionalities that were there in the old ones are not supported. How do I generate a service bus connection string using a given Sas Token? The older one could it do with the "ServiceBusConnectionBuilder". (https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.servicebus.servicebusconnectionstringbuilder.-ctor?view=azure-dotnet#Microsoft_Azure_ServiceBus_ServiceBusConnectionStringBuilder__ctor_System_String_)
. How can I do the same with the latest SDK?
To use a SAS token, we recommend that you construct the ServiceBusClient using the overload that accepts an AzureSasCredential rather than a connection string. This has the advantage of allowing the SAS Token to be updated without needing to destroy and recreate the client.
For example:
var credential = new AzureSasCredential("<< SHARED ACCESS KEY STRING >>");
var fullyQualifiedNamespace = "<< NAMESPACE (likely similar to {your-namespace}.servicebus.windows.net) >>";
await using var client = new ServiceBusClient(fullyQualifiedNamespace, credential);
// ...
Though we do not recommend or publicize it, the SharedAccessSignature token is supported in a connection string, resulting in something like:
"Endpoint=sb://<<namespace-name>>.servicebus.windows.net;SharedAccessSignature=<<SAS TOKEN>>"

Azure Identity: Trying to get GetUserDelegationKey to work with an application Service Principal

It's not a good sign when the method I'm asking about, GetUserDelegationKey, yields zero search results on SO. Good luck, me.
I have a C# console app, .Net framework 4.8, using Azure.Storage.Blobs and Azure.Identity that will run on customer servers and access Azure blob storage to hold some stuff. I'm doing all of this with the library, not rolling my own REST. Built with VS2019, testing on Win10.
The plan is to use a single Azure storage account that I own, and create one Container per customer project with per-customer credentials that permit them only their own container. Projects never ever talk to each other.
I could set up credentials in the Azure portal by hand, but I am stubbornly trying to do this in software, where a simple project-management app connects as the project app's service principal (which I defined in Azure AD), creates the container, then creates the shared access signatures with a limited lifetime.
The storage account name / container name / access signature would then be configured on the customer server.
I'm having a terrible time.
Note: this is using the newer BlobClient mechanisms, not the older CloudBlob stuff. Dunno if that matters.
This is all documented here at Microsoft, and following even the simple example gets me the same failure.
using System;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using Azure.Identity;
namespace Azure.Test
{
class Program
{
static void Main(string[] args)
{
var serviceClient = new BlobServiceClient(
new Uri("https://stevestorageacct.blob.core.windows.net"),
new DefaultAzureCredential(true)); // true=pop up login dlg
/*BOOM*/ UserDelegationKey key = serviceClient.GetUserDelegationKey(
DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow.AddDays(30));
// use the key to create the signatures
}
}
}
Even though this program couldn't be simpler, it fails every time with an XML error calling GetUserDelegationKey
Unhandled Exception: Azure.RequestFailedException: The value for one of the XML nodes is not in the correct format.
RequestId:c9b7d324-401e-0127-4a4c-1fe6ce000000
Time:2020-05-01T00:06:21.3544489Z
Status: 400 (The value for one of the XML nodes is not in the correct format.)
ErrorCode: InvalidXmlNodeValue
The XML being sent is supposed to be super simple, I think just the start/end dates for validity, but I have no idea how to get to it to inspect, and http is forbidden for this kind of call, so no Wireshark.
It also fails the same way when I use my application's service principal:
static void Main(string[] args)
{
var tokenCredential = new ClientSecretCredential(
"xxxx-xxxx-xxxx-xxxxx", // tenant ID
"yyyy-yyyy-yyyy-yyyyy, // application ID
"**************"); // client secret
var serviceClient = new BlobServiceClient(
new Uri("https://stevestorageacct.blob.core.windows.net"),
tokenCredential);
UserDelegationKey key = serviceClient.GetUserDelegationKey(
DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow.AddDays(30));
// ALSO: boom
I'm really at a loss.
I suppose I could try rolling my own REST and playing with it that way, but it doesn't feel like this should be necessary: this kind of error feels like a bug even if I'm doing something wrong. XML nodes?
Also open to entirely different ways of approaching this problem if they are superior, but would like to at least find out why this is failing.
I've had some issues with this also. The first things to try is removing the start time (pass null) or setting it ~15 minutes in the past. This is to avoid clock skew between the requesting pc and azure servers.
The second thing to verify is that the user that you are using has the "Storage Blob Data Contributor" role on the storage account. I had to grant it at the storage account level in the end otherwise it just refused to work for me. However in your use case it might be that you need to grant it at the container level to allow you to have one container per client.
Hope this helps.

How to subscribe message from Azure service bus topic subscription to web api?

We have multiple webapi(microservices). To communicate between these micro services we are using Azure service bus Topic\Subscription. How webapi controller can get the message subscription so that api can process the message and insert to database. I don't want to use any azure functions. I want subscribe message in my webapi(micro service directly).
You may create a client for the topic to subscribe the message and set the options to complete after the necessary process is done.
please find the below sample code, where you can read the subscription messages from a topic.
you can use WindowsAzure.ServiceBus -Nuget Package
Sample Code:
using Microsoft.ServiceBus.Messaging;
using Microsoft.ServiceBus;
var con = "your connection string for the topic";
var topic = "your topic name";
var subclient = SubscriptionClient.CreateFromConnectionString(con, topic, "yourSubscriptionName");
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
client.OnMessage(message =>
{
Console.WriteLine("Message:" + message.GetBody<string>());
message.Complete();
}, options);
you may put it in separate method and use as needful.
To poll you may create a webjob, or you can place the code which triggers the web api, so that it will poll the messages at the same time.

Authorization in Event Hubs

I am using SAS token authentication along with device-ID (or publisher-Id) in my event Hub publisher code. But i see that it is possible to send an event to any partition ID by using "CreatePartitionedSender" client even though I have authenticated using a device-ID. Whereas I do not want two different device-Ids publishing events in same partition. Is it possible that we can add some custom "authorization" code along with the SAS authentication to allow limited partition access to any device.
The idea behind adding authorization to device and partition-Id combination was to accommodate single event-hub for multiple tenants. Please advise if I am missing anything.
Please see below the code snippet for publisher:
var publisherId = "1d8480fd-d1e7-48f9-9aa3-6e627bd38bae";
string token = SharedAccessSignatureTokenProvider.GetPublisherSharedAccessSignature(
new Uri("sb://anyhub-ns.servicebus.windows.net/"),
eventHubName, publisherId, "send",
sasKey,
new TimeSpan(0, 5, 0));
var factory = MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb", "anyhub-ns", ""), new MessagingFactorySettings
{
TokenProvider = TokenProvider.CreateSharedAccessSignatureTokenProvider(token),
TransportType = TransportType.Amqp
});
var client = factory.CreateEventHubClient(String.Format("{0}/publishers/{1}", eventHubName, publisherId));
var message = "Event message for publisher: " + publisherId;
Console.WriteLine(message);
var eventData = new EventData(Encoding.UTF8.GetBytes(message));
await client.SendAsync(eventData);
await client.CreatePartitionedSender("5").SendAsync(eventData);
await client.CreatePartitionedSender("6").SendAsync(eventData);
I notice in your example code that you have
var connStr = ServiceBusConnectionStringBuilder.CreateUsingSharedAde...
and then have
CreateFromConnectionString(connectionString
This suggests that you may have used a Connection String containing the send key you used to generate the token rather than the limited access token. In my own tests I did not manage to connect to an EventHub using the EventHubClient, which makes an AMQP connection, with a publisher specific token. This doesn't mean it's not supported just that I got errors that made sense, and the ability to do so doesn't appear to be documented.
What is documented and has an example is making the publisher specific tokens and sending events to the EventHub using the HTTP interface. If you examine the SAS token generated you can see that the token grants access to
[namespace].servicebus.windows.net/[eventhubname]/publishers/[publisherId]
This is consistent with the documentation on the security model, and the general discussion of publisher policies in the overview. I would expect the guarantee on publisherId -> PartitionKey to hold with this interface. Thus each publisherId would have its events end up in a consistent partition.
This may be less than ideal for your multitenant system, but the code to send messages is arguably simpler and is a better match for the intended use case of per device keys. As discussed in this question you would need to do something rather dirty to get each publisher their own partition, and you would be outside the designed use cases.
Cross linking questions can be useful.
For a complete explanation on Event Hubs publisher policy refer this blog.
In short, If you want publisher policy - you will not get partitioned sender. Publisher policy is an extension to SAS security model, designed to support very high number of senders ( to a scale of million senders on event hub ).
With its current authentication model, you can not grant so fine-grained access to publishers. Authentication per partition is not currently supported as per Event Hubs Authentication and Security Model Overview.
You have to either "trust" your publishers, or think on different tenant scheme - i.e. Event Hub per tenant.

Why do calls to the Service Management API work but calls to the Scheduler API fail?

I'm trying to make some calls to the new Azure Scheduler API. However, all my requests come back with this error:
<Error xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<Code>AuthenticationFailed</Code>
<Message>The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription.</Message>
</Error>
I'm pretty sure that I have everything setup correct because I can make calls using the same code and certificate to the Azure Service Management API.
The code I'm using to attach the certificate to the web request is from the MSDN Sample. The Scheduler API calls that I've tried to make are the Check Name Availability, Create Cloud Service, and Create Job Collection.
I've also verified that my subscription is Active for the preview of the Scheduler.
Here is an example of a request I've tried:
Create Cloud Service
Request A cloud service is created by submitting an HTTP PUT operation
to the CloudServices OData collection of the Service Management API
Tenant.Replace with your subscription ID and
with your cloud service ID.
So for this I create a web request pointing to:
https://management.core.windows.net/[MySubId]/cloudServices/[MyNewServiceName]
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(requestUri);
// Define the requred headers to specify the API version and operation type.
request.Headers.Add("x-ms-version", "2012-03-01");
request.Method = "PUT";
request.ContentType = "application/xml";
Next I add the request body as specified in the documentation:
<CloudService xmlns:i='http://www.w3.org/2001/XMLSchema-instance' xmlns='http://schemas.microsoft.com/windowsazure'>
<Label>[MyServiceName]</Label>
<Description>testing</Description>
<GeoRegion>uswest</GeoRegion>
</CloudService>
And finally I add the certificate that I use with my subscription to the account.
// Attach the certificate to the request.
request.ClientCertificates.Add(certificate);
I try to get the response and instead I get the error shown above.
BTW - I've also tried different regions thinking maybe it was a region issue since the scheduler isn't supported in all regions, but I still get the same response.
You need to register the scheduler in your application first by calling (PUT):
<subscription id>/services?service=scheduler.JobCollections&action=register
If you want to do this in .NET you can use the new Management libraries:
var schedulerServiceClient = new SchedulerManagementClient(credentials);
var result = schedulerServiceClient.RegisterResourceProvider();
Console.WriteLine(result.RequestId);
Console.WriteLine(result.StatusCode);
Console.ReadLine();
More detail: http://fabriccontroller.net/blog/posts/a-complete-overview-to-get-started-with-the-windows-azure-scheduler/

Resources