Authorization in Event Hubs - azure

I am using SAS token authentication along with device-ID (or publisher-Id) in my event Hub publisher code. But i see that it is possible to send an event to any partition ID by using "CreatePartitionedSender" client even though I have authenticated using a device-ID. Whereas I do not want two different device-Ids publishing events in same partition. Is it possible that we can add some custom "authorization" code along with the SAS authentication to allow limited partition access to any device.
The idea behind adding authorization to device and partition-Id combination was to accommodate single event-hub for multiple tenants. Please advise if I am missing anything.
Please see below the code snippet for publisher:
var publisherId = "1d8480fd-d1e7-48f9-9aa3-6e627bd38bae";
string token = SharedAccessSignatureTokenProvider.GetPublisherSharedAccessSignature(
new Uri("sb://anyhub-ns.servicebus.windows.net/"),
eventHubName, publisherId, "send",
sasKey,
new TimeSpan(0, 5, 0));
var factory = MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb", "anyhub-ns", ""), new MessagingFactorySettings
{
TokenProvider = TokenProvider.CreateSharedAccessSignatureTokenProvider(token),
TransportType = TransportType.Amqp
});
var client = factory.CreateEventHubClient(String.Format("{0}/publishers/{1}", eventHubName, publisherId));
var message = "Event message for publisher: " + publisherId;
Console.WriteLine(message);
var eventData = new EventData(Encoding.UTF8.GetBytes(message));
await client.SendAsync(eventData);
await client.CreatePartitionedSender("5").SendAsync(eventData);
await client.CreatePartitionedSender("6").SendAsync(eventData);

I notice in your example code that you have
var connStr = ServiceBusConnectionStringBuilder.CreateUsingSharedAde...
and then have
CreateFromConnectionString(connectionString
This suggests that you may have used a Connection String containing the send key you used to generate the token rather than the limited access token. In my own tests I did not manage to connect to an EventHub using the EventHubClient, which makes an AMQP connection, with a publisher specific token. This doesn't mean it's not supported just that I got errors that made sense, and the ability to do so doesn't appear to be documented.
What is documented and has an example is making the publisher specific tokens and sending events to the EventHub using the HTTP interface. If you examine the SAS token generated you can see that the token grants access to
[namespace].servicebus.windows.net/[eventhubname]/publishers/[publisherId]
This is consistent with the documentation on the security model, and the general discussion of publisher policies in the overview. I would expect the guarantee on publisherId -> PartitionKey to hold with this interface. Thus each publisherId would have its events end up in a consistent partition.
This may be less than ideal for your multitenant system, but the code to send messages is arguably simpler and is a better match for the intended use case of per device keys. As discussed in this question you would need to do something rather dirty to get each publisher their own partition, and you would be outside the designed use cases.
Cross linking questions can be useful.

For a complete explanation on Event Hubs publisher policy refer this blog.
In short, If you want publisher policy - you will not get partitioned sender. Publisher policy is an extension to SAS security model, designed to support very high number of senders ( to a scale of million senders on event hub ).

With its current authentication model, you can not grant so fine-grained access to publishers. Authentication per partition is not currently supported as per Event Hubs Authentication and Security Model Overview.
You have to either "trust" your publishers, or think on different tenant scheme - i.e. Event Hub per tenant.

Related

Authorization failed using Azure Service Bus API to manage queues

I'm trying to use the Azure Service Bus api to purge messages from certain queues.
According to the documentation (https://learn.microsoft.com/en-us/rest/api/storageservices/clear-messages) I would do this using the following request:
https://myaccount.queue.core.windows.net/myqueue/messages
This seems simple enough, but the problem is I just cannot get this request to authorize correctly.
(result = 401 Unauthorized)
Apparently the signing process is rather elaborate and described here (https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key)
I had many attempts, like the following code:
var httpClient = new HttpClient();
using (HMACSHA256 hmac = new HMACSHA256())
{
var StringToSign = $"GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-date:{DateTime.UtcNow.ToString("o")}\nx-ms-version:2015-02-21\n\n\n\n";
var stringToSignHash = hmac.ComputeHash(Encoding.UTF8.GetBytes(StringToSign));
var base64StringToSign = Convert.ToBase64String(stringToSignHash);
var signature = $"{base64StringToSign}, {MySecretKey}";
using (HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Delete, "https://myaccount.servicebus.windows.net/myqueue/messages"))
{
//requestMessage.Headers.Add("x-ms-date", DateTime.UtcNow.ToString());
requestMessage.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("SharedKey", signature);
var result = httpClient.SendAsync(requestMessage).GetAwaiter().GetResult();
}
}
For MySecretKey I used my shared access policy key. I tried using as is, and also decoded base64 to ascii.
Does anyone have more success with this?
Are there more simple ways to access the api?
Thanks.
You can leverage the code here in different languages to generate the SAS token.
Updated:
Can you please confirm which API you are trying to execute? As per the code, you are calling the Delete method on myaccount.servicebus.windows.net (i.e. service bus resource). Delete API operation completes the processing of a locked message and deletes it from the queue or subscription. The samples/reference article that you have shared is for the delete messages from the storage queue. If your requirement is to purge all messages from the service bus queue then you need to use Recieve and Delete API. Alternative you can also use service bus explorer to purge the messages.
As an alternative you can use Service Bus Cloud Explorer which runs in the browser with logged-in user Azure account by default and have a purge functionality.
Keep in mind that it will take time to clear up queues or subscriptions. More messages it needs to purge; the more time it would take.

How to retrieve all FCM tokens from server side to subscribe users to a topic?

Background
I posted a question a few days ago and gained some insight: previous question. However, I did a poor job asking the question so I still don't know how I can retrieve all the users FCM tokens in order to use something like this: Subscribe the client app to a topic. This is also listed under the Server Environments documentation. My clients are on the iOS platform.
This function requires the client FCM tokens to be in a list to iterate over and subscribe each client to a topic to later be used for push notifications. Also I have almost 3,000 users which is more than the 1,000 device limit noted in the documentation.
I was also directed to some server documentation by another clever answer: Manage relationship maps for multiple app instances. However, after reading through the material I still believe I need an array of client registration tokens to use this method. My analysis could be totally incorrect. I am quite ignorant since I'm very young and have a ton to learn.
I also tried to get the client FCM tokens with Bulk retrieve user data, but this does not have access to device tokens.
Question
How cant I obtain all of the users registration tokens to provide to this function:
var registrationTokens = [];
admin.messaging().subscribeToTopic(registrationTokens, topic)
.then(function(response) {
console.log('Successfully subscribed to topic:', response);
})
.catch(function(error) {
console.log('Error subscribing to topic:', error);
});
Furthermore, if I have over 1000 users, let's say 3000. How can I make separate request to subscribe everyone and not surpass the 1000 device per request limit?
Additional question on device groups
I've been trying to accomplish a "Global" push notification by sending messages with topics. Is sending messages to device groups perhaps a better approach?
send different messages to different phone models, your servers can add/remove registrations to the appropriate groups and send the appropriate message to each group
After reading the documentation they both seem adequately to accomplish my goal, however, device groups allows the server to more accurately send messages to specified devices. Are one of these methods a better practice? Or for my case is the difference trivial?
The thing about tokens here is that they can change at any time like:
The app is restored on a new device
The user uninstalls/reinstall the app
The user clears app data.
so even if you save them some where then try to register them all at once, some of them may not be valid at that time.
better way to do this is form your registaration token on your client side (IOS):
Messaging.messaging().token { token, error in
if let error = error {
print("Error fetching FCM registration token: \(error)")
} else if let token = token {
print("FCM registration token: \(token)")
self.fcmRegTokenMessage.text = "Remote FCM registration token: \(token)"
}
}
then monitor changes on this token:
func messaging(_ messaging: Messaging, didReceiveRegistrationToken fcmToken: String?) {
print("Firebase registration token: \(String(describing: fcmToken))")
let dataDict:[String: String] = ["token": fcmToken ?? ""]
NotificationCenter.default.post(name: Notification.Name("FCMToken"), object: nil, userInfo: dataDict)
// TODO: If necessary send token to application server.
// Note: This callback is fired at each app startup and whenever a new token is generated.
}
and send changes to server (which can be any type of servers here) like Firebase Functions with Nodejs. Check here to know how to post a request to Firebase HTTP functions. Then, you can use the same code that you have posted here within that function to register the token to the topic.
This way, you will never exceed that limit and you keep track of all the users' registeration tokens changes.

How to publish Event Hub events to Event Grid Topic with Azure SDK?

I would like to set an Event Hub to publish events to an Event Grid Topic using Azure SDK.
This can be done in Azure Portal straight from the Event Hub Namespace, creating an Event Grid System Topic.
However, I can't seem to find a proper way using Azure SDK to either create an Event Grid System Topic, or create an Event Grid custom Topic and setting it as an endpoint for the Event Hub Namespace.
Any ideas?
The documentation is available, perhaps a bit scattered. If you navigate to the Event Grid documentation site, the Reference node in the menu tree has all the supported languages/SDKs. For .NET, it will bring you to the article on how to publish and subscribe (link). To manage topics, you would need the management SDK and the samples are available here.
Note that the samples are on the slightly outdated version of the management SDK (related issue) but you should be able to up the versions and use the samples.
Eventually used REST API to make HTTP request for creating an Event Grid System Topic.
Documentation for the API request:
https://learn.microsoft.com/en-us/rest/api/eventgrid/version2020-04-01-preview/systemtopics/createorupdate
Phew !! This issue screwed my mind for 3 days & I had almost lost hope but finally it worked.
Reference:
https://learn.microsoft.com/en-us/java/api/overview/azure/eventgrid?view=azure-java-stable
Code
final String clientId = "clientId";
final String tenantId = "tenantId";
final String clientSecret = "clientSecret";
final String subscriptionId = "subscriptionId";
ApplicationTokenCredentials credentials = new ApplicationTokenCredentials(clientId, tenantId, clientSecret,
AzureEnvironment.AZURE);
credentials.withDefaultSubscriptionId(subscriptionId);
EventGridManager eventGridManager = EventGridManager.configure().authenticate(credentials,
credentials.defaultSubscriptionId());
eventGridManager.eventSubscriptions().define("subscription")
.withScope("/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/microsoft.storage/storageaccounts/{storageAccountName}")
.withEventDeliverySchema(EventDeliverySchema.EVENT_GRID_SCHEMA)
.withDestination(new WebHookEventSubscriptionDestination().withEndpointUrl("{valid https url}"))
.create();
This creates a system topic with a subscription

Azure Identity: Trying to get GetUserDelegationKey to work with an application Service Principal

It's not a good sign when the method I'm asking about, GetUserDelegationKey, yields zero search results on SO. Good luck, me.
I have a C# console app, .Net framework 4.8, using Azure.Storage.Blobs and Azure.Identity that will run on customer servers and access Azure blob storage to hold some stuff. I'm doing all of this with the library, not rolling my own REST. Built with VS2019, testing on Win10.
The plan is to use a single Azure storage account that I own, and create one Container per customer project with per-customer credentials that permit them only their own container. Projects never ever talk to each other.
I could set up credentials in the Azure portal by hand, but I am stubbornly trying to do this in software, where a simple project-management app connects as the project app's service principal (which I defined in Azure AD), creates the container, then creates the shared access signatures with a limited lifetime.
The storage account name / container name / access signature would then be configured on the customer server.
I'm having a terrible time.
Note: this is using the newer BlobClient mechanisms, not the older CloudBlob stuff. Dunno if that matters.
This is all documented here at Microsoft, and following even the simple example gets me the same failure.
using System;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using Azure.Identity;
namespace Azure.Test
{
class Program
{
static void Main(string[] args)
{
var serviceClient = new BlobServiceClient(
new Uri("https://stevestorageacct.blob.core.windows.net"),
new DefaultAzureCredential(true)); // true=pop up login dlg
/*BOOM*/ UserDelegationKey key = serviceClient.GetUserDelegationKey(
DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow.AddDays(30));
// use the key to create the signatures
}
}
}
Even though this program couldn't be simpler, it fails every time with an XML error calling GetUserDelegationKey
Unhandled Exception: Azure.RequestFailedException: The value for one of the XML nodes is not in the correct format.
RequestId:c9b7d324-401e-0127-4a4c-1fe6ce000000
Time:2020-05-01T00:06:21.3544489Z
Status: 400 (The value for one of the XML nodes is not in the correct format.)
ErrorCode: InvalidXmlNodeValue
The XML being sent is supposed to be super simple, I think just the start/end dates for validity, but I have no idea how to get to it to inspect, and http is forbidden for this kind of call, so no Wireshark.
It also fails the same way when I use my application's service principal:
static void Main(string[] args)
{
var tokenCredential = new ClientSecretCredential(
"xxxx-xxxx-xxxx-xxxxx", // tenant ID
"yyyy-yyyy-yyyy-yyyyy, // application ID
"**************"); // client secret
var serviceClient = new BlobServiceClient(
new Uri("https://stevestorageacct.blob.core.windows.net"),
tokenCredential);
UserDelegationKey key = serviceClient.GetUserDelegationKey(
DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow.AddDays(30));
// ALSO: boom
I'm really at a loss.
I suppose I could try rolling my own REST and playing with it that way, but it doesn't feel like this should be necessary: this kind of error feels like a bug even if I'm doing something wrong. XML nodes?
Also open to entirely different ways of approaching this problem if they are superior, but would like to at least find out why this is failing.
I've had some issues with this also. The first things to try is removing the start time (pass null) or setting it ~15 minutes in the past. This is to avoid clock skew between the requesting pc and azure servers.
The second thing to verify is that the user that you are using has the "Storage Blob Data Contributor" role on the storage account. I had to grant it at the storage account level in the end otherwise it just refused to work for me. However in your use case it might be that you need to grant it at the container level to allow you to have one container per client.
Hope this helps.

Azure AD: How to make tokens have the "hasgroups" claim?

Our application allows assigning permission to groups, which means for every user, we have to reliably determine group membership. The user presents a token regularly obtained with ADAL (some use .NET, others use NodeJS, others use CLI).
Some users seem to be sending a token with the following claim:
"hasgroups": true,
That claim is documented in the Azure AD token reference page.
We would like to add a test case for that, but after following steps here and here, we always end up with a token with the following claims:
"_claim_names": {
"groups": "src1"
},
"_claim_sources": {
"src1": {
"endpoint": "https://graph.windows.net/{redacted}/users/{redacted}/getMemberObjects"
}
},
What is wrong with our setup? Why can't we get the hasgroups claim?
Here are some additional information:
Application type is Native (not WebApi).
Manifest says "oauth2AllowImplicitFlow": true.
The application is given access to Azure Key Vault.
We use the following code to get the token (in C#):
var userCredential = new UserCredential( _userName, _password );
result = context.AcquireToken( _resource, _clientId, userCredential );
Where:
_userName and _password are from a user with lots of groups.
_clientId is the application id of the native application - the one with "oauth2AllowImplicitFlow": true.
_resource is https://vault.azure.net.
The token is emitted correctly. The only issue is that it shows _claim_names and _claims_sources instead of hasgroups.
Where: • _userName and _password are from a user with lots of groups.
As the user is part of lots of groups (assuming 6 or more here).. Azure AD token will come back with a groups overage indicator instead of actual group ids in “groups” claim. I guess you know that and hence doing it intentionally.
var userCredential = new UserCredential( _userName, _password );
result = context.AcquireToken( _resource, _clientId, userCredential );
Since you're acquiring the token in a .NET based application using C# code, the token response is not really limited in length (like in cases for a web SPA, where it is being returned as a URI fragment and URL length has limits)
Looking at the documentation both "hasgroups" and "groups:src1" claims have the same intention of telling that there are too many groups to return as part of the token. Although there is a subtle difference:
in cases where URL limit applies, "hasgroups" will be sent as true (like implicit grant flow for SPA)
in cases where length is not limited (like in your case), Azure AD will still not return all the groups to make sure the token doesn't get too big, but it will send a little more information on how to get to all groups by sending the information on how you can query for all your groups. In this case it's sending the "groups:src1" and "_claim_sources" with source information instead of just the "hasgroups"
Claims in id_tokens
For anyone looking more on this. Please refer Doc saml-tokens
Note
Source : Azure Sample Link

Resources