I would like to set an Event Hub to publish events to an Event Grid Topic using Azure SDK.
This can be done in Azure Portal straight from the Event Hub Namespace, creating an Event Grid System Topic.
However, I can't seem to find a proper way using Azure SDK to either create an Event Grid System Topic, or create an Event Grid custom Topic and setting it as an endpoint for the Event Hub Namespace.
Any ideas?
The documentation is available, perhaps a bit scattered. If you navigate to the Event Grid documentation site, the Reference node in the menu tree has all the supported languages/SDKs. For .NET, it will bring you to the article on how to publish and subscribe (link). To manage topics, you would need the management SDK and the samples are available here.
Note that the samples are on the slightly outdated version of the management SDK (related issue) but you should be able to up the versions and use the samples.
Eventually used REST API to make HTTP request for creating an Event Grid System Topic.
Documentation for the API request:
https://learn.microsoft.com/en-us/rest/api/eventgrid/version2020-04-01-preview/systemtopics/createorupdate
Phew !! This issue screwed my mind for 3 days & I had almost lost hope but finally it worked.
Reference:
https://learn.microsoft.com/en-us/java/api/overview/azure/eventgrid?view=azure-java-stable
Code
final String clientId = "clientId";
final String tenantId = "tenantId";
final String clientSecret = "clientSecret";
final String subscriptionId = "subscriptionId";
ApplicationTokenCredentials credentials = new ApplicationTokenCredentials(clientId, tenantId, clientSecret,
AzureEnvironment.AZURE);
credentials.withDefaultSubscriptionId(subscriptionId);
EventGridManager eventGridManager = EventGridManager.configure().authenticate(credentials,
credentials.defaultSubscriptionId());
eventGridManager.eventSubscriptions().define("subscription")
.withScope("/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/microsoft.storage/storageaccounts/{storageAccountName}")
.withEventDeliverySchema(EventDeliverySchema.EVENT_GRID_SCHEMA)
.withDestination(new WebHookEventSubscriptionDestination().withEndpointUrl("{valid https url}"))
.create();
This creates a system topic with a subscription
Related
I'm trying to use the Azure Service Bus api to purge messages from certain queues.
According to the documentation (https://learn.microsoft.com/en-us/rest/api/storageservices/clear-messages) I would do this using the following request:
https://myaccount.queue.core.windows.net/myqueue/messages
This seems simple enough, but the problem is I just cannot get this request to authorize correctly.
(result = 401 Unauthorized)
Apparently the signing process is rather elaborate and described here (https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key)
I had many attempts, like the following code:
var httpClient = new HttpClient();
using (HMACSHA256 hmac = new HMACSHA256())
{
var StringToSign = $"GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-date:{DateTime.UtcNow.ToString("o")}\nx-ms-version:2015-02-21\n\n\n\n";
var stringToSignHash = hmac.ComputeHash(Encoding.UTF8.GetBytes(StringToSign));
var base64StringToSign = Convert.ToBase64String(stringToSignHash);
var signature = $"{base64StringToSign}, {MySecretKey}";
using (HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Delete, "https://myaccount.servicebus.windows.net/myqueue/messages"))
{
//requestMessage.Headers.Add("x-ms-date", DateTime.UtcNow.ToString());
requestMessage.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("SharedKey", signature);
var result = httpClient.SendAsync(requestMessage).GetAwaiter().GetResult();
}
}
For MySecretKey I used my shared access policy key. I tried using as is, and also decoded base64 to ascii.
Does anyone have more success with this?
Are there more simple ways to access the api?
Thanks.
You can leverage the code here in different languages to generate the SAS token.
Updated:
Can you please confirm which API you are trying to execute? As per the code, you are calling the Delete method on myaccount.servicebus.windows.net (i.e. service bus resource). Delete API operation completes the processing of a locked message and deletes it from the queue or subscription. The samples/reference article that you have shared is for the delete messages from the storage queue. If your requirement is to purge all messages from the service bus queue then you need to use Recieve and Delete API. Alternative you can also use service bus explorer to purge the messages.
As an alternative you can use Service Bus Cloud Explorer which runs in the browser with logged-in user Azure account by default and have a purge functionality.
Keep in mind that it will take time to clear up queues or subscriptions. More messages it needs to purge; the more time it would take.
I created one IoT Central app form https://apps.azureiotcentral.com/ in my own subscription for a PoC. Now my customer wants me to move it to their own subscription. Question, is it possible to move the whole app? Or do I need to create a new app and then export/import templates, devices and data?
Try the following Copy feature:
EDIT:
For creating a device instance assigned to the compability model can be used, for instance, the REST API.
The following is an example of the device provisioning using the REST API request for myScopeId, mydevice, deviceKey and CapabilityModelId:
PUT:
https://global.azure-devices-provisioning.net/myScopeId/registrations/mydevice/register?api-version=2019-03-31
headers:
Authorization: sas-token
payload:
{
"registrationId":"mydevice",
"payload":{
"__iot:interfaces":{
"CapabilityModelId":"urn:rigado:Cascade_500:1"
}
}
}
where the sas-token can be generated like is described here:
generateSasToken(string resourceUri, string key, string policyName, int expiryInSeconds = 3600)
where:
resourceUri = "myScopeId/registrations/mydevice"
key = deviceKey
policyName = "registration"
You can migrate your application without having to re-create it by visiting the Azure Portal (portal.azure.com) > search for "IoT Central Application" > Find your application and click on it. Inside your application you'll see the subscription it's currently using, and an option to change it. Follow the steps to migrate your subscription.
Ibiza portal screenshot highlighting where the "change" button is.
Keep in mind that moving your application from one subscription to another will not change where your application or device data is stored. For example, if you picked United States as the location for your application, the data will continue to be in the United States, even if your subscription/resource group is in a different region.
I trying to pull messages from azure service bus queue using Go.Queue topic name,subscription name,service name and shared access key value are the credentials.I'm not getting proper sample code for this.Please help me!!
The Go Cloud Development Kit Pub/Sub API is still a work in progress (it's one of our newer APIs). As of 2019-01-30, there is a pull request out for review that adds support, so stay tuned!
from azure.servicebus.control_client
import ServiceBusService
bus_service=ServiceBusService(service_namespace='<namespace>',
shared_access_key_name='<key_name>',
shared_access_key_value='<acess_key>')
topic_name = "<topic_name>"
subscription_name = "<subscription_name>"
message = bus_service.receive_subscription_message(topic_name,
subscription_name, peek_lock=True)
I am using Azure Functions to build some integrations between various systems. I new requirement is to respond to record updates in Salesforce. Some quick research yielded what seems like a good solution from the Salesforce side. Use Outbound messaging which can send SOAP requests on record modifications.
How to create Salesforce application that will send record to external web service when record created/changed(https://salesforce.stackexchange.com/questions/73425/how-to-create-salesforce-application-that-will-send-record-to-external-web-servi)
The challenge now is to be able create a SOAP listener in Azure Function. I have created basic HTTP Triggers for my other listeners. Is there anything "built-in" to Azure Functions that would allow me to easily consume the incoming SOAP request?
Salesforce has the basics for a solution based on a more traditional web service and an ASMX file but I am not sure if or how that can be applied in Azure Functions. (https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_om_outboundmessaging_listener.htm)
That notification is just a SOAP request that is made over HTTP, so really not too different than a regular HTTP trigger request.
Although you could just treat that as a plain request and parse the contents yourself, Azure Functions does expose the great WebHook support we get from ASP.NET WebHooks, and luckily, there is a Salesforce receiver that significantly simplifies this task.
DISCLAIMER: It's worth noting that although the receiver is technically enabled in Azure Functions, there's no official support for it yet, so you won't find a lot of documentation and help will be limited to what you get on SO and Forums. Official support to this and other receivers will hopefully be coming soon, which means documentation, templates and UI support will become available.
To get started, you need the following:
Create a new function, selecting the GenericWebHook - CSharp template (this works for node as well, but I'll focus on C# here.
Follow the steps outlined on the ASP.NET WebHooks integration with Salesforce post in order to create the outbound message. Here you want to use the Function Url given to you by the portal WITHOUT THE CODE QUERY STRING (having the code there wouldn't hurt, but the receiver does not use that information).
IMPORTANT: Get your Salesforce Organization ID, which will be used for authentication and is located under Administer > Company Profile > Company Information > Salesforce.com Organization ID and back in the Azure Functions portal, open the Keys panel, delete your default function key (not host key) and create a new key, named default (this name is important) using the Organization ID value you got from Salesforce.
Go to Integrate
On the integration page, select Advanced Editor on the upper right (as mentioned, there's no official support, so the UI does not expose this. We're putting our explorer hats on and venturing into a more advanced workflow here :) )
Change the webHookType property value to sfsoap and save the configuration. Your function.json config should look like the following:
function.json:
{
"bindings": [
{
"type": "httpTrigger",
"direction": "in",
"webHookType": "sfsoap",
"name": "req"
},
{
"type": "http",
"direction": "out",
"name": "res"
}
],
"disabled": false
}
Switch to the Develop tab. We're ready to write our code.
This is where the ASP.NET WebHooks receiver shines! It will parse the notification for you, exposing strong typed objects you can work with. All you need to do is modify the method/function signature you get withe template to use the SalesforceNotifications type, making sure you're referencing the required assembly (Microsoft.AspNet.WebHooks.Receivers.Salesforce, which is made available to you, so no need for package reference) and namespace reference (Microsoft.AspNet.WebHooks).
Here is a full sample of a function that will receive the request and log the Organization ID, Action ID, grab the first notification and log all of its properties:
#r "Microsoft.AspNet.WebHooks.Receivers.Salesforce"
using Microsoft.AspNet.WebHooks;
public static void Run(SalesforceNotifications req, TraceWriter log)
{
log.Info($"Salesforce webhook was triggered!");
log.Info(req.OrganizationId);
log.Info(req.ActionId);
var notification = req.Notifications.First();
foreach (var p in notification.Keys)
{
log.Info($"{p} = {notification[p]}");
}
}
This process will be a lot smoother when the receiver is officially supported, but even with the added steps, this still beats having to parse the SOAP messages yourself :)
I hope this helps!
I am using SAS token authentication along with device-ID (or publisher-Id) in my event Hub publisher code. But i see that it is possible to send an event to any partition ID by using "CreatePartitionedSender" client even though I have authenticated using a device-ID. Whereas I do not want two different device-Ids publishing events in same partition. Is it possible that we can add some custom "authorization" code along with the SAS authentication to allow limited partition access to any device.
The idea behind adding authorization to device and partition-Id combination was to accommodate single event-hub for multiple tenants. Please advise if I am missing anything.
Please see below the code snippet for publisher:
var publisherId = "1d8480fd-d1e7-48f9-9aa3-6e627bd38bae";
string token = SharedAccessSignatureTokenProvider.GetPublisherSharedAccessSignature(
new Uri("sb://anyhub-ns.servicebus.windows.net/"),
eventHubName, publisherId, "send",
sasKey,
new TimeSpan(0, 5, 0));
var factory = MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb", "anyhub-ns", ""), new MessagingFactorySettings
{
TokenProvider = TokenProvider.CreateSharedAccessSignatureTokenProvider(token),
TransportType = TransportType.Amqp
});
var client = factory.CreateEventHubClient(String.Format("{0}/publishers/{1}", eventHubName, publisherId));
var message = "Event message for publisher: " + publisherId;
Console.WriteLine(message);
var eventData = new EventData(Encoding.UTF8.GetBytes(message));
await client.SendAsync(eventData);
await client.CreatePartitionedSender("5").SendAsync(eventData);
await client.CreatePartitionedSender("6").SendAsync(eventData);
I notice in your example code that you have
var connStr = ServiceBusConnectionStringBuilder.CreateUsingSharedAde...
and then have
CreateFromConnectionString(connectionString
This suggests that you may have used a Connection String containing the send key you used to generate the token rather than the limited access token. In my own tests I did not manage to connect to an EventHub using the EventHubClient, which makes an AMQP connection, with a publisher specific token. This doesn't mean it's not supported just that I got errors that made sense, and the ability to do so doesn't appear to be documented.
What is documented and has an example is making the publisher specific tokens and sending events to the EventHub using the HTTP interface. If you examine the SAS token generated you can see that the token grants access to
[namespace].servicebus.windows.net/[eventhubname]/publishers/[publisherId]
This is consistent with the documentation on the security model, and the general discussion of publisher policies in the overview. I would expect the guarantee on publisherId -> PartitionKey to hold with this interface. Thus each publisherId would have its events end up in a consistent partition.
This may be less than ideal for your multitenant system, but the code to send messages is arguably simpler and is a better match for the intended use case of per device keys. As discussed in this question you would need to do something rather dirty to get each publisher their own partition, and you would be outside the designed use cases.
Cross linking questions can be useful.
For a complete explanation on Event Hubs publisher policy refer this blog.
In short, If you want publisher policy - you will not get partitioned sender. Publisher policy is an extension to SAS security model, designed to support very high number of senders ( to a scale of million senders on event hub ).
With its current authentication model, you can not grant so fine-grained access to publishers. Authentication per partition is not currently supported as per Event Hubs Authentication and Security Model Overview.
You have to either "trust" your publishers, or think on different tenant scheme - i.e. Event Hub per tenant.