I am developing a webhook for dm.version.added as documented here. According to the docs, the scope has to be 'folder' which allows writing hooks on the project level (i.e., monitoring one project per hook). Will it be possible to write the webhook for hubs level (i.e., monitoring multiple projects with the same webhook)?
Webhooks are for folders, as you mentioned, so you can set up a dm.version.added for the top most folder (Project Files) of each project in the hub. Unfortunately, you cannot set up a hub-level webhook.
Related
I have tried to find out how I can notify multiple microservice (MS) instances via eventdriven approach if their Azure App Configuration values are changed. I have found that I via Eventgrid can listen on changes in Azure App Configuration but I don't find any built in method to distribute event to multiple instances (many MS instances)... I can choose webhook but then it will be to one instance, I can choose other eventdriven approaches as Event Hub but then I have to setup that and I wonder what is best practice for this? I don't want each MS to poll for changes rather to be notified and receive what has been changed or is there a better built in approach/strategy?
For push-based configuration updates from Azure App Configuration, the suggested approach is to forward events to a Service Bus Topic. Azure Service Bus SDK provides RegisterMessageHandler method that allows clients to register a message handler that would be triggered for every message recieved in the topic. Each instance of the microservice could set up a subscription to this Service Bus Topic and register a message handler during service initialization to receive configuration updates.
The instructions to set up a service bus topic can be found here. Details on the protocols available to subscribe to service bus topics and the required firewall configuration can be found here. Since a single topic can support up to 2000 subscriptions, this approach would allow up to 2000 service instances.
I'm building automation for a Nest thermostat. The automation infrastructure is attempting to use GCP's Pub/Sub and Cloud Functions services.
When creating a new project on Google's Device Access Console I don't see a way to update the Pub/Sub topic. Technically, there is an edit button, but the topic text field is greyed out and the string can't be changed. Nor do I see a way to access the auto-populated topic from inside my GCP project. As a result, I don't see how I can build a Function subscriber to the topic from my GCP project.
Interestingly, there is a way to create a Pub/Sub subscriber because that interface provides a method to manually enter the Pub/Sub topic that is shown on the Device Access Console. I've done this and verified that device data flows correctly.
There is no manual entry option when creating a Function that subscribes to a topic.
When I click on the topic listed on the Pub/Sub Subscriber, I'm presented with an error message,
You do not have sufficient permissions to view this page
How can I build a Cloud Function that responds to Pub/Sub events for the device?
I am exploring the Azure IoT central rest APIs to create a custom Angular client. Is this possible or does it have any limitations? IoT Central is attractive due to its pricing. Specifically, I found that retrieving multiple telemetry values isn't possible as per the following documentation page. Which means you have to send individual "get" requests to fetch multiple telemetry.
Azure IoT Central (get telemetry value)
Is there a possibility to register a call back and receive regular updates of the values like in event hubs? Basically I want to develop a custom client facing app with the IoT Central Pricing. Is it possible?
It is possible, to receive regular updates on telemetry you can use continuous data export. You can export to Service Bus, Event Hubs and Blob Storage. See the documentation here on how to set that up. You can receive those events in any JavaScript application.
Please be aware that continuous data export will give you updates from all devices. If you need to filter them out you will need to build something to filter that out. One example I have built in the past is a .NET Core application that listens to the messages and sends that to the different clients over SignalR.
My Azure based SaaS system publishes events and I have customers who wish to subscribe to them - webhooks seem undeniably the right architecture (And I'm currently a happy consumer of webhooks). I've found lots of great documentation and case studies on best practices (e.g. http://resthooks.org) however I've not managed to find an existing architecture, framework, project, sample or solution that implements the best practices.
I could build my own solution however I don't want to reinvent the wheel. I was expecting to find an existing framework (e.g. on Github) created by people much smarter than I but haven't had any success.
I currently use a number of Azure services (such as Service Bus, Cosmos, Table Storage) internally and consume using Azure Functions but what I don't have is an architecture for allowing my customers to subscribe to these events.
Specifically I'm looking for best practices and code samples on how to manage potentially millions of subscribers (who are external customers) and the approach to distribute the webhooks out to each of them.
I already understand how to publish and consume webhooks where I am an individual subscriber and there are already some great samples available - https://github.com/aspnet/AspLabs/tree/master/src/WebHooks
Can anyone point me in the right direction? (Preferably to a .NET / C# based solution)
Not sure if this is the 'right' direction but here are my current thoughts on a solution.
We are currently using CosmosDb and are leveraging the change feed to trigger an Azure function execution. The code within the function does a specific task for all tenants in our system. This code will be changed to simply send a new event to the Event Grid topic. A 'in-house' subscription will then be added that will handle what the function code is currently doing today.
We will then follow the subscription management guidance Zapier offers. In a nutshell it is to expose the capability to our customers to subscribe to the events that we publish via a few endpoints. In addition to standard CRUD stuff when a tenant adds/removes a subscription the code will leverage the Event Grid Management SDK to add/remove subscriptions to the appropriate topics within Event Grid (samples here). The subscriptions that get added will have filters set to ensure each tenant only receives their own events.
There are limitations to the number of subscriptions and topics with in Azure (details here). These limitations are acceptable in our case but is something you might need to look into more if you need to reach 1mm subscribers.
Here is how I visualize it:
Not 100% we'll build this but if we do I'll post back here any gotchas we uncover.
Cheers!
What is the best practices around Azure EventGrid topics and Events?
Is it a bad idea to publish different event-types to the same Azure EventGrid topic?
e.g. multiple different domain events
When do we need different topics?
A single shared Topic for an entire application?
One Topic per Aggregate Root type?
One Topic per Event Type?
Any suggestions would be welcome as there are no clear answers
Part2.
What if I want to integrate with various Azure Logic Apps?
if multiple logic apps react to the same topic, would they steal messages from each-other?
Do each logic app create some invisible subscription?
No, it is not a bad idea to publish different event-types to the same Azure EventGrid topic: if the events are related to the same resource, it does make sense to publish them to the same EventGrid topic. Taking the example of a HR application, you can have EmployeeAdded and EmployeeRemoved events published on the same "employee" topic.
On the question on when different topics would be needed, I think it depends on a few factors such as how you are modelling the resources in your application, the events of interest on those resources, security model around which parts of the system should be able to publish to the topic / creation of event subscriptions on the topic. Ideally, all types of events for the same resource type (such as the "employee" resource type in the example above) can be on the same topic. When your system has more types of resources, you may want to create separate topics for each of them. Also, the desired security model has to be taken into account as well (e.g. let's say you want to restrict access to who can receive certain types of events).
Regarding the question about logic apps, if you create multiple logic apps that handle events from the same topic, each of them creates an event subscription on the same topic, and Event Grid would deliver the events on that topic to each of the event subscriptions. So, each logic app would receive the same event individually, and can process it independent of the other apps.
The Azure Event Grid (AEG) is not a generic Pub/Sub model. This model is based on the source of the events, where each event source (topicType) is handling own interest.
Subscriber subscribes an interest to the event source (topic) using a subscription. Note, that the AEG allows to subscribe only one topic in the subscription. There is a limit 500 subscriptions per topic.
In other words, if there is a multiple interest for event source (topic) by the same subscriber, this model requires to create a multiple subscriptions (one per topic) per subscriber. The filtering of the interest is possible only within the same topic.
The source of events in the AEG can be extended by custom topics (maximum 100 per Azure Subscription).
Based on the above, I do recommend for custom topics use the same model like is built-in for azure event sources (topicTypes) with a multiple eventTypes, which can be simplified a continuously deployment over environments.
To the Part2: The AEG doesn't use an 'invisible' subscription as a part of the integration. Every subscription created to the topic is visible and accessable, for example using a REST API
Update:
Azure Event Grid recently release (in preview - version 2018-09-15-preview) a new feature which can be helped for your solution using an Event Domain and Domain Topics, more details here.
You can use an updated tool Azure Event Grid Tester for testing all new preview release features, which they are not yet implemented in the Azure portal UI.