I have an Micro-Service which is a Consumer/Subscriber service, that receives messages from other 2 Services.
So, I want this Micro-Service to Subscribe from the other 2 micro-services that have different Topic and Subscription.
The constructing/Initializing the service/Azure Service Bus Properties is as below :
services.AddServiceBusNotifications(Configuration, "Communication");
services.AddServiceBusNotifications(Configuration, "SSO");
And the code for Subscribing to this 2 services is as below:
var eventBus = applicationBuilder.ApplicationServices.GetRequiredService<IEventBus>();
eventBus.Subscribe<MeetinginviteCreatedEvent, MeetinginviteCreatedEventHandler>();
eventBus.Subscribe<NewUserCreatedEvent, NewUserCreatedEventHandler>();
So, both the above 2 lines are taking the settings(Topic/Subscription) from the same topic & Subscriber, but I want to take the Topic and Subscription for 2 different Microservices.
How Can I use the Same Subscriber for Multiple Topic and Subscription(Different Microservices).
I have followed the code from the URL : https://github.com/pmchlk/service-bus-messaging
There is a lot going in the code that you've shared since you have a wrapper around the Azure Service Bus SDK but if my understanding of it is correct, one way to solve your issue is to simply forward messages from the individual subscriptions into another topic/subscription and use that in your service.
This would be simpler if it works like I expect it to, based on my understanding of your code.
The other options would be to either change your Dependency Injection code to register transient Event Bus objects instead of a singleton or update your Event Bus class to handle multiple subscriptions. Both of these would require quite a few changes in your custom implementation.
Related
Summary
I'm trying to reason about a few integration patterns around using Azure Service Bus and Event Grid in the context of a microservices-based architecture. I like service bus for the transaction support, load leveling, sessions and so on. Just looking for feedback on the way I'm reasoning about them being used together for internal microservice communication and also provide events to external partners via webhooks.
Option 1
Say I already use service bus for microservices communication, but I would also like to publish to event grid to support publishing via webhooks to partner services outside my trust boundary. I'm thinking to have an Azure function (or similar) to receive my custom service bus topic events and the publish them to Event Grid.
This would also allow for the webhook event to be different than the internal events.
Is there any other option (e.g., some service bus premium feature) that could provide that integration out-of-the-box (ootb)? I'm only aware of service bus system events being pushed to Event Grid ootb.
Option 2
Have your microservices publish events only to Event Grid and then have Event Grid events pushed to your service bus topic. It would allow the webhook option in certain cases without much extra, but it seems odd for all my microservices to publish to Event Grid if I need to them have that generally pushed to service bus for load leveling and so on.
I would rather use service bus for my microservice communication. There are other drawbacks as well. It would only seem to make the webhook option a little more straightforward.
Option 3
Have the event publisher that wants to send webhook-related events publish to both Event Grid and service bus. This seems like a really bad idea and generally clumsy, even if you were able to abstract it away somewhat in code. It seems like it would be a worse version of option 1.
I have 3 Azure service bus topics, 3 azure app services subscribe to these topic messages and perform some actions and return results.
I want to push a message to another topic with the collective results (failure/success) from the above 3 app services.
How can this be achieved?
The sample flow chart below:
One of the options to solve this is to leverage the Azure Service Bus Message Sessions feature. It allows handling messages and keeping the state using a single service. You'd have 3 message types (events) published by the upstream services (Payment, Email, and Notification), and a fourth message type, a timeout. This is also known as a saga pattern. I've described the implementation with Azure Service Bus in detail in this blog post.
I have a single client application that will publish the message to a single location/target, and I need that message to then end up in 3 separate other queues as well ( and subsequently processed)
Basically here is the use case:
A website collects customers information in a lead form. That lead information is pushed to an restful web api. The restfull web api then publishes a message to a single location and then returns a success result to client. In the background, the message ends up on 4 queues, and ultimately sent to 4 different other web services (some external, some internal)
The system needs to be performant with respect to how quickly all 4 of the queues are processed from the 4 queues. But the volume of leads is not necessarily extremely high. (perhaps a few hundred leads per day)
Here is an image of what I am thinking
You could use a Topic in combination with the autoforwarding feature.
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-auto-forwarding
A single publish to a topic could then be setup to auto-forward to 4 separate queues.
Instead of uses Queues, you should use Topics:
a queue is often used for point-to-point communication, topics are useful in publish/subscribe scenarios.
Topics and subscriptions provide a "one-to-many" form of communication, using a publish/subscribe pattern.
If you really need to use a Queue there is no other way than to send copies of the message to the different queues.
One best solution to solve your business scenario, is using a Service Bus Topic with four Topic Subscriptions. You can send the message to the topic. You can create filters (or) Topic subscription rules to filter the messages received by the Service Bus Topic.
You can then set the auto-forward property of each topic subscription to the desired Service Bus Queue.
I have scenario that I want to read all messages from all topics a and a specific subscription using azure logic app connector.
Example: topic ATopic, BTopic has a same name "TestSubscription" subscription
Using azure logic app I want to read all topics messages on above "TestSubscription"
I have mark in read in below image at particular place I want a dynamic topic name
The common pattern for scenarios like this is to have a Logic App for each Trigger/Topic/Subscription.
Then these Logic Apps, two in your case, call a third work Logic App which does any real message processing.
You can use the Event Grid trigger if you have Service Bus premium, to subscribe to Microsoft.ServiceBus.ActiveMessagesAvailableWithNoListeners event. Once the Logic App is triggered, you can then use the Service Bus action to retrieve the actual message and process it.
I'm looking into using Azure Service Bus for publishing/delivering messages between different services in our application, and are trying to find a decent design for the topic and subscriptions within a Service Bus namespace.
The intent of the system is for service-a to publish a message with type service-a.test-event to a bus, and have any service listening on that event type get the message delivered. It will be a fairly low volume
I'm a bit torn on which of the following designs to use:
The Service Bus namespace has one topic events where all messages from all services are delivered. Any service subscribing to events from any other service create a subscription in this topic using filters to get the message types they want - one subscription per message type (eg. service-b-service-a-test-event).
The Service Bus namespace has one topic per publisher (eg. events-service-a). Any subscribing service to events from this service create a subscription in the topic using filters to get the message types they want - one subscription per message type (eg. service-b-test-event).
Service Bus seems to have a limit of 2000 subscriptions per topic, which as far as I can tell will be sufficient for us. If I had suspicions otherwise, option #2 would probably be the best choice (as I can have 10.000 topics per namespace). None of the other limitations of Service Bus, as far as I can tell, really impacts which of these options I should go for.
One additional requirement is that I want to have a service subscribing to any event from any service for recording reasons. If I went for option #1, that'd be very simple to implement. For option #2 however, that service needs to somehow make sure it has a subscription within any event topic in the namespace - and reconfigure itself once new topics are added and old topics are removed. That's outside the scope of this question, but a requirement for the design none the less.
When deciding about a topology, take into consideration things that matter to you the most. One of the principles of pub/sub is to decouple between publishers and subscribers. With topology #2 (topic per publisher) this principle is violates since every subscriber would have to know the name of the publisher. And if publisher of an event would change, all your subscribers would have to get that knowledge in order to re-subscribe. Topology #1 solves that problem by abstracting publishers away by using a single topic.
Also, the 2nd requirement you had (to audit all of the events) would be much easier to implement with this topology as you only have to maintain a single wiretap subscription on that topic.
You're correct about limitation of 2000 subscriptions per topic. Saying that, 2000 subscriptions is quite a lot. Once your system get to that number of subscribers, you'd probably want to review your architecture/topology anyway.
These two topologies are sound extremely similar to the NServiceBus on Azure Service Bus as a transport topologies. You could have a look to see some of the other ideas you could use for your topology, including benefits of one over another.
Disclaimer: I am working on NServiceBus Azure Service Bus transport. You don't have to use NServiceBus, though a lot of questions like the one you're posting here become a non-issue when you do.