Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I have a specific case that I would like to solve using an Azure Messaging Service, but I'm not sure which one to use. There are 7 to choose from and I think I have narrowed it down to 2 options.
Azure Service Bus Topic
Azure Event Hubs
I will try to explain my needs using the following diagram. Please bare in mind that this is just a fictional scenario, just to illustrate what I'm after.
A user updates a Product. An HTTP request is send to an MVC application.
The MVC applications puts a UpdateProductCommandon the bus (what Azure Bus?)
Inside Azure, wether it is an Azure Function, or something else, the command must be processed.
Inside the command handler I want to publish an Event telling all listening parties that a certain task has been processed.
So an ProductUpdatedEvent gets published
There will be more than 1 application interested in consuming this event. These apps can live inside Azure as an Azure Function. But it must also be possible for applications to consume events that are hosted on an external server. Inside my own IIS server for example.
Requirements:
When an application/eventhandler that consumes an event goes down for x time, and later gets up again, then it should be able to process all the events it missed.
An event can be consumed by more than one event handler. All event handlers should process the event.
An event must be able to carry data. Like on the right in the screenshot. I want to be able to send data about the product that got updated.
Which Azure Messaging Service technology would best fit this description?
I would recommend Azure Service Bus.
Clients can be responsible for creating their own subscriptions and get their own instance of the message you send. So in the final part of your diagram the Product Microservice/Inventory Microservice would each be able to process the message at their own rate and if one went down it wouldn't affect the other, they would both be able to read their own message. It gets sent once and read twice.
You can have a look at the tiers of Service Bus to see if the cost/storage meets your needs but you should be able to store millions and millions (80Gb) of messages on the topic depending on what you put in the message. Each message can be up to 1Mb of text so your number will differ depending on what you're doing. Then the microservices comes back online it can work through the backlog.
My 2 cents on this, as both can be used to achieve the same, and some architects would decide for one, the others for the other:
I believe you have read the documentation which starts from the distinction between the message and the event, and the philosophy behind.
"The publisher of the message has an expectation about how the consumer handles the message"
"The publisher of the event has no expectation about how the event is handled"
From this, and based on your requirements, I understood that your publisher does not care how the handler will handle the event - for the publisher it does not represent the value as handlers should not send any response/confirmation. This goes more in the Event-Hub 'spirit'
Now, you said that you want to carry the data. By the definition, again from the documentation:
"An event is a lightweight notification of a condition or a state change"
"The message contains the data that triggered the message pipeline."
So, 'carrying a data' is more a message, than the event, as the message contains the information, and the event contains the fact that the state has changed. This goes more in the Service bus topic 'spirit' and philosophy, as the message contains high-value transactional data that should not be lost.
Another requirement, and in my opinion the most important one, says that you would like to have more than one event handler.
Now, the way to have more than one event-handler with Event-Hub is to create for each one of them a separate Consumer Group.
The way how you can have more than one event-handler with the Service-Bus topic is that you just subscribe.
So finally, my 2 cents, if you want more to have more flexibility with handlers, I would go with Service Bus topics, as you can add as many as you want subscribers(event handlers) in the runtime as you want, without any adjustment on Service Bus topic itself,
If you think that your solution will go more in the direction that you will have a finite number of handlers/consumers, and you might have concurrent event publishers, I would then choose the Event-Hub here, and I would just create for each of my handlers a Consumer group - which is not what I understood from your initial requirements.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
We have a microservices architecture to support a big application. All the services communicate using azure service bus as a medium. Currently, we are sending the notifications(immediate/scheduled) from different services on per need basis. Here comes the need for a separate notifications service that could take that load and responsibility of formatting and sending notifications(email, text etc).
What I have thought:
Notification service will have its own database which will have data related to notifications(setup, templates, schedules etc) and also some master data(copied from other sources). I don't want to copy all the transactional data to this DB(for abvious reasons) but we might need transactional and historic data to form a notification. I am planning to subscribe to service bus events (published by other services) and onus of sending the data needed for formatting the notification will be on service raising the service bus event. Notification service will rely on that data to fill up the template(stored in ots own DB) and then send the notification.
Job of notifications service will be to listen to service bus events and then fill up the template from data in event and then send the notification.
Questions:
What if the data received by notification service from service bus event does not have all necessary data needed in notification template. How do I query/get the missing data from other service.?
Suppose a service publishes 100 events for a single operation and we need to send single notification that that whole operation. How does the notification service manage that since it will get 100 different messages separately.?
Since the notification trigger depends on data sent from other sources(service bus event), what happens when we have a notification which is scheduled(lets say 6am everyday). How do we get the data needed for notification(since data is not there in notification DB)?
I am looking for some experience advice and some material to refer. Thanks in advance.
You might have to implement a notification as a service which means, imagine you are exporting your application as a plugin in Azure itself. few points here.....
your notification will only accept when it is valid information,
Have a caching system both front end(State management) and backend, microservices(Redis or any caching system)
Capture EventId on each operation, it's a good practice we track the complex operation of our application in this way you can solve duplicate notification, take care that if possible avoid such type of notifications to the user, or try to send one notification convening a group of notifications in one message,
3.Put a circuit breaker logic here to handle your invalid notification, put this type of notification in the retry queue of 30mins maybe? and republish the event again
References
https://www.rabbitmq.com/dlx.html
https://microservices.io/patterns/reliability/circuit-breaker.html
https://redis.io/topics/introduction
Happy coding :)
In microservice and domain driven design it's sometimes hard to work out when to start splitting services. Having each service be responsible for construction and sending its own notifications is perfectly valid.
It is when there is a need to have additional decisions be made, that are not related to the 'origin' service, where things become more tricky.
EG. 1
You have an order microservice that sends an email to the sales team and the user when an order is placed.
Then the payment service updates sales and the user with an sms message when the payment is processed.
You could then decide you and the user to manage their notification preferences. They can now decide if they want sms / email / push message, and which messages they would like to receive.
We now have a problem. These notification prefrences would need to be understood by every service sending messages. Any new team or service that starts sending messages needs to also remember to implement these preferences.
You may also want the user to view all historic messages they have been sent. Again you get into a problem where there is no single source for that information.
EG 2
We now have notification service, it is listening for order created, order updated, order completed events and payment processed events.
It is listing for:
Order Created
Order Updated
Only to make sure it has the information it needs to construct the messages. It is common and in a lot of requirements to have system wide redundancy of data when using microservices. You need to imagine that each service is an island, so while it feels wasteful to store that information again, if it is required that service to perform is work then it is valid.
Note: don't store the data wholesale, store only what is relevant for that service.
We can then use the:
Order Complete
Payment Processed
events as triggers to actually start constructing and sending the messages.
Problems:
Understanding if the service has all the required data
This is up to the service to determine. If the Order Complete event comes through, but it has not yet received an order created event, then the service should store the order complete event and try to process again in the future when all the information is available.
100 events resulting in a notification
Data aggregation is also an important microservice concept, and there are many ways to ensure completeness that will come down to your specific use case.
Ok so i'm relatively new to the servicebus. Working on a project where we use Azure servicebus for queueing messages. Our architecture roughly looks like the following:
So the idea is that in our SourceSystem all kinds of stuff happens, which leads to messages being put on the servicebustopics. Now our responsibility is syncing these events to the external client so they are aware of what we are doing.
Now the issue is that currently we dont use servicebus sessions so message order isnt guaranteed. Also consider the following scenario:
OrderCreated
OrderUpdate 1
OrderUpdate 2
OrderClosed
What happens now is if the externalclients API is down for say OrderUpdate 1 and OrderUpdate 2, we could potentially send the messages in order: OrderCreated, OrderClosed, OrderUpdate 1, OrderUpdate 2.
Currently we just retry a message a few times and then it moves into the deadletter queue for manual reprocessing.
What steps should we take to better guarantee message order? I feel like in the scope of an order, message order needs to be guaranteed.
Should we force the sourcesystem to put all messages for a order in a servicebus session? But how can we handle this with multiple topics? And what do we do if message 1 from a session ends up in the deadletter?
There are a lot of considerations here, should we use a single topic so its easier to manage the sessions? But this opens up other problems with different message structures being in a single topic?
Id love to hear your opinions on this
Have a look at Durable Functions in Azure. You can use the 'Async Http API' or one of the other patterns to achieve the orchestration you need to do.
NServicebus' Sagas might also be a good option, here is an article that does a very good comparison between NServicebus and Durable Functions.
If the external client has to receive all those events and order matters, sending those messages to multiple topics where a topic is per message type will make your mission extremely hard to accomplish. For ordered messaging first you need to use a single entity (queue or topic) with Sessions enabled. That way you can guarantee ordered message processing. In case you have multiple external clients, you'd need to have a session-enabled entity (topic) per external client.
Another option is to implement a pattern known as Process Manager. The process manager would be responsible to make the decisions about the incoming messages and conclude when the work for a given order is completed or not.
There are also libraries (MassTransit, NServiceBus, etc) that can help you. NServiceBus implements Process Manager via a feature called Saga (tutorial) and MassTransit has it as well (documentation).
Sorry if this is long and/or open ended... there is just so many options in azure I'm struggling to choose which pieces to use where, so Im hoping someone can help point the way.
I am trying to build something as described in the image below. I am kind of "routing" messages from a multi-tenant service bus topic (top of diagram) to tenant specific REST endpoints (bottom of diagram).
The hardest part that I am facing for far is to have kind of a concept of a function with dynamic queue/event triggers that change over time, but I'm looking for advice on any parts of the solution.
I have 4 constraints to deal with:
Those messages HAVE to eventually reach the corresponding rest endpoint
Those REST endpoint nodes get added and removed dynamically, but I do know (via event grid message) when that happens.
Those REST endpoint nodes can also be taken offline anytime, or for long periods of time (days), and all messages should eventually be delivered
I have a mostly working solution for F1, but looking for better ideas, but really its F2 that I am struggling with.
F1
Currently what I have is:
1 queue per tenant server, created/deleted based on event grid message
An Azure Function that can return a queue name based on service bus message contents
A logic app (WIP) that will take messages off the topic subscription, use the function to determine the destination queue name, add the webhook URI to the message properties, and forward the message to that queue.
I think that F1 will eventually work correctly :D... but f2..
F2
This is the tricky part, now I have N queues, that come and go over time. I can't figure out how to listen to all the queues via 1 function, so im thinking of trying to maybe rollout 1 function again per queue. It would be responsible for pulling messages off the queue and sending to the REST webhook URI.
But then when the rest endpoint is down it should pause, and not drain the queue, also not sure how to do this efficiently, maybe another logic app with polling?
I'm thinking of maybe using event grid instead of queues, because I think they have more serverless support in general, but I'm not sure this will solve the problem either.
Appreciate any thoughts
If there are no longer any publishers or subscribers reading nor writing to a Queue, Topic, or Subscription, because of crashes or other abnormal terminations (instance restart, etc.), is that Queue/Topic/Subscription effectively orphaned?
I tested this by creating a few Queues, and then terminating the applications. Those Queues were still on the Service Bus a long time later. It seems that they will just stay there forever. That would be wonderful if we WANTED that behavior, but in this case, we do not.
How can we detect and delete these Queues, Topics, and Subscriptions? They will count towards Azure limits, etc, and we cannot have these orphaned processes every time an instance is restarted/patched/crashes.
If it helps make the question clearer, this is a unique situation in which the Queues/Topics/Subscriptions have special names, or special Filters, and a very limited set of publishers (1) and subscribers (1) for a limited time. This is not a case where we want survivability. These are instance-specific response channels. Whether we use Queues or Subscriptions is immaterial. If the instance is gone, so is the need for that Queue (or Subscription).
This is part of a solution where each web role has a dedicated response channel that it monitors. At any time, this web role may have dozens of requests pending via other messaging channels (Queues/Topics), and it is waiting for the answers on multiple threads. We need the response to come back to the thread that placed the message, so that the web role can respond to the caller. It is no good in this situation to simply have a Subscription based on the machine, because it will be receiving messages for other threads. We need each publishing thread to establish a dedicated response channel, so that the only thing on that channel is the response for that thread.
Even if we use Subscriptions (with some kind of instance-related filter) to do a long-polling receive operation on the Subscription, if the web role instance dies, that Subscription will be orphaned, correct?
This question can be boiled down like so:
If there are no more publishers or subscribers to a Queue/Topic/Subscription, then that service is effectively orphaned. How can those orphans be detected and cleaned up?
In this scenario you are looking for the Queue/Subscriptions to be "dynamic" in nature. They would be created and removed based on use as opposed to the current explicit provisioning model for these entities. Service Bus provides you with the APIs to perform create/delete operations so you can plug these on role OnStart/OnStop events appropriately. If those operations fail for some reason then the orphaned entities will exist. Again you can run clean up operation on them based on some unique identifier for the name of the entities. An example of this can be seen here: http://windowsazurecat.com/2011/08/how-to-simplify-scale-inter-role-communication-using-windows-azure-service-bus/
In the near future we will add more metadata and query capabilities to Queues/Topics/Subscriptions so you can see when they were last accessed and make cleanup decisions.
Service Bus Queues are built using the “brokered messaging” infrastructure designed to integrate applications or application components that may span multiple communication protocols, data contracts, trust domains, and/or network environments. The allows for a mechanism to communicate reliably with durable messaging.
If a client (publisher) sends a message to a service bus queue and then crashes the message will be stored on the Queue until as consumer reads the message off the queue. Also if your consumer dies and restarts it will just poll the queue and pick up any work that is waiting for it (You can scale out and have multiple consumers reading from queue to increase throughput), Service Bus Queues allow you to decouple your applications via durable cloud gateway analogous to MSMQ on-premises (or other queuing technology).
What I'm really trying to say is that you won't get an orphaned queue, you might get poisoned messages that you will need to handled, this blog post gives some very detailed information re: Service Bus Queues and their Capacity and Quotas which might give you a better understanding http://msdn.microsoft.com/en-us/library/windowsazure/hh767287.aspx
Re: Queue Management, you can do this via Visual Studio (1.7 SDK & Tools) or there is an excellent tool called Service Bus Explorer that will make your life easier for queue managagment: http://code.msdn.microsoft.com/windowsazure/Service-Bus-Explorer-f2abca5a
*Note the default maximum number of queues is 10,000 (per service namespace, this can be increased via a support call)
As Abhishek Lai mentioned there is no orphan detecting capability supported.
Orphan detection can be implement externally in multiple ways.
For example, whenever you send/receive a message, update a timestamp in an SQL database to indicate that the queue/tropic/subscription is still active. This timestamp can then be used to determine orphans.
If your process will crash which is very much possible there will be issue with the message delivery within the queue however queue will still be available to process your request. Handling Application Crashes and Unreadable Messages with Windows Azure Service Bus queues are described here:
The Service Bus provides functionality to help you gracefully recover from errors in your application or difficulties processing a message. If a receiver application is unable to process the message for some reason, then it can call the Abandon method on the received message (instead of the Complete method). This will cause the Service Bus to unlock the message within the queue and make it available to be received again, either by the same consuming application or by another consuming application.
In the event that the application crashes after processing the message but before the Complete request is issued, then the message will be redelivered to the application when it restarts. This is often called At Least Once Processing, that is, each message will be processed at least once but in certain situations the same message may be redelivered. If the scenario cannot tolerate duplicate processing, then application developers should add additional logic to their application to handle duplicate message delivery. This is often achieved using the MessageId property of the message, which will remain constant across delivery attempts.
If there are no longer any processes reading nor writing to a queue, because of crashes or other abnormal terminations (instance restart, etc.), is that queue effectively orphaned?
No the queue is in place to allow communication to occur via Brokered Messages, if all your apps die for some reason then the queue still exists and will be there when they become alive again, it's the communication channel for loosely decoupled applications. Regards Billing 'Messages are charged based on the number of messages sent to, or delivered by, the Service Bus during the billing month' you won't be charged if a queue exists but nobody is using it.
I tested this by creating a few queues, and then terminating the
applications. Those queues were still on the machine a long time
later.
The whole point of the queue is to guarantee message delivery of loosely decoupled applications. Think of the queue as an entity or application in its own right with high availability (SLA) as its hosted in Azure, your producer/consumers can die/restart and the queue will be active in Azure. *Note I got a bit confused with your wording re: "still on the machine a long time later", the queue doesn't actually live on your machine, it sits up in Azure in a designated service bus namespace. You can view and managed the queues via the tools I pointed out in the previous answer.
How can we detect and delete these queues, as they will count towards
Azure limits, etc.
As stated above the default maximum number of queues is 10,000 (per service namespace, this can be increased via a support call), queue management can be done via the tools stated in the other answer. You should only be looking to delete queue's when you no longer have producer/consumers looking to write to them (i.e. never again). You can of course create and delete queues in your producer/consumer applications via the namespaceManager.QueueExists, more information here How to Use Service Bus Queues
If it helps make the question clearer, this is a unique situation in which the queues have special names, and a very limited set of publishers (1) and subscribers (1) for a limited time.
It sounds like you need to use Topics & Subscriptions How to Use Service Bus Topics/Subscriptions, this link also has a section on 'How to Delete Topics and Subscriptions' If you have a very limited lifetime then you could handle topic creation/deletion in your app's otherwise you could have have a separate Queue/Topic/Subscription setup/deletion script to handle this logic...
We are thinking of speparate Queues for:
Request (RequestQueue)
Response (ResponseQueue)
Scenario:
Worker role will putMessage to RequestQueue e.g. GetOrders
Third party will monitor RequestQueue. If they see GetOrders
request they will getMessage, process them and put the response in
ResponseQueue.
Question:
If I putMessage to RequestQueue, I will like to get results back from ResponseQueue. Is there easy way to achieve this and how?
Thank you.
No, this is not possible. If you put a message in a queue, you must pop the message from the same queue (it will not magically appear in any other queue). Perhaps if you explained more why you think you need two separate queues here for push/pop, there might be a more expansive answer and suggestion.
EDIT: Perhaps I misunderstood your intent. I guess I don't get the question now - can you help clarify. You seem to be asking how to put a message on one queue, acknowledge it by putting another message on another queue, and have someone read the acknowledgment from the second queue? What is the question here? I should point out that you won't want some 3rd party to read directly from a Windows Azure queue as that would require sharing the master storage key with them (a non-starter). Perhaps you are looking for how to have 3rd parties read from a queue?
EDIT 2: Sounds like you want to consume messages with a 3rd Party. Windows Azure queues probably are not a good fit as I mentioned due to security reasons (you need to share the master key). Instead, you could either layer a WCF service over the queue (using queues via proxy) or use the queueing from the Service Bus - that will allow you to have separate credentials. Using the Service Bus capability might be the right choice here in terms of simplicity. Take a look here for demos.
Have a worker of some sort monitor the question queue, then post an answer to the answer queue. Interface out the queue managers and you shouldn't have any problems using any sort of queue tech. Also, the worker doesn't really need to use a queue for answers..
Caveats:
Worker service has access to both queues
Each queue item contains a serialized foreign key to identify themselves.