You can subscribe to asynchronous updates from Azure topics and queues by using SubscriptionClient/QueueClient's .OnMessage call which will presumably create a separate thread polling the topic/queue with default settings and calling a defined callback if it receives anything.
Azure website says that receiving a message is a billable action, which is understandable. However, it isn't clear enough if each those poll requests are considered billable even when they do not return anything, i.e. the queue in question has no pending messages.
Based on the Azure Service Bus Pricing FAQ - the answer to your question is yes
In general, management operations and “control messages,” such as
completes and deferrals, are not counted as billable messages. There
are two exceptions:
Null messages delivered by the Service Bus in
response to requests against an empty queue, subscription, or message
buffer, are also billable. Thus, applications that poll against
Service Bus entities will effectively be charged one message per poll.
Setting and getting state on a MessageSession will also result in
billable messages, using the same message size-based calculation
described above.
Given the price is $0.01 per 10,000 messages, I don't think you should worry too much about that.
Related
We have a usecase where we need to schedule jobs which will be sent as a message from Azure web api service to Azure Service Bus Queue. As we need to schedule it at later point in time one solution is to use Scheduled Delivery ScheduledEnqueueTimeUtc.
What i understand is message gets engqueued only after the time specified expires . My concern is what happens if Web API crashes or undergoes upgrade meanwhile.
1.Will the messages be lost as its not enqueued yet?
2.Where does this messages are stored in the intermediate time ?
Second solution is to use visibilityTimeOut of storage queue where messages are enqueued and will not be impacted by Web API.
From stability and scalability perspective which would be a better option ?
The message is sent to Service Bus, which is enqueued (available to receive) according to the schedule. So, to answer your queries
Nope
In the queue, just not available to receive
visibilityTimeOut is for storage queues. Refer the comparison doc for making the decision.
Note that while you cannot receive scheduled messages, you can peek them.
I have a slightly philosophical problem. We are using Storage Queues for processing the "tickets". The way we have implemented that is we have a background service (worker role) that is polling the storage queue and finding out if there is any ticket to be processed. The nature of the work we do is seasonal. Which means that there won't be tickets all the time to be processed. The problem we are facing with this is - since multiple worker role instances are continuously polling the storage queue, we have cost impact as it's just too many GetMessage() calls.
I came across the Service Bus queue which has event-based capability. There we have the concept of OnMesage() which gets called every time a new message becomes available on a service bus queue.
But my question is - does OnMessage() goes ahead and calls Receive() internally? Which means is it just syntax sugar and internally it is still polling going on and would there be a cost impact in Service Bus Queue case as well?
Any insights into this will be helpful.
Azure Service Bus client is using long polling to retrieve messages from the broker.
By default, it's set to 1 minute or when a message arrives. So if you have a message showing up earlier than 1 minute elapses, it will be retrieved and another poll for 1 minute will be issues. OnMessage/MessageHandler are no exception. It's a higher level abstraction on top of low level receive operation.
In Azure, we have two separate messaging technologies and it's not very well documented when to use what? While EventGrid is really cool, I did not come across when to use EventGrid(scenarios) vs the Storage/ServiceBus queue? Can someone help?
E.g. if I have the following scenario :
A status of a flag changes and based on that, I want to trigger an algorithm that would do recalculations, few inserts/updates etc. in the database.
For implementing this - I can either use EventGrid or Storage Queue. How do we figure what to use in such scenario? I was looking for some kind of guidance.
Basically, Azure Event Grid handles events and Azure ServiceBus handles messages.A message is raw data produced by a service to be consumed or stored. Events are also messages (lightweigth), but they don’t generally convey a publisher intent, other than to inform.
1) If the purpose is to just to store the information ServiceBus can be used.
2) If the information received is used to trigger another service Azure Event Grid can be used.
Find more info here
https://learn.microsoft.com/en-us/azure/event-grid/compare-messaging-services
https://azure.microsoft.com/en-us/blog/events-data-points-and-messages-choosing-the-right-azure-messaging-service-for-your-data/
Events are like notifications from a service to inform the world that something happened in the domain of the publisher (similar to an email notification). There is no expectations from the publisher to have any actions taken. A message is a command you send to a specific receiver with the expectation of the message to be processed (like an asynchronous post request).
Events will work in pub/sub pattern and multiple subscribers could be configured to the events. The service that needs to react to an event will get notified by the event grid when an event occurs (http call from event grid to the receiver). The event will remain in the event grid until deletion (cleanup) and there is no garantie of keeping the original order (no FIFO).
In the other hand, messages will be added to a queue and will be deleted once the “message processor” is done with it. The messages in the queue will keep the original order (FIFO). The message processor has to pull messages from the queue.
In your scenario, you could use a combination of both. Service A sends an event “StatusChanged”, then you can configure a subscription to that event and send a message to a queue, then have your logic to process that message. This will end up with a fully async communication pattern. This is ideal to support scenarios where you processor is down or too busy. The incoming messages will simply get accumulated in the queue and eventually being processed once the service is back up and running. And without affecting the original service that sent the “StatusChanged” event..
My scenario: I have an Azure Storage Queue where messages can come in at any time. If I have 10 items in that queue, it's imperative that they be processed in order. I'm using c# and the windows azure storage SDK.
If the first item fails after, say, 2 seconds it remains invisible on the queue for another 28 seconds (30 second invisibility by default).
Now, my worker will just continue to check a queue for messages and process them as and when. If a queue message fails, it remains invisible and so the next queue item will be processed before the first message is retried.
This seems like really basic functionality for anyone needing a queue where the items are processed in order.
No, I can't set the timeout to a smaller amount because tasks can take varying lengths of time.
George, if you are looking for a messaging queue solution that processes items in order, you should consider using Azure Service Bus Queues:
As a solution architect/developer, you should consider using Service Bus queues when:
Your solution must be able to receive messages without having to poll the queue. With Service Bus, this can be achieved through the use of the long-polling receive operation using the TCP-based protocols that Service Bus supports.
Your solution requires the queue to provide a guaranteed first-in-first-out (FIFO) ordered delivery.
You want a symmetric experience in Azure and on Windows Server (private cloud).
For more information, see Service Bus for Windows Server.
Your solution must be able to support automatic duplicate detection.
There is a good article comparing both Storage Queues and Service Bus: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-azure-and-service-bus-queues-compared-contrasted , you may find the latter better suitable for your case.
I am using Azure Service Bus to implement message communication between separate bounded contexts. I am curious about what techniques people use to ensure that domain events raised in one bc are guaranteed to be received by another consuming bc.
For example, say the "orders" bc raises an "orderPlaced" event, how can I ensure that this event is received by a "shipping" bc. I understand that 2 phase commit is not advisable in cloud, so what is the alternative? How do I mitigate against the order being placed, but the message failing to be sent to the service bus in the event of a network failure?
Thoughts would be welcomed. Thanks.
If you send a BrokeredMessage to a Service Bus Queue and receive an acknowledgement, the message has been successfully stored in the queue. You don't have to worry about the message dying in transit due to a network error after you've been told it is persisted.
What you can worry about is a Service Bus Queue falling offline for a period of time and being unavailable. During an outage, your orderPlaced message wouldn't be able to get into the queue in the first place, and your shipping logic wouldn't be able to receive orders that are already persisted in your queue.
Note that Service Bus Queue outages are transient and the Queue recovers and returns to normal service. At that time, your shipping app could drain the queue of existing messages, and your ordering app could once again insert orderPlaced messages. I don't actually recall the last time I've seen one of my Service Bus Queues go down - it's a rare event.
If you are super-concerned about never ever ever EVER dropping a message, look at paired namespaces. Basically, this allows for failover to standby queues so that you can insert messages while your primary is down. Automatic detection checks to see when your primary queue comes back online. And a siphon process sucks messages that were inserted into the failover queue during the outage back into the primary once the primary comes back online.
Edit: When sending, there is still the chance that even though you had a valid Service Bus Queue connection in your QueueClient or MessagingFactory, the underlying Service Bus Queue just went down like a glass-jawed prizefighter. The vast majority of the time, these errors are transient. To handle them, set the RetryPolicy property of your MessagingFactory or QueueClient. Off the top of my head, I think that the only policy currently available is the RetryExponential policy. This will perform a back-off that will retry sending the message until the specified number of attempts are exhausted. This is the easy-peasy way to handle transient errors that pop up in your Service Bus Queue connection.