Currently we are trying to determine which message service we should use - service bus or event hubs. And then we found that there are ambiguities on the pricing so hope we could get answer here.
Service bus charge
https://azure.microsoft.com/en-us/pricing/details/service-bus/
in this link above we see that standard tier service bus charges on broker connection. and it measures per hour with the peak of the concurrent connections. So problem is this ,say if I have 5K client, but they don't keep the connection alive, say they they send the message once per minute and only open/close the connection at that time, then the peak of concurrent connections might even lower than 100. In this case, how would connections be calculated. Is it going to be calculated in accumulative way or as the peak of certain time in this hour.
in FAQ "What is a brokered connection and how do I get charged for them?" mentioned that "An HTTP call to receive a message from a Service Bus topic or queue that has a receive timeout value greater than zero.", so what about using HTTP call to send message, how it would be count?
And for HTTP call limitation. We could find that there is 5K AMQP connection limit, but what about HTTP? Is it the net messaging refer to? https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-quotas .
Event Hubs charge
https://azure.microsoft.com/en-us/pricing/details/event-hubs/
In this link above we did not see how it charges on connection, but in FAQ it is saying the charge on connection, free on HTTP while AMQP charge is the same is service bus. So here comes two questions,
Is this charge tier matching? Charge on standard tier of event hub on connection is same as standard tier service bus and premium event hubs same as premium service bus? Asking because premium service bus does not charge on connection.
2 For HTTP messaging, same question as in service bus, how many concurrent http connection event hub could support concurrently?
Service Bus Charge
As the FAQ What is a brokered connection and how do I get charged for them? depicts, you will get charged by the hour and prorated for 730 hours in the month.
If all your clients connect to send messages per minute continuously, the charge would be
5000 * 24 * 30.5 / 730 = 5013
Since 1K connections are included, you would be charged for the remaining 4013 connections.
In this case, this call would just count towards a messaging operation.
As mentioned in the docs, REST operations don't count. You could however hit the concurrent receive requests limit.
Event Hubs Charge
Yes
Same as above. No concurrent limit.
Related
I am working with Service Bus Queues in Logic Apps but sometimes when a lot of messages are coming through at a certain time I am starting to get this error while trying to complete the message. It's not related to Message lock lost or Lock Duration, I have looked into other sources online but couldn't find a solid answer to why is this happening and what to do to avoid this completely.
Partitioning may be a solution but I am not 100% sure.
Error: No session available to complete the message with the lock token
Failed
Thanks,
Arslan
As per the Connect to Azure Service Bus from Azure logic apps Microsoft document and as you clearly mentioned that this issue is not because of Message lock lost or lock duration. Maybe the issue could be because of exceeding the session count limit. (1500 unique sessions).
From a service bus, the Service Bus connector can save up to 1,500 unique sessions at a time to the connector cache, per Service Bus messaging entity, such as a subscription or topic. If the session count exceeds this limit, old sessions are removed from the cache. For more information, see Message sessions.
For other technical details about triggers, actions, and limits, which are described by the connector's Swagger description, review the connector reference page. For more about Azure Service Bus Messaging, see What is Azure ServiceBus?
Either you can reduce the sessions per time or you can raise a feature request
We have a usecase where we need to schedule jobs which will be sent as a message from Azure web api service to Azure Service Bus Queue. As we need to schedule it at later point in time one solution is to use Scheduled Delivery ScheduledEnqueueTimeUtc.
What i understand is message gets engqueued only after the time specified expires . My concern is what happens if Web API crashes or undergoes upgrade meanwhile.
1.Will the messages be lost as its not enqueued yet?
2.Where does this messages are stored in the intermediate time ?
Second solution is to use visibilityTimeOut of storage queue where messages are enqueued and will not be impacted by Web API.
From stability and scalability perspective which would be a better option ?
The message is sent to Service Bus, which is enqueued (available to receive) according to the schedule. So, to answer your queries
Nope
In the queue, just not available to receive
visibilityTimeOut is for storage queues. Refer the comparison doc for making the decision.
Note that while you cannot receive scheduled messages, you can peek them.
So, I'm analyzing whether using Functions to consume Service bus messages from a topic is viable.
The Service Bus pricing plan we chose has a limit on the number of 'Brokered connections' (max 1000) per month.
My understanding is that in the typical usage scenario the consumer/listener/subscriber connects to a topic and stays connected persistently, receiving multiple messages during a long period of time (a day, even a week) without breaking the connection, and that this is counted as 1 'brokered connection'. In the end you can receive thousands of messages on a single brokered connection.
How does this work with an Azure function binding? From what I read in the documentation, a function can be idle (ie not running), so it wouldnt be able to maintain this persistent connection.
Is there a separate component for Functions that keeps this connection alive listening for incoming messages? Or are we going to get billed for a new brokered connection each time the function goes idle and then restarts?
I'm including a screenshot of the current plan features:
https://azure.microsoft.com/en-us/pricing/details/service-bus/
Later in the same link:
edit
From Docs:
Service Bus charges for the peak number of concurrent brokered connections that exceed the included quantity (1,000 in the Standard tier). Peaks are measured on an hourly basis, prorated by dividing by 744 hours in a month, and added up over the monthly billing period. The included quantity (1,000 brokered connections per month) is applied at the end of the billing period against the sum of the prorated hourly peaks.
They specifically mention "1,000 brokered connections per month" in the last sentence.
This is an example:
Each of 10,000 devices connects via a single AMQP connection, and receives commands from a Service Bus topic. The devices send telemetry events to an Event Hub. If all devices connect for 12 hours each day, the following connection charges apply (in addition to any other Service Bus topic charges): 10,000 connections * 12 hours * 31 days / 744 = 5,000 brokered connections. After the monthly allowance of 1,000 brokered connections, you would be charged for 4,000 brokered connections, at the rate of $0.03 per brokered connection, for a total of $120.
So I guess all of this is for 10.000 subscribers connecting to the topic at the same time during 12hs, if they connected for 24hs each day then 9,000 brokered connections would be billed (10,000 minus the included 1,000)?
In any case, I'm also trying to validate whether persistent connections are possible with functions (I'm told they are with webjobs).
Azure Functions has a separate component known as the ScaleController that monitors your Service Bus for events 24/7.
Since the underlying SB message receiver in Functions is implemented in WebJobs, there will be a single connection that can retrieve multiple messages throughout the lifetime of the Function instance, although due of its current limitation, it is passing along those messages one at a time to your Function code for processing.
You are only charged when your Function code actually runs. Here's a link that gives you an overview of the ScaleController:https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale#runtime-scaling
If you are anticipating relatively high load, have you considered using Event Hub instead of Service Bus? ServiceBus-Triggers in Azure Functions currently can only process one message at a time, which is not optimal for high load scenarios. Here's a GitHub issue that is opened to track this feature request: https://github.com/Azure/azure-webjobs-sdk/issues/1024
EventHub-Triggers can process messages in batches and that should give your more bang for the buck per Function execution. See https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-hubs
Our Application is using Azure service bus for messaging purpose. We created few topics and subscribers. We will send around 500 messages per day, but in the graph it shows hundreds of thousands of requests for 500 messages. Our billing price is also around 800$ per month. This is too much for 500 * 30 messages. I am not able to understand why the price is this much and also I want to know what does the requests in the graph mean?.
If the reason for the price is because of number of requests then how can I reduce the number of requests?. Messages I am seeing correctly. The problem is only with requests.
This is Just a sample Graph for your reference(Not original). In original graph I see around 100k of requests for 500 messages.
Here, under FAQ:
https://azure.microsoft.com/en-us/pricing/details/service-bus/
How is the Operations meter calculated for queues and topics?
For brokered entities (queues and topics/subscriptions), an operation is any API interaction with Service Bus service on any protocol.
A send, receive delete for a message that is less than or equal to 64KB in size is considered as one billable operation. If the message is greater than 64KB in size, the number of billable operations is calculated according to the message size in multiples of 64KB. For example, an 8 KB message sent to the Service Bus will be billed as one operation, but a 96 KB message sent to the Service Bus will be billed as two operations. Reading the 8KB message with a lock and then completing or explicitly abandoning the message will be billed as two operations. Renewing the lock on a message also incurs an operation.
Multiple deliveries of the same message (for example, message fan out to multiple subscribers or message retrieval after abandon, deferral, or dead lettering) will be counted as independent operations. For example, in the case of a topic with three subscriptions, a single 64KB message sent and subsequently received will generate four billable operations, one “in” plus three “out”, assuming all messages are delivered to all subscriptions and deleted during the read.
Additionally creating, reading (listing), updating and deleting a queue, topic or subscription will each incur an operation charge.
Operations are API calls made against queue or topic/subscription service endpoints. This includes Management, Send/Receive and Session State Operations.
This is what I found in the docs, but not fully sure if that's what you were looking for:
When creating a queue or subscription client, you can specify a receive mode: Peek-lock or Receive and Delete. The default receive mode is PeekLock. When operating in the default mode, the client sends a request to receive a message from Service Bus. After the client has received the message, it sends a request to complete the message.
When setting the receive mode to ReceiveAndDelete, both steps are combined in a single request. These steps reduce the overall number of operations, and can improve the overall message throughput. This performance gain comes at the risk of losing messages.
You can subscribe to asynchronous updates from Azure topics and queues by using SubscriptionClient/QueueClient's .OnMessage call which will presumably create a separate thread polling the topic/queue with default settings and calling a defined callback if it receives anything.
Azure website says that receiving a message is a billable action, which is understandable. However, it isn't clear enough if each those poll requests are considered billable even when they do not return anything, i.e. the queue in question has no pending messages.
Based on the Azure Service Bus Pricing FAQ - the answer to your question is yes
In general, management operations and “control messages,” such as
completes and deferrals, are not counted as billable messages. There
are two exceptions:
Null messages delivered by the Service Bus in
response to requests against an empty queue, subscription, or message
buffer, are also billable. Thus, applications that poll against
Service Bus entities will effectively be charged one message per poll.
Setting and getting state on a MessageSession will also result in
billable messages, using the same message size-based calculation
described above.
Given the price is $0.01 per 10,000 messages, I don't think you should worry too much about that.