Azure Service Bus Throttling - azure

Question

We have an architecture built around service bus, which is working well. We are currently also using the same name-spaced service bus to do our logging via a bespoke Serilog sink (which is going to be replaced with datadog).
This has caused us in our test environment, to occasionally be throttled via service bus (we are on standard and not premium). We know that its a problem with our side and are working to resolve it.
However, I want to update our code to handle the throttling exception more gracefully and potentially add a delay before it tries again.
What is not clear from the documentation is the exception details:
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-throttling
We currently catch MessagingException - its not clear if this message will be thrown.
Has anyone handled throttled exceptions in this way?
At the moment the resolution seems to be catch the error, check the exception message against the exception message (I think looking at the errorcode is the least tightly coupled implementation) and put it dedicated code to react to it.

Answer 1

We have few exceptions in service bus such as QuotaExceededExceptions, ServerBusyExceptions, MessageSizeExceededExceptions, TransactionSizeExceededExceptions.
And in some throttling conditions there are several retries to ensure to deliver the messages at the end.
With Azure Monitor we can handle the throttles from metrics:
Below are few steps which I went through:
Can check throttle requests
We can select ServiceBusThrottling
You can check this blog to understand about handling throttle calls with Azure functions for Service bus.
Below is the sample code:
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.ServiceBus.Messaging;
using System.Data.SqlClient;
using System.Text;
using System;
namespace BallardChalmers.BackgroundFunctions
{
public static class BallardChalmersBackgroundTaskCreated
{
[FunctionName("BallardChalmersBackgroundTaskCreated")]
public static void
Run([ServiceBusTrigger("BallardChalmersBackgroundTaskCreated", AccessRights.Send, Connection = "ServiceBusConnection")]string queueMessage, TraceWriter log)
{
log.Info($"Service_TaskManager task added to queue with ID: {queueMessage}");
try
{
string connectionString = System.Configuration.ConfigurationManager.AppSettings["SQLConnectionString"]; ;
using (SqlConnection connection = new SqlConnection(connectionString))
{
// Create wait period between 1 and 60 seconds
System.Random random = new System.Random();
int waitPeriod = random.Next(1, 60);
log.Info($"Wait period: {waitPeriod.ToString()}");
connection.Open();
StringBuilder sb = new StringBuilder();
sb.Append("INSERT FunctionsLog ");
sb.Append("SELECT '" + queueMessage + "','Started','" + DateTime.Now.ToString() + "','" + DateTime.Now.ToString() + "'," + waitPeriod.ToString());
String sql = sb.ToString();
log.Info($"Insert command: {sql}");
using (SqlCommand command = new SqlCommand(sql, connection))
{
command.ExecuteNonQuery();
}
log.Info($"Inserted to FunctionsLog");
System.Threading.Thread.Sleep(1000 * waitPeriod);
log.Info($"Wait completed");
sb = new StringBuilder();
sb.Append("UPDATE FunctionsLog ");
sb.Append("SET [Status]='Completed', Updated='" + DateTime.Now.ToString() + "'");
sb.Append("WHERE ID='" + queueMessage + "'");
sql = sb.ToString();
log.Info($"Update command: {sql}");
using (SqlCommand command = new SqlCommand(sql, connection))
{
command.ExecuteNonQuery();
}
log.Info($"Updated to FunctionsLog");
}
}
catch (SqlException e)
{
log.Error($"Error writing to database: {e.Message + ":::" + e.StackTrace}");
}
}
}
}