I'm testing retry options for Azure Service Bus publisher/subscriber client because after a sudden connection failure the client will not retry to send or receive messages.
Following is the code for publisher client sendMessage() method and I have set maximum delivery count to 1000 for the subscription. Still the client uses default retryPolicy values and I cannot see it retries as I have given in amqpRetryOptions.
static void sendMessage() {
// create Retry Options for the Service Bus client
AmqpRetryOptions amqpRetryOptions = new AmqpRetryOptions();
amqpRetryOptions.setDelay(Duration.ofSeconds(1));
amqpRetryOptions.setMaxRetries(120);
amqpRetryOptions.setMaxDelay(Duration.ofMinutes(5));
amqpRetryOptions.setMode(AmqpRetryMode.EXPONENTIAL);
amqpRetryOptions.setTryTimeout(Duration.ofSeconds(5));
// create a Service Bus Sender client for the queue
ServiceBusSenderClient senderClient = new ServiceBusClientBuilder()
.connectionString(connectionString)
.retryOptions(amqpRetryOptions)
.sender()
.topicName(topicName)
.buildClient();
// send one message to the topic
senderClient.sendMessage(new ServiceBusMessage("Hello, World! "));
System.out.println("Sent a single message to the topic");
}
Is my approach wrong?
If so, what is the standard way?
If not how can approach retry mechanism?
If not how to
I was able to get retrying mechanism work using ServiceBusSenderAsyncClient. Also, I could catch exceptions to check whether the cause is transient or not.
static void sendMessage() {
// create Retry Options for the Service Bus client
AmqpRetryOptions amqpRetryOptions = new AmqpRetryOptions();
amqpRetryOptions.setDelay(Duration.ofSeconds(1));
amqpRetryOptions.setMaxRetries(5);
amqpRetryOptions.setMaxDelay(Duration.ofSeconds(15));
amqpRetryOptions.setMode(AmqpRetryMode.EXPONENTIAL);
amqpRetryOptions.setTryTimeout(Duration.ofSeconds(5));
// instantiate a client that will be used to call the service
ServiceBusSenderAsyncClient serviceBusSenderAsyncClient = new ServiceBusClientBuilder()
.connectionString(connectionString)
.retryOptions(amqpRetryOptions)
.sender()
.topicName(topicName)
.buildAsyncClient();
// create a message
ServiceBusMessage serviceBusMessage = new ServiceBusMessage("Hello, World!\n")
// send the message to the topic
serviceBusSenderAsyncClient.sendMessage(serviceBusMessage).subscribe(
unused -> System.out.println("Message sent successfully"),
error -> {
ServiceBusException serviceBusException = (ServiceBusException) error;
System.out.println(serviceBusException.isTransient());
},
() -> {
System.out.println("Message sent successfully");
}
);
}
Related
with servicebus_client:
receiver = servicebus_client.get_subscription_receiver(topic_name=TOPIC_NAME, subscription_name=SUBSCRIPTION_NAME, max_wait_time=5)
with receiver:
for msg in receiver:
print("Received: " + str(msg))
receiver.complete_message(msg)
This code is taken from the Azure service bus code snippet in Microsoft portal. The code works fine till the it sends the messages, but when it reaches this part of the code, it is not getting in the for loop. The print statement is not getting execute. But I can see the Incoming and outcoming messages on the particular metrics page of that topic. Can I get some help here as I am new to Azure Service Bus?
Code for receive message from topic. its working as expected. you can try again with all library reference and follow all steps.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Azure.Messaging.ServiceBus;
namespace SubscriptionReceiver
{
internal class Program
{
// connection string to your Service Bus namespace
static string connectionString = "Endpoint=sb://xxx/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxx";
// name of the Service Bus topic
static string topicName = "mytopic";
// name of the subscription to the topic
static string subscriptionName = "S1";
// the client that owns the connection and can be used to create senders and receivers
static ServiceBusClient client;
// the processor that reads and processes messages from the subscription
static ServiceBusProcessor processor;
static async Task Main()
{
// The Service Bus client types are safe to cache and use as a singleton for the lifetime
// of the application, which is best practice when messages are being published or read
// regularly.
//
// Create the clients that we'll use for sending and processing messages.
client = new ServiceBusClient(connectionString);
// create a processor that we can use to process the messages
processor = client.CreateProcessor(topicName, subscriptionName, new ServiceBusProcessorOptions());
try
{
// add handler to process messages
processor.ProcessMessageAsync += MessageHandler;
// add handler to process any errors
processor.ProcessErrorAsync += ErrorHandler;
// start processing
await processor.StartProcessingAsync();
Console.WriteLine("Wait for a minute and then press any key to end the processing");
Console.ReadKey();
// stop processing
Console.WriteLine("\nStopping the receiver...");
await processor.StopProcessingAsync();
Console.WriteLine("Stopped receiving messages");
}
finally
{
// Calling DisposeAsync on client types is required to ensure that network
// resources and other unmanaged objects are properly cleaned up.
await processor.DisposeAsync();
await client.DisposeAsync();
}
}
// handle received messages
static async Task MessageHandler(ProcessMessageEventArgs args)
{
string body = args.Message.Body.ToString();
Console.WriteLine($"Received: {body} from subscription: {subscriptionName}");
// complete the message. messages is deleted from the subscription.
await args.CompleteMessageAsync(args.Message);
}
// handle any errors when receiving messages
static Task ErrorHandler(ProcessErrorEventArgs args)
{
Console.WriteLine(args.Exception.ToString());
return Task.CompletedTask;
}
}
}
Output :- Receive Message from Topic Subscription
I am trying a sample code of Azure Event Hub Producer and trying to send some message to Azure Event Hub.
The eventhub and its policy is correctly configured for sending and listening messages. I am using Dotnet core 3.1 console application. However, the code doesn't move beyond CreateBatchAsync() call. I tried debugging and the breakpoint doesn't go to next line. Tried Try-catch-finally and still no progress. Please guide what I am doing wrong here. The Event hub on Azure is shows some number of successful incoming requests.
class Program
{
private const string connectionString = "<event_hub_connection_string>";
private const string eventHubName = "<event_hub_name>";
static async Task Main()
{
// Create a producer client that you can use to send events to an event hub
await using (var producerClient = new EventHubProducerClient(connectionString, eventHubName))
{
// Create a batch of events
using EventDataBatch eventBatch = await producerClient.CreateBatchAsync();
// Add events to the batch. An event is a represented by a collection of bytes and metadata.
eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes("First event")));
eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes("Second event")));
eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes("Third event")));
// Use the producer client to send the batch of events to the event hub
await producerClient.SendAsync(eventBatch);
Console.WriteLine("A batch of 3 events has been published.");
}
}
}
The call to CreateBatchAsync would be the first need to create a connection to Event Hubs. This indicates that you're likely experiencing a connectivity or authorization issue.
In the default configuration you're using, the default network timeout is 60 seconds and up to 3 retries are possible, with some back-off between them.
Because of this, a failure to connect or authorize may take up to roughly 5 minutes before it manifests. That said, the majority of connection errors are not eligible for retries, so the failure would normally surface after roughly 1 minute.
To aid in your debugging, I'd suggest tweaking the default retry policy to speed things up and surface an exception more quickly so that you have the information needed to troubleshoot and make adjustments. The options to do so are discussed in this sample and would look something like:
var connectionString = "<< CONNECTION STRING FOR THE EVENT HUBS NAMESPACE >>";
var eventHubName = "<< NAME OF THE EVENT HUB >>";
var options = new EventHubProducerClientOptions
{
RetryOptions = new EventHubsRetryOptions
{
// Allow the network operation only 15 seconds to complete.
TryTimeout = TimeSpan.FromSeconds(15),
// Turn off retries
MaximumRetries = 0,
Mode = EventHubsRetryMode.Fixed,
Delay = TimeSpan.FromMilliseconds(10),
MaximumDelay = TimeSpan.FromSeconds(1)
}
};
await using var producer = new EventHubProducerClient(
connectionString,
eventHubName,
options);
I created a simple Azure Service bus (Queue) and a client that is sending message to service bus. Using below code to send message:
using Microsoft.Azure.ServiceBus;
using Microsoft.Extensions.Configuration;
public async Task SendMessageAsync<T>(T message, string queueName)
{
try
{
var queueClient = new QueueClient(_config.GetConnectionString("AzureServiceBus"), queueName);
string messageBody = JsonSerializer.Serialize(message);
var byteMessage = new Message(Encoding.UTF8.GetBytes(messageBody));
queueClient.SendAsync(byteMessage);
Console.WriteLine((message as Employee).FirstName);
}
catch (Exception ex)
{
var c = ex;
}
}
Sending message using:
using SenderApp;
Console.WriteLine("Hello, World!");
QueueService service = new QueueService();
for (int i = 0; i < 100; i++)
{
Employee e = new Employee();
e.FirstName = "1 " + i.ToString();
e.LastName = "2 " + i.ToString();
service.SendMessageAsync<Employee>(e, "employeequeue");
}
When I try to see active messages, There is nothing in the queue:
However I do see some traffic. But the number of message I sent (over 100) is not equal to number of incoming request show (62) at the bottom of the image. I am not sure what is happening to my messages? This defeats the purpose of the queue.
Please guide me why I am not seeing any messages. What is the best way to handle this ?
I am using following nuget packages:
<PackageReference Include="Microsoft.Azure.ServiceBus" Version="5.2.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.Abstractions" Version="6.0.0" />
<PackageReference Include="Microsoft.Extensions.Hosting" Version="6.0.1" />
A message sent to an Azure Service Bus queue will be delivered to the queue unless operation is failing. In that case, an exception will be thrown. Check the following:
Exception handling doesn't swollow exceptions
Await asynchronous send operations to ensure messages are dispatched
Namespace/queue used for sending is what you use to receive
There are no competing consumers, actively receiving messages from the queue.
Validate TCP ports needed for AMQP are not blocked. If those ports are blocked, you could configure your client to use WebSockets.
So I still dont know what caused this issue. But I realized Microsoft.Azure.ServiceBus package was deprecated and later I started using Azure.Messaging.ServiceBus package to connect to service bus and things started to work.
I used following code to send message to queue:
string connectionString = "Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=f3f+qMYTyVwE18YNl5J6ygJFi30v6J/Smph5HZvyQyE=";
string queueName = "employeequeue";
// since ServiceBusClient implements IAsyncDisposable we create it with "await using"
await using var client = new ServiceBusClient(connectionString);
// create the sender
ServiceBusSender sender = client.CreateSender(queueName);
// create a message that we can send. UTF-8 encoding is used when providing a string.
ServiceBusMessage message = new ServiceBusMessage("Hello world! " + id);
// send the message
await sender.SendMessageAsync(message);
return "Sent";
Used following code to receive message:
string queueName = "employeequeue";
// since ServiceBusClient implements IAsyncDisposable we create it with "await using"
await using var client = new ServiceBusClient(connectionString);
// create a receiver that we can use to receive and settle the message
ServiceBusReceiver receiver = client.CreateReceiver(queueName);
// the received message is a different type as it contains some service set properties
ServiceBusReceivedMessage receivedMessage = await receiver.ReceiveMessageAsync();
string body = receivedMessage.Body.ToString();
// complete the message, thereby deleting it from the service
await receiver.CompleteMessageAsync(receivedMessage);
More info is available # https://github.com/Azure/azure-sdk-for-net/blob/Azure.Messaging.ServiceBus_7.7.0/sdk/servicebus/Azure.Messaging.ServiceBus/README.md
in my case AMOP protocol is blocked by firewall. I just can work with Azure eventhub with https protocol.
I just find:
.NET EventHubConnectionOptions.TransportType property with EventHubsTransportType.AmqpTcp or EventHubsTransportType.AmqpWebSockets
I do not know, how to change the protocol in receiver of my application.
this is my sample code:
static async Task Main()
{
// Read from the default consumer group: $Default
string consumerGroup = EventHubConsumerClient.DefaultConsumerGroupName;
// Create a blob container client that the event processor will use
storageClient = new BlobContainerClient(blobStorageConnectionString, blobContainerName);
// Create an event processor client to process events in the event hub
processor = new EventProcessorClient(storageClient, consumerGroup, ehubNamespaceConnectionString, eventHubName);
// Register handlers for processing events and handling errors
processor.ProcessEventAsync += ProcessEventHandler;
processor.ProcessErrorAsync += ProcessErrorHandler;
// Start the processing
await processor.StartProcessingAsync();
// Wait for 30 seconds for the events to be processed
await Task.Delay(TimeSpan.FromSeconds(30));
// Stop the processing
await processor.StopProcessingAsync();
}
static async Task ProcessEventHandler(ProcessEventArgs eventArgs)
{
// Write the body of the event to the console window
Console.WriteLine("\tReceived event: {0}", Encoding.UTF8.GetString(eventArgs.Data.Body.ToArray()));
// Update checkpoint in the blob storage so that the app receives only new events the next time it's run
await eventArgs.UpdateCheckpointAsync(eventArgs.CancellationToken);
}
It's a bit convoluted process.
You would want to use EventProcessorClient(BlobContainerClient, String, String, EventProcessorClientOptions) constructor and then specify ConnectionOptions in your client options.
There's a property called TransportType in ConnectionOptions which is of type EventHubsTransportType.
You can specify TransportType as EventHubsTransportType.AmqpWebSockets and then you should be able to use AMQP over WebSockets.
I have a scenario in which I am calling RegisterMessageHandler of SubscriptionClient class of Azure Service Bus library.
Basically I am using trigger based approach while receiving the messages from Service Bus in one of my services in Service Fabric Environment as a stateless service.
So I am not closing the subscriptionClient object immediately, rather I am keeping it open for the lifetime of the Service so that it keeps on receiving the message from azure service bus topics.
And when the service needs to shut down(due to some reasons), I want to handle the cancellation token being passed into the service of Service Fabric.
My question is how can I handle the cancellation token in the RegisterMessageHandler method which gets called whenever a new message is received?
Also I want to handle the closing of the Subscription client "Gracefully", i.e I want that if a message is already being processed, then I want that message to get processed completely and then I want to close the connection.
Below is the code I am using.
Currently We are following the below approach:
1. Locking the process of the message using semaphore lock and releasing the lock in finally block.
2. Calling the cancellationToken.Register method to handle cancellation token whenever cancellation is done. Releasing the lock in the Register Method.
public class AzureServiceBusReceiver
{
private SubscriptionClient subscriptionClient;
private static Semaphore semaphoreLock;
public AzureServiceBusReceiver(ServiceBusReceiverSettings settings)
{
semaphoreLock = new Semaphore(1, 1);
subscriptionClient = new SubscriptionClient(
settings.ConnectionString, settings.TopicName, settings.SubscriptionName, ReceiveMode.PeekLock);
}
public void Receive(
CancellationToken cancellationToken)
{
var options = new MessageHandlerOptions(e =>
{
return Task.CompletedTask;
})
{
AutoComplete = false,
};
subscriptionClient.RegisterMessageHandler(
async (message, token) =>
{
semaphoreLock.WaitOne();
if (subscriptionClient.IsClosedOrClosing)
return;
CancellationToken combinedToken = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, token).Token;
try
{
// message processing logic
}
catch (Exception ex)
{
await subscriptionClient.DeadLetterAsync(message.SystemProperties.LockToken);
}
finally
{
semaphoreLock.Release();
}
}, options);
cancellationToken.Register(() =>
{
semaphoreLock.WaitOne();
if (!subscriptionClient.IsClosedOrClosing)
subscriptionClient.CloseAsync().GetAwaiter().GetResult();
semaphoreLock.Release();
return;
});
}
}
Implement the message client as ICommunicationListener, so when the service is closed, you can block the call until message processing is complete.
Don't use a static Semaphore, so you can safely reuse the code within your projects.
Here is an example of how you can do this.
And here's the Nuget package created by that code.
And feel free to contribute!