Does anyone know of a way that I can get notified of a new subscription occurring for a publisher in NServiceBus? I'd like to run some code every time a new subscription occurs.
Technique 1
Implementing your own subscription storage would give you access to the client address and messages types they are interested in.
Technique 2
public void Init()
{
IBus bus = NServiceBus.Configure.With()
.DefaultBuilder()
.Log4Net()
.XmlSerializer()
.MsmqTransport()
.DisableRavenInstall()
.UnicastBus()
.CreateBus()
.Start();
IUnicastBus ubus = bus as IUnicastBus;
if (null != ubus)
{
ubus.ClientSubscribed += (s, e) => { Console.WriteLine("Client Subscribed {0}:{1}", e.SubscriberReturnAddress.Machine, e.SubscriberReturnAddress.Queue); };
}
}
Related
I am looking into how the app insights work with different types of dependencies. I have a question on using the AppInsights ASP Net Core sdk with Messaging Service Bus sdk.
How can I capture messages to service bus while sending or receiving using this sdk in dependency? I understand that this is not something we would like to log all the time and I will make it configurable.
Thanks
You can create your own sender and implement the desired logic inside the SendMessageAsync method:
public class TelemetryEnabledServiceBusSender : ServiceBusSender
{
private readonly TelemetryClient telemetryClient;
internal TelemetryEnabledServiceBusSender(ServiceBusClient client, string topic, TelemetryClient telemetryClient) : base(client, topic)
{
this.telemetryClient = telemetryClient;
}
public override async Task SendMessageAsync(ServiceBusMessage message, CancellationToken cancellationToken = default)
{
telemetryClient.TrackTrace(message.Body.ToString());
await base.SendMessageAsync(message, cancellationToken);
}
}
use it like this:
var serviceBusSender = new TelemetryEnabledServiceBusSender(serviceBusClient, serviceBusData.Topic, telemetryClient);
await serviceBusSender.SendMessageAsync(message);
Logging processed messages is even simpler and can be done using the ServiceBusProcessor
serviceBusProcessor.ProcessMessageAsync += ProcessMessageAsync;
...
private async Task ProcessMessageAsync(ProcessMessageEventArgs arg)
{
telemetryClient.TrackTrace(arg.Message.Body.ToString());
...
}
Adding my approach as an answer here as it is too long to comment.
var telemetry = new DependencyTelemetry(
"Azure Service Bus",
_serviceBusClient.FullyQualifiedNamespace,
"ServiceBusReceiver.Receive",
string.Empty
);
using var operation =
_telemetryClient.StartOperation(telemetry);
try
{
var receivedMessage = await _serviceBusReceiver.ReceiveMessageAsync();
if (receivedMessage?.Body is not null)
{
message = receivedMessage.Body.ToString();
telemetry.Data = message;
// do something
}
telemetry.Success = true;
//Consider set telemetry duration
}
catch (Exception e)
{
// log exception
_telemetryClient.TrackException(e);
telemetry.Success = false;
throw;
}
finally
{
_telemetryClient.TrackTrace("Done");
_telemetryClient.StopOperation(operation);
}
Thanks to #Peter Bons
Dealing with poison messages (throwing exception while consuming) from Azure Service Bus can lead to loops till number of retries has reached maxDeliveryCount setting of topic subscription.
Does the SequenceNumber of message added by Azure Service bus keeps on increasing on each failed attempt till it reaches maxDeliveryCount ?
Setting maxDeliveryCount = 1, is that best practice to deal with poison messages so that consumer never attempt twice to process message once it failed
Best practices depend on your application and your retry approach.
Most of time I noticed message get failed
Dependent service not available (Redis, SQL connection issue)
Faulty message (message doesn't have a mandatory parameter or some value is incorrect)
Process code issue (bug in message processing code)
For the 1st and 3rd scenario, I created C# web job to run and reprocess deadletter message.
Below is my code
internal class Program
{
private static string connectionString = ConfigurationSettings.AppSettings["GroupAssetConnection"];
private static string topicName = ConfigurationSettings.AppSettings["GroupAssetTopic"];
private static string subscriptionName = ConfigurationSettings.AppSettings["GroupAssetSubscription"];
private static string databaseEndPoint = ConfigurationSettings.AppSettings["DatabaseEndPoint"];
private static string databaseKey = ConfigurationSettings.AppSettings["DatabaseKey"];
private static string deadLetterQueuePath = "/$DeadLetterQueue";
private static void Main(string[] args)
{
try
{
ReadDLQMessages(groupAssetSyncService, log);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
throw;
}
finally
{
documentClient.Dispose();
}
Console.WriteLine("All message read successfully from Deadletter queue");
Console.ReadLine();
}
public static void ReadDLQMessages(IGroupAssetSyncService groupSyncService, ILog log)
{
int counter = 1;
SubscriptionClient subscriptionClient = SubscriptionClient.CreateFromConnectionString(connectionString, topicName, subscriptionName + deadLetterQueuePath);
while (true)
{
BrokeredMessage bmessgage = subscriptionClient.Receive(TimeSpan.FromMilliseconds(500));
if (bmessgage != null)
{
string message = new StreamReader(bmessgage.GetBody<Stream>(), Encoding.UTF8).ReadToEnd();
syncService.UpdateDataAsync(message).GetAwaiter().GetResult();
Console.WriteLine($"{counter} message Received");
counter++;
bmessgage.Complete();
}
else
{
break;
}
}
subscriptionClient.Close();
}
}
For 2nd scenario, we manually verify deadletter messages (Custom UI/ Service Bus explore), sometimes we correct message data or sometimes we purge message and clear queue.
I won't recommend maxDeliveryCount=1. If some network/connection issue occurs, the built-in retry will process and clear from the queue. When I was working in a finance application, I was keeping maxDeliveryCount=5 while in my IoT application is maxDeliveryCount=3.
If you are reading messages in batch, a complete batch will re-process if an error occurred any of message.
SequenceNumber The sequence number can be trusted as a unique identifier since it is assigned by a central and neutral authority and not by clients. It also represents the true order of arrival, and is more precise than a time stamp as an order criterion, because time stamps may not have a high enough resolution at extreme message rates and may be subject to (however minimal) clock skew in situations where the broker ownership transitions between nodes.
I'm new to Azure Service Fabric. I follow the tutorial to create hello demo service for Stateless.
It's simple service and I can find the Event Message in local VS IDE Diagnostic Events to show the message that I print
ServiceEventSource.Current.ServiceMessage(this.Context, "Working-{0}", ++iterations);
Like below picture:
But I can't see any log for cluster manager explorer.
Is it possible to show the event log in this explorer? How to do it?
There are my demo event source class code;
[NonEvent]
public void Message(string message, params object[] args)
{
if (this.IsEnabled())
{
string finalMessage = string.Format(message, args);
Message(finalMessage);
}
}
private const int MessageEventId = 1;
[Event(MessageEventId, Level = EventLevel.Informational, Message = "{0}")]
public void Message(string message)
{
if (this.IsEnabled())
{
WriteEvent(MessageEventId, message);
}
}
Pretty sure that currently the Service Fabric Explorer (SFX) only shows node level events and not application specific events.
According to the resent 7.0 release announcement (https://techcommunity.microsoft.com/t5/Azure-Service-Fabric/Service-Fabric-7-0-Release/ba-p/1015482) work is ongoing to display application specific events in SFX
I got into an issue with IMessageSessionAsyncHandlerFactory where new instances of IMessageSessionAsyncHandler are not created when the volume of writing goes to 0 and then up to a normal level.
To be more precise, I'm using SessionHandlerOptions with a value of 500 for MaxConcurrentSessions. This allows reading at a speed of more than 1k msg/s.
The queue I'm reading from is a partitioned queue.
The volume of messages in the queue is rather constant, but from time to time it gets down to 0. When the volume gets back to the normal level, the SessionFactory is not spawning any handlers so I'm not able to read messages anymore. It's like the sessions were not correctly recycled or are held into a sort of continuous waiting.
Here is the code for the factory registering:
private void RegisterHandler()
{
var sessionHandlerOptions = new SessionHandlerOptions
{
AutoRenewTimeout = TimeSpan.FromMinutes(1),
MessageWaitTimeout = TimeSpan.FromSeconds(1),
MaxConcurrentSessions = 500
};
_queueClient.RegisterSessionHandlerFactoryAsync(new SessionHandlerFactory(_callback), sessionHandlerOptions);
}
The factory class:
public class SessionHandlerFactory : IMessageSessionAsyncHandlerFactory
{
private readonly Action<BrokeredMessage> _callback;
public SessionHandlerFactory(Action<BrokeredMessage> callback)
{
_callback = callback;
}
public IMessageSessionAsyncHandler CreateInstance(MessageSession session, BrokeredMessage message)
{
return new SessionHandler(session.SessionId, _callback);
}
public void DisposeInstance(IMessageSessionAsyncHandler handler)
{
var disposable = handler as IDisposable;
disposable?.Dispose();
}
}
And the handler:
public class SessionHandler : MessageSessionAsyncHandler
{
private readonly Action<BrokeredMessage> _callback;
public SessionHandler(string sessionId, Action<BrokeredMessage> callback)
{
SessionId = sessionId;
_callback = callback;
}
public string SessionId { get; }
protected override async Task OnMessageAsync(MessageSession session, BrokeredMessage message)
{
try
{
_callback(message);
}
catch (Exception ex)
{
Logger.Error(...);
}
}
I can see that the session handlers are closed and that the factories are disposed when the writing/reading is at a normal level. However, once the queue empties, there's no way new session handlers are created. Is there a policy for allocating session IDs that forbids reallocating the same sessions after a period of inactivity?
Edit 1:
I'm adding two pictures to illustrate the behavior:
When the writer is stopped and restarted, the running reader is not able to read as much as before.
The number of sessions created after that moment is also much lower than before:
The volume of messages in the queue is rather constant, but from time to time it gets down to 0. When the volume gets back to the normal level, the SessionFactory is not spawning any handlers so I'm not able to read messages anymore. It's like the sessions were not correctly recycled or are held into a sort of continuous waiting.
When using IMessageSessionHandlerFactory to control how the IMessageSessionAsyncHandler instances are created, you could try to log the creation and destruction for all of your IMessageSessionAsyncHandler instances.
Based on your code, I created a console application to this issue on my side. Here is my code snippet for initializing queue client and handling messages:
InitializeReceiver
static void InitializeReceiver(string connectionString, string queuePath)
{
_queueClient = QueueClient.CreateFromConnectionString(connectionString, queuePath, ReceiveMode.PeekLock);
var sessionHandlerOptions = new SessionHandlerOptions
{
AutoRenewTimeout = TimeSpan.FromMinutes(1),
MessageWaitTimeout = TimeSpan.FromSeconds(5),
MaxConcurrentSessions = 500
};
_queueClient.RegisterSessionHandlerFactoryAsync(new SessionHandlerFactory(OnMessageHandler), sessionHandlerOptions);
}
OnMessageHandler
static void OnMessageHandler(BrokeredMessage message)
{
var body = message.GetBody<Stream>();
dynamic recipeStep = JsonConvert.DeserializeObject(new StreamReader(body, true).ReadToEnd());
lock (Console.Out)
{
Console.ForegroundColor = ConsoleColor.Cyan;
Console.WriteLine(
"Message received: \n\tSessionId = {0}, \n\tMessageId = {1}, \n\tSequenceNumber = {2}," +
"\n\tContent: [ title = {3} ]",
message.SessionId,
message.MessageId,
message.SequenceNumber,
recipeStep.title);
Console.ResetColor();
}
Task.Delay(TimeSpan.FromSeconds(3)).Wait();
message.Complete();
}
Per my test, the SessionHandler could work as expected when the volume of messages in the queue from normal to zero and from zero to normal for some time as follows:
I also tried to leverage QueueClient.RegisterSessionHandlerAsync to test this issue and it works as well. Additionally, I found this git sample about Service Bus Sessions, you could refer to it.
Using methods below, I previously (until last week) could create a service bus with a "mixed" type and then could add a notification hub. However, it suddenly stopped working. It creates a service bus with "Messaging" type and when it tries to create the notification hub, I get the error below:
Enity xxxxxx, create notification hub failed with error forbidden
public ServiceBusNamespaceResponse CreateServiceBus(SubscriptionCloudCredentials credentials, string regoin)
{
var serviceBushubClient = CloudContext.Clients.CreateServiceBusManagementClient(credentials);
var checkserviceBusResponse = serviceBushubClient.Namespaces.CheckAvailability(_deploymentName);
if (checkserviceBusResponse.IsAvailable)
{
try
{
var serviceBusClientResponse = serviceBushubClient.Namespaces.Create(_deploymentName, regoin);
_serviceBusEndpoint = serviceBusClientResponse.Namespace.ServiceBusEndpoint.ToString();
return serviceBusClientResponse;
}
catch (CloudException ex)
{
Console.WriteLine(ex.ErrorMessage);
return null;
}
}
return null;
}
and
public bool CreateNotificationHub(SubscriptionCloudCredentials credentials)
{
SBNotificationHubManager notificationHub;
EntityDescription servesBus = new EntityDescription();
servesBus.Name =_deploymentID;
var des = new MyNotificationHubDescription("push-hub-" + TenantID, servesBus);
notificationHub =
ResourceFactory.Get( _subscriptionID,
new X509Certificate2(Convert.FromBase64String(RowData._base64EncodedCert)),
SBRestResourceType.NHub, des) as SBNotificationHubManager;
if (notificationHub != null)
{
// Console.WriteLine("Created Notification Hub: {0}{1}", Environment.NewLine, notificationHub.LookUp().ToString());
if (notificationHub.Create())
{
Console.WriteLine("Created Notification Hub: {0}{1}", Environment.NewLine, notificationHub.LookUp().ToString());
notificationHub.WaitUntillActive();
return true;
}
}
return false;
}
I was wondering if something recently changed in azure? Can anyone please advise how to define the type of service bus (messaging/notification hub) when creating it in c#. The default with the above method is messaging. I need mixed, but notification hub type also works fine in my case.
Thank you
Now you must explicitly select Messaging (for topics/queues/event hubs) or NotificationHub (for notification hubs). That is idea.
About implementation. Observing source codes I can see overload which accepts ServiceBusNamespaceCreateParameters argument:
Task<ServiceBusNamespaceResponse> CreateNamespaceAsync(string namespaceName, ServiceBusNamespaceCreateParameters namespaceEntity, CancellationToken cancellationToken);
That class has public property of NamespaceType
public NamespaceType NamespaceType{...
So you probably could set it to NamespaceType.NotificationHub and then go ahead to create namespace.