I am looking into how the app insights work with different types of dependencies. I have a question on using the AppInsights ASP Net Core sdk with Messaging Service Bus sdk.
How can I capture messages to service bus while sending or receiving using this sdk in dependency? I understand that this is not something we would like to log all the time and I will make it configurable.
Thanks
You can create your own sender and implement the desired logic inside the SendMessageAsync method:
public class TelemetryEnabledServiceBusSender : ServiceBusSender
{
private readonly TelemetryClient telemetryClient;
internal TelemetryEnabledServiceBusSender(ServiceBusClient client, string topic, TelemetryClient telemetryClient) : base(client, topic)
{
this.telemetryClient = telemetryClient;
}
public override async Task SendMessageAsync(ServiceBusMessage message, CancellationToken cancellationToken = default)
{
telemetryClient.TrackTrace(message.Body.ToString());
await base.SendMessageAsync(message, cancellationToken);
}
}
use it like this:
var serviceBusSender = new TelemetryEnabledServiceBusSender(serviceBusClient, serviceBusData.Topic, telemetryClient);
await serviceBusSender.SendMessageAsync(message);
Logging processed messages is even simpler and can be done using the ServiceBusProcessor
serviceBusProcessor.ProcessMessageAsync += ProcessMessageAsync;
...
private async Task ProcessMessageAsync(ProcessMessageEventArgs arg)
{
telemetryClient.TrackTrace(arg.Message.Body.ToString());
...
}
Adding my approach as an answer here as it is too long to comment.
var telemetry = new DependencyTelemetry(
"Azure Service Bus",
_serviceBusClient.FullyQualifiedNamespace,
"ServiceBusReceiver.Receive",
string.Empty
);
using var operation =
_telemetryClient.StartOperation(telemetry);
try
{
var receivedMessage = await _serviceBusReceiver.ReceiveMessageAsync();
if (receivedMessage?.Body is not null)
{
message = receivedMessage.Body.ToString();
telemetry.Data = message;
// do something
}
telemetry.Success = true;
//Consider set telemetry duration
}
catch (Exception e)
{
// log exception
_telemetryClient.TrackException(e);
telemetry.Success = false;
throw;
}
finally
{
_telemetryClient.TrackTrace("Done");
_telemetryClient.StopOperation(operation);
}
Thanks to #Peter Bons
Related
we have a scenario where we must integrate requests with the same destination system, which exposes its operations with REST APIs (provided by a third party, most likely not Azure). So this is a scenario where n messages are mapped in n actions on the same destination system. There is no multicast or broadcast.
So we are considering Service Bus to achieve this, based on previous experiences on other use cases, and taking advantage of dead letter mechanism among other things.
We need to integrate 6 or 7 different actions with the 3rd party. So on Service Bus we can achieve this by creating 1 topic per action, and this is important because the data that travels on the message is different from action to action.
But we are facing a situation when consuming topics. We are able to have an hosted service in Azure (App Service) that listens on a specific topic and does its stuff.
But since we are trying to listen on several topics, we would like to avoid writing and deploying multiple app services, we would like (if possible) to have a single app service where we 'trigger' each ServiceBusProcessor (one per topic) and even though they all rely on the limits of the app service itself, each processor is independent and is listening on its topic in parallel and processing.
I'll share a code sample below of our hosted service, but we found out two options, we would like to have opinions:
Option 1: we send all messages to the same topic, then by using filters we determine which is the appropriate action. This would make code simple, but it would put all messages on the same 'line' which would make the topic an all purpose topic, which seems wrong
Option 2: based on our sample below, which represents a single hosted service which listens on a single topic, we would break it and inject a List of listeners that implement the same interface, and each one of them would be working independently on its topic and its message. We are not sure if this is feasible and if it works properly, because the app service would have to handle multiple ServiceBusProcessors side by side.
We'd like to know if we are missing some option, or if there is any other better way to achieve this. Hope I've explained it well.
I send below a sample of our hosted service. Thanks a lot.
public class MyService : IHostedService, IMyService
{
private ILogger<MyService> _logger;
public MyService(ILogger<MyService> logger)
{
_logger = logger;
}
public Task StartAsync(CancellationToken cancellationToken)
{
ServiceBusClient client = new ServiceBusClient("connectionString");
ServiceBusProcessor processor = client.CreateProcessor("topicName", "subscriptionName");
processor.ProcessMessageAsync += ProcessMessageAsync;
processor.ProcessErrorAsync += ProcessErrorAsync;
_logger.LogInformation("Listener initialized");
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
public async Task ProcessMessageAsync(ProcessMessageEventArgs args)
{
var body = args.Message.Body;
// Do stuff with this body...
await args.CompleteMessageAsync(args.Message);
}
public Task ProcessErrorAsync(ProcessErrorEventArgs args)
{
_logger.LogError($"Error ocurred: {args.Exception.ToString()} with message: {args.Exception.Message}");
return Task.CompletedTask;
}
}
Then at ConfigureServices:
services.AddHostedService<MyService>();
So, following option 2, the sample above would be transformed in the following, considering 2 listeners:
public interface IMyService
{
}
public interface IMyListener
{
Task Initialize();
Task ProcessMessageAsync(ProcessMessageEventArgs args);
Task ProcessErrorAsync(ProcessErrorEventArgs args);
}
public class BaseListener
{
private string _connectionString;
private string _topicName;
private string _subscriptionName;
private ILogger<BaseListener> _logger;
public BaseListener(ILogger<BaseListener> logger, string connectionString, string topicName, string subscriptionName)
{
this._connectionString = connectionString;
this._topicName = topicName;
this._subscriptionName = subscriptionName;
this._logger = logger;
}
public Task Initialize()
{
ServiceBusClient client = new ServiceBusClient(this._connectionString);
ServiceBusProcessor processor = client.CreateProcessor(this._topicName, this._subscriptionName);
processor.ProcessMessageAsync += ProcessMessageAsync;
processor.ProcessErrorAsync += ProcessErrorAsync;
_logger.LogInformation("Listener initialized");
return Task.CompletedTask;
}
public async Task ProcessMessageAsync(ProcessMessageEventArgs args)
{
var body = args.Message.Body;
// Do stuff with this body...
await args.CompleteMessageAsync(args.Message);
}
public Task ProcessErrorAsync(ProcessErrorEventArgs args)
{
return Task.CompletedTask;
}
}
public class MyListener1: BaseListener, IMyListener
{
public MyListener1(ILogger<MyListener1> logger) : base(logger, "connectionString", "topic1", "subscription")
{
}
}
public class MyListener2 : BaseListener, IMyListener
{
public MyListener2(ILogger<MyListener2> logger) : base(logger, "connectionString", "topic2", "subscription")
{
}
}
public class MyService : IHostedService, IMyService
{
private ILogger<MyService> _logger;
private IEnumerable<IMyListener> _listeners;
public MyService(ILogger<MyService> logger, IEnumerable<IMyListener> listeners)
{
_logger = logger;
_listeners = listeners;
}
public Task StartAsync(CancellationToken cancellationToken)
{
foreach(var listener in this._listeners)
{
listener.Initialize();
}
_logger.LogInformation("Listeners initialized");
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
}
And on ConfigureServices:
services.AddHostedService<MyService>();
services.AddSingleton<IMyListener, MyListener1>();
services.AddSingleton<IMyListener, MyListener2>();
I'm facing a strange issue, and I ran out of the possible causes. The scenario is
Fetch incoming message from queue
Process it and then add new message to another queue
but the thing is, if I finish the long running task for the incoming message, and then try to add new message to another queue, I don't receive it. If I just add a face message to that another queue, then I am able to receive the real message after the long-running operation is finished. But why ? I don't want to put any fake messages to the queue, but without that my scenario doesn't work. Any ideas ?
public class WorkerRole : RoleEntryPoint
{
// QueueClient is thread-safe. Recommended that you cache
// rather than recreating it on every request
Microsoft.ServiceBus.Messaging.QueueClient Client;
ManualResetEvent CompletedEvent = new ManualResetEvent(false);
public override void Run()
{
MyResult result = null;
var queueClient = new Microsoft.Azure.ServiceBus.QueueClient("QueueConnectionString", "QueueName");
Client.OnMessage(async (receivedMessage) =>
{
try
{
using (Stream stream = receivedMessage.GetBody<Stream>())
{
using (StreamReader reader = new StreamReader(stream))
{
string json = reader.ReadToEnd();
OCRQueueItem_Incoming item = JsonConvert.DeserializeObject<IncomingClass>(json);
var someClass = new OCRManager();
var message = new Message(Encoding.UTF8.GetBytes("test 1"));
await queueClient.SendAsync(message);
result = new SomeManager().RunLongRunningTask(item); //it runs for 1-2min
}
}
}
catch (Exception ex) { }
finally
{
var json = JsonConvert.SerializeObject(result);
var message = new Message(Encoding.UTF8.GetBytes(json));
await queueClient.SendAsync(message);
}
});
CompletedEvent.WaitOne();
}
public override bool OnStart()
{
ServicePointManager.DefaultConnectionLimit = 12;
string connectionString = CloudConfigurationManager.GetSetting("Queue.ConnectionString");
Client = Microsoft.ServiceBus.Messaging.QueueClient.Create(connectionString);
return base.OnStart();
}
public override void OnStop()
{
Client.Close();
CompletedEvent.Set();
base.OnStop();
}
}
I've tried to come up with something from the example in the WebJobsSDK gitHub
var eventHubConfig = new EventHubConfiguration();
string eventHubName = "MyHubName";
eventHubConfig.AddSender(eventHubName,"Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=SendRule;SharedAccessKey=xxxxxxxx");
eventHubConfig.AddReceiver(eventHubName, "Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=ReceiveRule;SharedAccessKey=yyyyyyy");
config.UseEventHub(eventHubConfig);
JobHost host = new JobHost(config);
But I'm afraid that's not far enough for someone of my limited "skillset"!
I can find no instance of JobHostConfiguration that has a UseEventHub property (using the v1.2.0-alpha-10291 version of the Microsoft.AzureWebJobs package), so I can't pass the EventHubConfiguration to the JobHost.
I've used EventHub before, not within the WebJob context. I don't see if the EventHostProcessor is still required if using the WebJob triggering...or does the WebJob trigger essentially act as the EventHostProcessor?
Anyway, if anyone has a more complete example for a simpleton like me that would be really sweet! Thanks
From the documentation here, you should have all the information you need.
What you are missing is a reference of the Microsoft.Azure.WebJobs.ServiceBus.1.2.0-alpha-10291 nuget package.
The UseEventHub is an extension method that is declared in this package.
Otherwise your configuration seems ok.
Here is an example on how to receive or send messages from/to an EventHub:
public class BasicTest
{
public class Payload
{
public int Counter { get; set; }
}
public static void SendEvents([EventHub("MyHubName")] out Payload x)
{
x = new Payload { Counter = 100 };
}
public static void Trigger(
[EventHubTrigger("MyHubName")] Payload x,
[EventHub("MyHubName")] out Payload y)
{
x.Counter++;
y = x;
}
}
EventProcessorHost is still required, as the WebJob just provides the hosting environment for running it. As far as I know, EventProcessorHost is not integrated so deeply into WebJob, so its triggering mechanism cannot be used for processing EventHub messages. I use WebJob for running EventProcessorHost continuously:
public static void Main()
{
RunAsync().Wait();
}
private static async Task RunAsync()
{
try
{
using (var shutdownWatcher = new WebJobsShutdownWatcher())
{
await Console.Out.WriteLineAsync("Initializing...");
var eventProcessorHostName = "eventProcessorHostName";
var eventHubName = ConfigurationManager.AppSettings["eventHubName"];
var consumerGroupName = ConfigurationManager.AppSettings["eventHubConsumerGroupName"];
var eventHubConnectionString = ConfigurationManager.ConnectionStrings["EventHub"].ConnectionString;
var storageConnectionString = ConfigurationManager.ConnectionStrings["EventHubStorage"].ConnectionString;
var eventProcessorHost = new EventProcessorHost(eventProcessorHostName, eventHubName, consumerGroupName, eventHubConnectionString, storageConnectionString);
await Console.Out.WriteLineAsync("Registering event processors...");
var processorOptions = new EventProcessorOptions();
processorOptions.ExceptionReceived += ProcessorOptions_ExceptionReceived;
await eventProcessorHost.RegisterEventProcessorAsync<CustomEventProcessor>(processorOptions);
await Console.Out.WriteLineAsync("Processing...");
await Task.Delay(Timeout.Infinite, shutdownWatcher.Token);
await Console.Out.WriteLineAsync("Unregistering event processors...");
await eventProcessorHost.UnregisterEventProcessorAsync();
await Console.Out.WriteLineAsync("Finished.");
}
catch (Exception ex)
{
await HandleErrorAsync(ex);
}
}
}
private static async void ProcessorOptions_ExceptionReceived(object sender, ExceptionReceivedEventArgs e)
{
await HandleErrorAsync(e.Exception);
}
private static async Task HandleErrorAsync(Exception ex)
{
await Console.Error.WriteLineAsync($"Critical error occured: {ex.Message}{ex.StackTrace}");
}
Using methods below, I previously (until last week) could create a service bus with a "mixed" type and then could add a notification hub. However, it suddenly stopped working. It creates a service bus with "Messaging" type and when it tries to create the notification hub, I get the error below:
Enity xxxxxx, create notification hub failed with error forbidden
public ServiceBusNamespaceResponse CreateServiceBus(SubscriptionCloudCredentials credentials, string regoin)
{
var serviceBushubClient = CloudContext.Clients.CreateServiceBusManagementClient(credentials);
var checkserviceBusResponse = serviceBushubClient.Namespaces.CheckAvailability(_deploymentName);
if (checkserviceBusResponse.IsAvailable)
{
try
{
var serviceBusClientResponse = serviceBushubClient.Namespaces.Create(_deploymentName, regoin);
_serviceBusEndpoint = serviceBusClientResponse.Namespace.ServiceBusEndpoint.ToString();
return serviceBusClientResponse;
}
catch (CloudException ex)
{
Console.WriteLine(ex.ErrorMessage);
return null;
}
}
return null;
}
and
public bool CreateNotificationHub(SubscriptionCloudCredentials credentials)
{
SBNotificationHubManager notificationHub;
EntityDescription servesBus = new EntityDescription();
servesBus.Name =_deploymentID;
var des = new MyNotificationHubDescription("push-hub-" + TenantID, servesBus);
notificationHub =
ResourceFactory.Get( _subscriptionID,
new X509Certificate2(Convert.FromBase64String(RowData._base64EncodedCert)),
SBRestResourceType.NHub, des) as SBNotificationHubManager;
if (notificationHub != null)
{
// Console.WriteLine("Created Notification Hub: {0}{1}", Environment.NewLine, notificationHub.LookUp().ToString());
if (notificationHub.Create())
{
Console.WriteLine("Created Notification Hub: {0}{1}", Environment.NewLine, notificationHub.LookUp().ToString());
notificationHub.WaitUntillActive();
return true;
}
}
return false;
}
I was wondering if something recently changed in azure? Can anyone please advise how to define the type of service bus (messaging/notification hub) when creating it in c#. The default with the above method is messaging. I need mixed, but notification hub type also works fine in my case.
Thank you
Now you must explicitly select Messaging (for topics/queues/event hubs) or NotificationHub (for notification hubs). That is idea.
About implementation. Observing source codes I can see overload which accepts ServiceBusNamespaceCreateParameters argument:
Task<ServiceBusNamespaceResponse> CreateNamespaceAsync(string namespaceName, ServiceBusNamespaceCreateParameters namespaceEntity, CancellationToken cancellationToken);
That class has public property of NamespaceType
public NamespaceType NamespaceType{...
So you probably could set it to NamespaceType.NotificationHub and then go ahead to create namespace.
I have the code for using HttpClient below but now I want to use Windows Azure Mobiles Services.. My app was not published because they said I am not checking for a connection.. What is a simple way to do this ?
private void StackPanel_Loaded(object sender, RoutedEventArgs e)
{
SystemTray.ProgressIndicator = new ProgressIndicator();
//try to ping service before getting high scores
try
{
SetProgressIndicator(true);
SystemTray.ProgressIndicator.Text = "Loading...";
GetHighScores(); //?????????
// HttpClient httpClient = new HttpClient();
// HttpResponseMessage response = await httpClient.GetAsync("http://punkoutersoftware.azurewebsites.net/api/drunkmeterscore");
// response.EnsureSuccessStatusCode();
SetProgressIndicator(false);
}
catch (HttpRequestException e)
{
MessageBox.Show("High Scores not available at the moment");
}
}
private async void GetHighScores()
{
try
{
scores = await scoreTable.OrderByDescending(x => x.Score).ToCollectionAsync();
}
catch (MobileServiceInvalidOperationException e)
{
MessageBox.Show(e.Message, "Error loading items", MessageBoxButton.OK);
}
ListItems.ItemsSource = scores;
}
I doubt the verification process tests with an unresponsive Azure Mobile Services. They're likely testing with no data connection. You can check for a data connection before doing any operation that might need a data connection like this:
if(!DeviceNetworkInformation.IsNetworkAvailable)
{
// inform user to get a data connection
}