I am using Microservices deployed on Kubernetes built using asp.net core 2.2. Each microservice is subscribed to different azure service bus "Subscription". In application start up the service add subscription rules (filters) for events that it want to listen to.
Everything works fine and service process messages that it's subscribed to except when i kill the pod and Kubernetes bring back the replica and then service stops processing the messages and they keep on stacking in the queue. Regardless of how much i wait they are not processed even when i restart the pod.
My Startup.cs
public static IServiceCollection RegisterEventBus(this IServiceCollection services, IConfiguration configuration)
{
var subscriptionClientName = configuration["SubscriptionClientName"];
services.AddSingleton<IEventBus, EventBusServiceBus>(sp =>
{
var serviceBusPersisterConnection = sp.GetRequiredService<IServiceBusPersisterConnection>();
var iLifetimeScope = sp.GetRequiredService<ILifetimeScope>();
var logger = sp.GetRequiredService<ILogger<EventBusServiceBus>>();
var eventBusSubcriptionsManager = sp.GetRequiredService<IEventBusSubscriptionsManager>();
return new EventBusServiceBus(serviceBusPersisterConnection, logger,
eventBusSubcriptionsManager, subscriptionClientName, iLifetimeScope);
});
services.AddSingleton<IEventBusSubscriptionsManager, InMemoryEventBusSubscriptionsManager>();
services.AddTransient<PaymentProcessedIntegrationEventHandler>();
services.AddTransient<OrderStatusChangedIntegrationEventHandler>();
return services;
}
//Subscribe to Events
private void ConfigureEventBus(IApplicationBuilder app)
{
var eventBus = app.ApplicationServices.GetRequiredService<IEventBus>();
eventBus.Subscribe<PaymentProcessedIntegrationEvent, PaymentProcessedIntegrationEventHandler>();
eventBus.Subscribe<OrderStatusChangedIntegrationEvent, OrderStatusChangedIntegrationEventHandler>();
}
I copied official Azure Service bus class from eshopcontainers from github and using it in my project here.
https://github.com/ImranMA/MicroCouriers/blob/master/src/BuildingBlocks/EventBus/EventBusServiceBus/EventBusServiceBus.cs
Any help is highly appreciated !
Related
Creating an event using the create event API call in Microsoft Graph with the .NET SDK. This is my code:
private async Task addEvent(Event #event)
{
using(var task = Task.Run(async() => await client
.Me
.Calendars[calendarID]
.Events.Request()
.AddAsync(#event)))
{
while (!task.IsCompleted)
Thread.Sleep(200);
}
}
This is running and working as expected on my local machine and on IIS in a VM by another provider - has been for about 6 months now.
However, when this runs on an Azure App Service, it doesn't throw an exception but no events are actually created. Read-only calls to get events and calendars work, but this one just doesn't. Any ideas appreciated - maybe there's a setting in an app service?
I'm looking to create a WebJob that takes in a request and sends a response, much like an Azure Function with an HTTP trigger. I want to use a WebJob instead because I need to use wkhtmltopdf, which cannot run on a Consumption plan, and we are already paying for an App Service that it can run on.
I know how to run the WebJob using an HTTP POST from this link: https://stackoverflow.com/a/42824776/443044.
What I cannot figure out is how to create the WebJob itself.
Here is my Program class:
public class Program
{
[NoAutomaticTrigger]
public static void TestMethod(TextWriter logger)
{
logger.WriteLine("TEST: " + req.Content.ToString());
}
// Please set the following connection strings in app.config for this WebJob to run:
// AzureWebJobsDashboard and AzureWebJobsStorage
static void Main()
{
var config = new JobHostConfiguration();
...
var host = new JobHost(config);
host.Call(typeof(Program).GetMethod("TestMethod"), null);
}
}
The program throws an exception if I try to give TestMethod the return type of HttpResponseMessage or a parameter of type HttpRequestMessage.
How can I achieve the request/response functionality like with an Azure Function?
we are already paying for an App Service -> You do realize you can host your azure function on an existing app plan as well? learn.microsoft.com/en-us/azure/azure-functions/….
But AFAIK webjobs do not have capabilities to respond to requests.
I want to migrate typical windows service which is written in .net to Azure using Service fabric.
To implement this , I am creating one service fabric application containing one micro service as guest executable which uses .exe of windows service and deploying application package to service fabric cluster.
After deploying service fabric application on cluster I want windows service should install & start automatically on all nodes however at any time application is running on any single node. I want windows service should run on only one node at a time.
Please kindly help to implement this.
You can certainly run your service as a guest executable. Making sure it only runs on one node can be done by setting the instance count to 1 in the manifest, like so:
<Parameters>
<Parameter Name="GuestService_InstanceCount" DefaultValue="-1" />
</Parameters>
...
<DefaultServices>
<Service Name="GuestService">
<StatelessService ServiceTypeName="GuestServiceType"
InstanceCount="[GuestService_InstanceCount]">
<SingletonPartition />
</StatelessService>
</Service>
</DefaultServices>
Or, you could actually migrate it, not just re-host it in the SF environment...
If your Windows Service is written in .NET and the you wan't to benefit from Service Fabric then the job of migrating the code from a Windows Service to a Reliable Service in Service Fabric should not be to big.
Example for a basic service:
If you start by creating a Stateless Service in a Service Fabric application you end up with a service implementation that looks like (comments removed):
internal sealed class MigratedService : StatelessService
{
public MigratedService(StatelessServiceContext context)
: base(context)
{ }
protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
return new ServiceInstanceListener[0];
}
protected override async Task RunAsync(CancellationToken cancellationToken)
{
// TODO: Replace the following sample code with your own logic
// or remove this RunAsync override if it's not needed in your service.
long iterations = 0;
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
ServiceEventSource.Current.ServiceMessage(this.Context, "Working-{0}", ++iterations);
await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
}
}
The RunAsync method starts running as soon as the Service is up and running on a node in the cluster. It will continue to run until the cluster, for some reason, decides to stop the service, or move it to another node.
In your Windows Service code you should have a method that is run on start. This is usually where you set up a Timer or similar to start doing something on a continuous basis:
protected override void OnStart(string[] args)
{
System.Timers.Timer timer = new System.Timers.Timer();
timer.Interval = 60000; // 60 seconds
timer.Elapsed += new System.Timers.ElapsedEventHandler(this.OnTimer);
timer.Start();
}
public void OnTimer(object sender, System.Timers.ElapsedEventArgs args)
{
...
DoServiceStuff();
Console.WriteLine("Windows Service says hello");
}
Now grab that code in OnTimer and put it in your RunAsync method (and any other code you need):
protected override async Task RunAsync(CancellationToken cancellationToken)
{
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
DoServiceStuff();
ServiceEventSource.Current.ServiceMessage(this.Context,
"Reliable Service says hello");
await Task.Delay(TimeSpan.FromSeconds(60), cancellationToken);
}
}
Note the Task.Delay(...), it should be set to the same interval as your Windows Service had for it's Timer.
Now, if you have logging in your Windows Service and you use ETW, then that should work out of the box for you. You simply need to set up some way of looking at those logs from Azure now, for instance using Log Analytics (https://learn.microsoft.com/en-us/azure/log-analytics/log-analytics-service-fabric).
Other things you might have to migrate is if you run specific code on shut down, on continue, and if you have any parameters sent to the service on startup (for instance connection strings to databases). Those need to be converted to configuration settings for the service, look at SO 33928204 for a starting point for that.
The idea behind service fabric is so that it manages your services, from deployment and running. Once you've deployed your service/application to the service fabric instance it will be just like running a windows service (kinda) so you wont need to install your windows service. If you're using something like TopShelf you can just run the exe and everything will run totally fine within service fabric.
With the Azure WebJobs SDK, the process of adding logging to your functions is relatively straightforward: Add a TextWriter param to your triggered function, and write to it. That's it.
The SDK will will then associate and display these logs with their execution instances in the WebJobs Dashboard, which provides a relatively data-rich, yet frictionless view into the operationalization of your webjobs.
While this data is replicated into a user-accessible Azure Storage Blob Container, more custom code would be required to periodically push these logs to App Insights, which is undesirable.
Looking for ideas or solutions for how to push all logs pushed via the injected TextWriter to be pushed to AppInsights (or OMS, for that matter), complete with the webjobs execution/trigger instance metadata, thereby allowing a unified consumption experience for various log analytics.
Based on this Feature being tracked in the WebJobs SDK, I'm assuming that for now this is not possible? A long while back I looked into trying to inject my own TextWriter instance, but I would've had to fork the WebJobs SDK and use my customized assembly that changed a lot of architecture.
You can write a custom TraceWriter that sends log to AppInsights:
using System.Collections.Generic;
using System.Diagnostics;
using Microsoft.ApplicationInsights;
using Microsoft.Azure.WebJobs.Host;
public class AppInsightsTraceWriter : TraceWriter
{
private readonly TelemetryClient _telemetryClient;
public AppInsightsTraceWriter(TraceLevel level, TelemetryClient telemetryClient)
: base(level)
{
_telemetryClient = telemetryClient;
}
public override void Trace(TraceEvent traceEvent)
{
var eventTelemetry = new EventTelemetry() {Name = "WebjobTraceEvent"};
eventTelemetry.Properties.Add(traceEvent.Level.ToString(), traceEvent.ToString());
_telemetryClient.TrackEvent(eventTelemetry);
}
}
In this example, I inject the TelemetryClient class because you should only have one instance of the TelemetryClient class in your application.
So now you just need to configure the Jobhost to use your custom writer :
// Initialize the webjob configuration.
var config = new JobHostConfiguration();
// Only one instance of the telemetry client is needed
var telemetryClient = new TelemetryClient() {InstrumentationKey = "MyInstrumentationKey"};
// Add the app insights tracer for webjob logs/traces.
config.Tracing.Tracers.Add(new AppInsightsTraceWriter(TraceLevel.Info, telemetryClient));
// Detect when the webjob shut down
var cancellationToken = new WebJobsShutdownWatcher().Token;
cancellationToken.Register(() =>
{
// Before shut down, flush the app insights client.
telemetryClient.Flush();
});
new JobHost(config).RunAndBlock();
So if you have a function like that:
public static void ProcessQueueMessage([QueueTrigger("myqueue")] string logMessage, TextWriter log)
{
log.WriteLine(logMessage);
}
Every time you use log.WriteLine, an event will be sent to App Insights.
Note: if this sample also logs from the JobHost are sent to AppInsights.
This is super old (not sure why SO decided to put it in the sidebar for me after this long) but for anyone else that stumbles upon this, app insights is now the recommended way to monitor webjob executions.
Check out the documentation here which steps through the process of connecting app insights to webjobs.
This link walks you through configuring the logging portion of a new webjobs project. Check through the earlier sections to make sure that you've got all the prerequisites.
https://learn.microsoft.com/en-us/azure/app-service/webjobs-sdk-get-started#add-application-insights-logging
static async Task Main()
{
var builder = new HostBuilder();
builder.UseEnvironment(EnvironmentName.Development);
builder.ConfigureWebJobs(b =>
{
b.AddAzureStorageCoreServices();
b.AddAzureStorage();
});
builder.ConfigureLogging((context, b) =>
{
b.AddConsole();
// If the key exists in settings, use it to enable Application Insights.
string instrumentationKey = context.Configuration["APPINSIGHTS_INSTRUMENTATIONKEY"];
if (!string.IsNullOrEmpty(instrumentationKey))
{
b.AddApplicationInsightsWebJobs(o => o.InstrumentationKey = instrumentationKey);
}
});
var host = builder.Build();
using (host)
{
await host.RunAsync();
}
}
I will share the detailed steps to use Application Insights in Azure Web Job, please refer to it.
Create an new Azure Application Insights in Azure portal
Create Azure Web Job project in Visual Studio and install
Microsoft.ApplicationInsights
Set the instrumentation key and send
telemetry
public static void ProcessQueueMessage([QueueTrigger("queuename")] string message, TextWriter log)
{
TelemetryClient tc = new TelemetryClient();
tc.InstrumentationKey = "key copied from Azure portal";
tc.TrackTrace(message);
tc.Flush();
//log.WriteLine(message);
}
This documentation explained how to monitor usage and performance in Windows Desktop apps, you could refer to it to know how to use Azure Application Insights in non-web application. Besides, ApplicationInsights.Helpers.WebJobs could be also helpful.
I have the following scenario:
I have an azure webjob (used to send mails) and I need to check the progress of the webjob in my web application.
I am using SignalR to communicate with clients from my server.
When I want to send an email, I push a message in the queue and the azure webjob does his job.
The question is, how can I communicate the progress of the webjob to the client? Originally my idea was to push a message from the webjob, so the Hub could read it from the queue. Then, I would notify clients from the hub. However, I am not able to find a way to communicate the webjob and the hub, I do not know how to trigger an action in the hub when a message is pushed into the queue or in the service bus. That is to say, I dont know how to subscribe the hub to a certain message of the queue.
Can someone help me with it?
The way that I have done it is to set up the webjob as a SignalR client, push messages via SignalR from the webjob to the server, then relay those messages to the SignalR web clients.
Start by installing the SignalR web client (nuget package ID is Microsoft.AspNet.SignalR.Client) on your webjob.
Then in your webjob, initialise your SignalR connection hub and send messages to the server, e.g.:
public class Functions
{
HubConnection _hub = new HubConnection("http://your.signalr.server");
var _proxy = hub.CreateHubProxy("EmailHub");
public async Task ProcessQueueMessageAsync([QueueTrigger("queue")] EmailDto message)
{
if (_hub.State == ConnectionState.Disconnected)
{
await _hub.Start();
}
...
await _proxy.Invoke("SendEmailProgress", message.Id, "complete");
}
}
Your SignalR server will receive these messages and can then relay them to the other SignalR clients, e.g.:
public class EmailHub : Hub
{
public void SendEmailProgress(int messageId, string status)
{
Clients.All.broadcastEmailStatus(messageId, status);
}
}