I have the following WebJob Function...
public class Functions
{
[NoAutomaticTrigger]
public static void Emailer(IAppSettings appSettings, TextWriter log, CancellationToken cancellationToken)
{
// Start the emailer, it will stop on dispose
using (IEmailerEndpoint emailService = new EmailerEndpoint(appSettings))
{
// Check for a cancellation request every 3 seconds
while (!cancellationToken.IsCancellationRequested)
{
Thread.Sleep(3000);
}
log.WriteLine("Emailer: Canceled at " + DateTime.UtcNow);
}
}
}
I have been looking at how this gets instantiated which I can do with the simple call...
host.Call(typeof(Functions).GetMethod("MyMethod"), new { appSettings = settings })
However it's got me wondering how the TextWriter and CancellationToken are included in the instantiation. I have spotted that JobHostingConfiguration has methods for AddService and I have tried to inject my appSettings using this but it has failed with the error 'Exception binding parameter'.
So how does CancellationToken get included in the instantiation and what is JobHostingConfiguration AddService used for?
how does CancellationToken get included in the instantiation
You could use the WebJobsShutdownWatcher class because it has a Register function that is called when the cancellation token is canceled, in other words when the webjob is stopping.
static void Main()
{
var cancellationToken = new WebJobsShutdownWatcher().Token;
cancellationToken.Register(() =>
{
Console.Out.WriteLine("Do whatever you want before the webjob is stopped...");
});
var host = new JobHost();
// The following code ensures that the WebJob will be running continuously
host.RunAndBlock();
}
what is JobHostingConfiguration AddService used for?
Add Services: Override default services via calls to AddService<>. Common services to override are the ITypeLocator and IJobActivator.
Here is a custom IJobActivator allows you to use DI, you could refer to it to support instance methods.
Related
I have Teams Bot with a code pretty much like example of proactive messages from ms docs.
When I run bot locally - either with Bot Emulator, or Ngrok + Teams Bot registered in Azure, notifications work as expected. When I publish my bot to Azure (as an app service), it does not have conversation references to send proactive message.
My notify controller looks the same as the sample, I just added printing how many conv references are there.
public class NotifyController : ControllerBase {
private readonly IBotFrameworkHttpAdapter _adapter;
private readonly string _appId;
private readonly ConcurrentDictionary<string, ConversationReference> _conversationReferences;
public NotifyController(IBotFrameworkHttpAdapter adapter, IConfiguration configuration, ConcurrentDictionary<string, ConversationReference> conversationReferences) {
_adapter = adapter;
_conversationReferences = conversationReferences;
_appId = configuration["MicrosoftAppId"] ?? "<my-app-id>";
}
public async Task<IActionResult> Get() {
foreach (var conversationReference in _conversationReferences.Values) {
Debug.WriteLine($"awaiting callback from convdId({conversationReference.Conversation.Id}) from user {conversationReference.User.Name}{conversationReference.User.Id}");
await ((BotAdapter)_adapter).ContinueConversationAsync(_appId, conversationReference, BotCallback, default);
}
return new ContentResult() {
Content = $"<html><body><h1>Proactive messages have been sent to {_conversationReferences.Count} conversations.</h1></body></html>",
ContentType = "text/html",
StatusCode = (int)HttpStatusCode.OK,
};
}
private async Task BotCallback(ITurnContext turnContext, CancellationToken cancellationToken) {
await turnContext.SendActivityAsync("proactive hello", cancellationToken: cancellationToken);
}
}
ConfigureServices in Startup.cs:
public void ConfigureServices(IServiceCollection services) {
services.AddHttpClient().AddControllers().AddNewtonsoftJson();
services.AddSingleton<IBotFrameworkHttpAdapter, AdapterWithErrorHandler>();
services.AddSingleton<IStorage, MemoryStorage>();
services.AddSingleton<UserState>();
services.AddSingleton<ConversationState>();
services.AddSingleton<MainDialog>();
services.AddTransient<IBot, DialogBot<MainDialog>>();
services.AddSingleton<ConcurrentDictionary<string, ConversationReference>>();
}
Also in Bot code there are functions (does not matter of the commented function is uncommented, behavior is the same):
public override async Task OnTurnAsync(ITurnContext turnContext, CancellationToken cancellationToken = default) {
await base.OnTurnAsync(turnContext, cancellationToken);
//AddConversationReference(turnContext.Activity as Activity);
await ConversationState.SaveChangesAsync(turnContext, false, cancellationToken);
await UserState.SaveChangesAsync(turnContext, false, cancellationToken);
}
...
private void AddConversationReference(Activity activity) {
var conversationReference = activity.GetConversationReference();
ConversationReferences.AddOrUpdate(conversationReference.User.Id, conversationReference, (key, newValue) => conversationReference);
}
protected override Task OnConversationUpdateActivityAsync(ITurnContext<IConversationUpdateActivity> turnContext, CancellationToken cancellationToken) {
AddConversationReference(turnContext.Activity as Activity);
return base.OnConversationUpdateActivityAsync(turnContext, cancellationToken);
}
I don't know where the references are set to investigate why they are not set :/ running locally, I can see proper number of conversations.
I also added logs to see if comments of Hilton Giesenow are applicable; looks like conversation references are not set, so I don't have any data to save. Sometimes (I don't know under what circumstances) OnTurnAsync is called and then there is data in conversation reference.
What you're using is just a sample, as a result it's there to teach important concepts, but not meant to be used 100% as is in production. In the sample, for instance, it stores all the data for the application in memory (notice this line: services.AddSingleton<IStorage, MemoryStorage>();). That means that the information being stored, like the conversation references, are only available while the application is running, and only on the single computer it's running on.
For a production scenario, you need to be saving these conversation references to a "durable" storage (one that lasts as long as you need), for example a database or a file storage of some sort. Then you can retrieve it from that storage whenever you need it, e.g. to send a proactive message.
On a related note, when you store something in a permanent location, you generally need to "key" it, which means to store it in a way you can uniquely find each record as needed in the future. In the case of a Teams bot, using the AadObjectId for the user is a good starting point for this, as it's unique to every single user.
we have a scenario where we must integrate requests with the same destination system, which exposes its operations with REST APIs (provided by a third party, most likely not Azure). So this is a scenario where n messages are mapped in n actions on the same destination system. There is no multicast or broadcast.
So we are considering Service Bus to achieve this, based on previous experiences on other use cases, and taking advantage of dead letter mechanism among other things.
We need to integrate 6 or 7 different actions with the 3rd party. So on Service Bus we can achieve this by creating 1 topic per action, and this is important because the data that travels on the message is different from action to action.
But we are facing a situation when consuming topics. We are able to have an hosted service in Azure (App Service) that listens on a specific topic and does its stuff.
But since we are trying to listen on several topics, we would like to avoid writing and deploying multiple app services, we would like (if possible) to have a single app service where we 'trigger' each ServiceBusProcessor (one per topic) and even though they all rely on the limits of the app service itself, each processor is independent and is listening on its topic in parallel and processing.
I'll share a code sample below of our hosted service, but we found out two options, we would like to have opinions:
Option 1: we send all messages to the same topic, then by using filters we determine which is the appropriate action. This would make code simple, but it would put all messages on the same 'line' which would make the topic an all purpose topic, which seems wrong
Option 2: based on our sample below, which represents a single hosted service which listens on a single topic, we would break it and inject a List of listeners that implement the same interface, and each one of them would be working independently on its topic and its message. We are not sure if this is feasible and if it works properly, because the app service would have to handle multiple ServiceBusProcessors side by side.
We'd like to know if we are missing some option, or if there is any other better way to achieve this. Hope I've explained it well.
I send below a sample of our hosted service. Thanks a lot.
public class MyService : IHostedService, IMyService
{
private ILogger<MyService> _logger;
public MyService(ILogger<MyService> logger)
{
_logger = logger;
}
public Task StartAsync(CancellationToken cancellationToken)
{
ServiceBusClient client = new ServiceBusClient("connectionString");
ServiceBusProcessor processor = client.CreateProcessor("topicName", "subscriptionName");
processor.ProcessMessageAsync += ProcessMessageAsync;
processor.ProcessErrorAsync += ProcessErrorAsync;
_logger.LogInformation("Listener initialized");
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
public async Task ProcessMessageAsync(ProcessMessageEventArgs args)
{
var body = args.Message.Body;
// Do stuff with this body...
await args.CompleteMessageAsync(args.Message);
}
public Task ProcessErrorAsync(ProcessErrorEventArgs args)
{
_logger.LogError($"Error ocurred: {args.Exception.ToString()} with message: {args.Exception.Message}");
return Task.CompletedTask;
}
}
Then at ConfigureServices:
services.AddHostedService<MyService>();
So, following option 2, the sample above would be transformed in the following, considering 2 listeners:
public interface IMyService
{
}
public interface IMyListener
{
Task Initialize();
Task ProcessMessageAsync(ProcessMessageEventArgs args);
Task ProcessErrorAsync(ProcessErrorEventArgs args);
}
public class BaseListener
{
private string _connectionString;
private string _topicName;
private string _subscriptionName;
private ILogger<BaseListener> _logger;
public BaseListener(ILogger<BaseListener> logger, string connectionString, string topicName, string subscriptionName)
{
this._connectionString = connectionString;
this._topicName = topicName;
this._subscriptionName = subscriptionName;
this._logger = logger;
}
public Task Initialize()
{
ServiceBusClient client = new ServiceBusClient(this._connectionString);
ServiceBusProcessor processor = client.CreateProcessor(this._topicName, this._subscriptionName);
processor.ProcessMessageAsync += ProcessMessageAsync;
processor.ProcessErrorAsync += ProcessErrorAsync;
_logger.LogInformation("Listener initialized");
return Task.CompletedTask;
}
public async Task ProcessMessageAsync(ProcessMessageEventArgs args)
{
var body = args.Message.Body;
// Do stuff with this body...
await args.CompleteMessageAsync(args.Message);
}
public Task ProcessErrorAsync(ProcessErrorEventArgs args)
{
return Task.CompletedTask;
}
}
public class MyListener1: BaseListener, IMyListener
{
public MyListener1(ILogger<MyListener1> logger) : base(logger, "connectionString", "topic1", "subscription")
{
}
}
public class MyListener2 : BaseListener, IMyListener
{
public MyListener2(ILogger<MyListener2> logger) : base(logger, "connectionString", "topic2", "subscription")
{
}
}
public class MyService : IHostedService, IMyService
{
private ILogger<MyService> _logger;
private IEnumerable<IMyListener> _listeners;
public MyService(ILogger<MyService> logger, IEnumerable<IMyListener> listeners)
{
_logger = logger;
_listeners = listeners;
}
public Task StartAsync(CancellationToken cancellationToken)
{
foreach(var listener in this._listeners)
{
listener.Initialize();
}
_logger.LogInformation("Listeners initialized");
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
}
And on ConfigureServices:
services.AddHostedService<MyService>();
services.AddSingleton<IMyListener, MyListener1>();
services.AddSingleton<IMyListener, MyListener2>();
I am maintaining a legacy Cloud Services application hosted on Azure targeting .net 4.6.1. Inside the Application_Start method of the Global.asax on the Web Role we are registering an event handler for RoleEnvironment.StatusCheck however our logs are demonstrating that this event call back is never being called or triggered.
According to this blog: https://convective.wordpress.com/2010/03/18/service-runtime-in-windows-azure/ we were expecting this event to be triggered every 15 seconds and we believe this was happening however has since stopped. We expect that the stopped working around the time we installed some new DLLs into the solution (some of these dlls include: Microsoft.Rest.ClientRuntime.dll, Microsoft.Azure.Storage.Common.dll, Microsoft.Azure.Storage.Blob.dll, Microsoft.Azure.KeyVault.dll)
We've tried RDP-ing onto the VM to check the event logs but nothing obvious is there. Any suggestions on where we may be able to search for clues?
It seems your event handler is not registered. Try below code with a different approach:
public class WorkerRole : RoleEntryPoint
{
public override bool OnStart()
{
RoleEnvironment.StatusCheck += RoleEnvironmentStatusCheck;
return base.OnStart();
}
// Use the busy object to indicate that the status of the role instance must be Busy
private volatile bool busy = true;
private void RoleEnvironmentStatusCheck(object sender, RoleInstanceStatusCheckEventArgs e)
{
if (this.busy)
{
// Sets the status of the role instance to Busy for a short interval.
// If you want the role instance to remain busy, add code to
// continue to call the SetBusy method
e.SetBusy();
}
}
public override void Run()
{
Trace.TraceInformation("Worker entry point called", "Information");
while (true)
{
Thread.Sleep(10000);
}
}
public override void OnStop()
{
base.OnStop();
}
}
I have a function that is bound to a QueueTrigger. In this function I generate a file and write this to a Blob Storage.
But before writing (uploading) the file I want to make sure that the container exists. Is the Configure method in the startup class that inherits FunctionsStartup the right place? It feels wrong to do it every time the trigger runs, isn't it?
I'm using DI to supply my function class some services.
[FunctionName("MyFunction")]
public async Task Run([QueueTrigger(MyQueueName, Connection = "AzureWebJobsStorage")]
MyObject queueMessage, ILogger log)
{
var bytes = Encoding.UTF8.GetBytes("MyFileContent");
// Check if container exists - but not everytime?
var blobClient = new BlobClient(_settings.ConnectionString, _settings.ContainerName, _settings.FileName);
await using var memoryStream = new MemoryStream(bytes);
await blobClient.UploadAsync(memoryStream, true);
}
using MyApp.FunctionApp;
using MyApp.FunctionApp.Options;
using Microsoft.Azure.Functions.Extensions.DependencyInjection;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
[assembly: FunctionsStartup(typeof(Startup))]
namespace MyApp.FunctionApp
{
public class Startup : FunctionsStartup
{
public override void Configure(IFunctionsHostBuilder builder)
{
// Some startup tasks here like ensuring existence of a Blob Container?
builder.Services.AddOptions<Storage>().Configure<IConfiguration>((settings, configuration) =>
{
configuration.GetSection("Storage").Bind(settings);
});
}
}
}
Depending on the frequency of how often you want to check, you could even do something as simple as this:
//shared variable for all instances that run on the same VM
private static bool HaveCheckedBlobContainer = false;
Then, on each invocation:
if (!HaveCheckedBlobContainer)
{
//perform check ...
HaveCheckedBlobContainer = true;
}
I'll generally have an Initialize() method to set up some expensive instances that need to be stored in static member variables. I'll call Initialize() on each invocation, and use a check such as
_someMemberVariable ??= getItFromMyDiContainerOrInstantiateId();
So that it's only executed once, regardless of invocation count.
I am looking at this example to run a durable function Activity after a set timeout.
https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-eternal-orchestrations
This will allow my function activity to perform processing of data, then wait exactly 1 hour before it attempts to load again. This will continue to run forever. Perfect.
However, when publishing the Function to Azure, I don't want to have to manually invoke/start the function via the associated HTTP Trigger. I just want the durable function to kickoff automatically and start processing.
Is this possible? If not, what is a suggested work around?
Thanks!
As discussed in the comments, one way of doing this would be to add a new Task in your Release pipeline.
Here is what I understood of your setup from your question:
[FunctionName("ClientFunction")]
public static async Task<HttpResponseMessage> OnHttpTriggerAsync([HttpTrigger(AuthorizationLevel.Anonymous, "post")]
HttpRequestMessage request, [OrchestrationClient] DurableOrchestrationClient starter, ILogger logger)
{
// Triggers the orchestrator.
string instanceId = await starter.StartNewAsync("OrchestratorFunction", null);
return new HttpResponseMessage(HttpStatusCode.OK);
}
[FunctionName("OrchestratorFunction")]
public static async Task DoOrchestrationThingsAsync([OrchestrationTrigger] DurableOrchestrationContext context, ILogger logger)
{
DateTime deadline = context.CurrentUtcDateTime.Add(TimeSpan.FromHours(1));
await context.CreateTimer(deadline, CancellationToken.None);
// Triggers some yout activity.
await context.CallActivityAsync("ActivityFunction", null);
}
[FunctionName("ActivityFunction")]
public static Task DoAnAwesomeActivity([ActivityTrigger] DurableActivityContext context)
{
}
Now, every time you deploy a new version of the Function App, you need the orchestrator to be run. However, I do not think it can be started by itself.
What I propose is to have a simple bash script (using curl or something else) that would call the ClientFunction at the appropriate URL.
On top of that, one of the nice things of this solution is that you could make the deployment fail if the Azure Function does not respond.
This seems to be working too.
[FunctionName("AutoStart")]
public static async Task Run([TimerTrigger("*/5 * * * * *", RunOnStartup = true, UseMonitor = false)]TimerInfo myStartTimer,
[DurableClient] IDurableClient orchestrationClient, ILogger log)
{
string instanceId = await orchestrationClient.StartNewAsync("Start_Orchestrator", null);
}
I don't know if there are hidden problems with this, but I'm experimenting now with having a TimerTrigger that runs on startup and also once a day at midnight (or whatever schedule you want). That TimerTrigger will search the list of instances for any running instances of this orchestration, terminate them, then start a new one.
private const string MyOrchestrationName = "MyOrchestration";
[FunctionName("MyOrchestration_Trigger")]
public async Task MyOrchestrationr_Trigger(
[TimerTrigger("0 0 0 * * *", RunOnStartup = true)] TimerInfo timer,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log,
CancellationToken cancellationToken)
{
// Get all the instances currently running that have a status of Pending, Running, ContinuedAsNew
var instances = await starter.ListInstancesAsync(new OrchestrationStatusQueryCondition()
{
ShowInput = false,
RuntimeStatus = new List<OrchestrationRuntimeStatus>() { OrchestrationRuntimeStatus.Suspended, OrchestrationRuntimeStatus.Pending, OrchestrationRuntimeStatus.Running, OrchestrationRuntimeStatus.ContinuedAsNew }
}, cancellationToken);
// Find any instances of the current orchestration that are running.
var myInstances = instances.DurableOrchestrationState.Where(inst => inst.Name == MyOrchestrationName);
List<Task> terminateTasks = new List<Task>();
foreach (var instance in myInstances )
{
// Delete any instances that are currently running.
terminateTasks.Add(starter.TerminateAsync(instance.InstanceId, $"Restarting eternal orchestration"));
}
await Task.WhenAll(terminateTasks);
// Start the new task now that other instances have been terminated.
string instanceId = await starter.StartNewAsync(MyOrchestrationName, null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
}
I think at least for my purposes this will be safe. Any activities that are running when you terminate will still run to completion (which is what I want in my case), so you would just kill it and restart it on a schedule.