WebJobs with DI in .NET Core that runs at set intervals - azure

I already have a WebJob that I created in .NET Core 2.1 which also uses DI. This particular WebJob runs continuously and the new one I'm trying to create will run at set intervals.
In the following code, I'm telling the WebJob to run continuously and need to remove those lines but I want to make sure I'm doing it right. I'm putting //REMOVE on the lines that I need to remove. Could someone please verify I'm doing it right?
Again, the idea is to create a WebJob that will run at set intervals so I need to remove the lines that indicate a continuously running WebJob.
static void Main(string[] args)
{
IServiceCollection serviceCollection = new ServiceCollection();
ConfigureServices(serviceCollection);
var configuration = new JobHostConfiguration();
configuration.Queues.MaxPollingInterval = TimeSpan.FromSeconds(1); // REMOVE
configuration.Queues.BatchSize = 1; // REMOVE
configuration.JobActivator = new CustomJobActivator(serviceCollection.BuildServiceProvider());
configuration.UseTimers();
var host = new JobHost(configuration);
host.RunAndBlock(); // REMOVE
}
private static void ConfigureServices(IServiceCollection services)
{
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.Build();
// Resolve repositories
services.AddTransient<IMyRepository, MyRepository>();
// Create instances of clients
services.AddSingleton(new MyCustomClient(configuration));
// Azure connection strings for the WebJob
Environment.SetEnvironmentVariable("AzureWebJobsDashboard", configuration.GetConnectionString("WebJobsDashboard"));
Environment.SetEnvironmentVariable("AzureWebJobsStorage", configuration.GetConnectionString("WebJobsStorage"));
}

I'm putting //REMOVE on the lines that I need to remove. Could someone please verify I'm doing it right?
AFAIK, the code lines you want to remove could not tell webjob to run at set intervals.
config.Queues.MaxPollingInterval = TimeSpan.FromSeconds(1);
MaxPollingInterval is the max amount of time the WebJob will check the queue. If the queue is empty the WebJob will start checking less frequently up to a max of 10 min.
config.Queues.BatchSize = 2; //the amount of items your WebJob will process at the same time
For more details, you could refer to this article about webjob JobHostConfiguration.
When using the Azure WebJobs SDK you can use TimerTrigger to declare job functions that run on a schedule.
public static void StartupJob(
[TimerTrigger("0 0 */2 * * *", RunOnStartup = true)] TimerInfo timerInfo)
{
Console.WriteLine("Timer job fired!");
}
You can get TimerTrigger and other extensions by installing the Microsoft.Azure.WebJobs.Extensions nuget package.
When using the TimerTrigger, be sure to add a call to config.UseTimers() to your startup code to register the extension.
config.UseTimers(); //allows us to use a timer trigger in our functions.
When using the Azure WebJobs SDK, you deploy your code to a Continuous WebJob, with AlwaysOn enabled. You can then add many scheduled functions you desire in that WebJob.

Related

Azure Web Job always Failed

I have this Azure WebJob function that runs every Saturday. But the Azure function always tagged as Failed but the Job runs successfully when finished as I checked the log.
Already increase WEBJOBS_IDLE_TIMEOUT and SCM_COMMAND_IDLE_TIMEOUT in Configuration but still tagged as Failed. But still got this error.
Command 'cmd /c ""Software.. ...' was aborted due to no output nor CPU activity for 121 seconds. You can increase the SCM_COMMAND_IDLE_TIMEOUT app setting (or WEBJOBS_IDLE_TIMEOUT if this is a WebJob) if needed.
The number of data to be processed is unpredictable, it depends the number of users inputted the values, so the processing time would be between 1 to 40 minutes, 1 minute for least data and 40 minutes for larger data.
I'm currently using the latest version of WebJob SDK.
Here's the code snippet.
public class ProcessDataFunction
{
private readonly IProcessData _processData;
public ProcessDataFunction(IProcessData processData)
{
_processData = processData;
}
[Singleton]
public async Task ProcessDataMessage([TimerTrigger("0 0 12 * * 6", RunOnStartup = true)] TimerInfo myTimer, ILogger logger, CancellationToken cancellationToken)
{
logger.LogInformation("Long running Job Started...");
var dateSync = DateTimeOffset.UtcNow;
await _processData.ProcessAsync(cancellationToken, dateSync);
logger.LogInformation("Long running Job Finished...");
}
}
class Program
{
static async Task Main()
{
var builder = new HostBuilder();
builder.ConfigureWebJobs(b =>
{
b.AddTimers();
b.AddAzureStorageCoreServices();
});
builder.ConfigureLogging((context, b) =>
{
b.AddConsole();
});
builder.ConfigureServices((context, services) =>
{
services.ConfigureHttpClients(context.Configuration)
.ConfigureDataProcessor()
.ConfigureDbContext(context.Configuration);
});
var host = builder.Build();
using (host)
{
await host.RunAsync();
}
}
}
Thanks! Just to confirm if you have enabled Always on setting? As mentioned in the document:
“A web app can time out after 20 minutes of inactivity, and only requests to the actual web app can reset the timer. Viewing the app's configuration in the Azure portal or making requests to the advanced tools site (https://<app_name>.scm.azurewebsites.net) doesn't reset the timer. If you set your web app to run continuously, run on a schedule, or use event-driven triggers, enable the Always on setting on your web app's Azure Configuration page. The Always on setting helps to make sure that these kinds of WebJobs run reliably. This feature is available only in the Basic, Standard, and Premium pricing tiers. “
Also suggest you for triggered jobs in your Program.cs,
try to replace
host.Run(); by host.Start();
Further you may also refer to this blog might helps.

Whats is the difference between 'Settings.job' and 'TimerTrigger' in Azure WebJobs SDK 3.0

There are many tutorials using the following code to create a Webjobs via the WebJob SDK 3.0 library. Specifically 'TimerTrigger'
public void DoSomethingUseful([TimerTrigger("0 */1 * * * *", RunOnStartup = false)] TimerInfo timerInfo, TextWriter log)
{
// Act on the DI-ed class:
string thing = _usefulRepository.GetFoo();
Console.WriteLine($"{DateTime.Now} - {thing}");
}
The above example should run this method as a webjob every 1 minute. However this doesn't work.
I have managed to get the webjob to work when including a setting.job file.
setting.job: { "schedule": "0 */1 * * * *" }
My question is what is the different between these two?
Update:
Please go to the azure webjobs log, then you can see it actually runs as per the timerTrigger defined by SDK(even though the Schedule is n/a, and settings.job is blank, it does not matter):
In short, When using webjob sdk 3.x, you can use TimerTrigger attribute to run the function as per the time you defined. Without using webjobs SDK(like use .zip file or publish a console project from visual studio), you can use setting.job to defined timer instead of TimerTrigger attribute.
1.When you're using webjobs SDK 3.x for timer trigger, you should add this line of code: config.AddTimers(); .
Here are my code using webjobs SDK 3.x(it's a .net core 2.2 console project created in visual studio):
The packages with latest version: Microsoft.Azure.WebJobs / Microsoft.Azure.WebJobs.Extensions / Microsoft.Extensions.Logging.Console
The code in Program.cs:
class Program
{
static void Main(string[] args)
{
var builder = new HostBuilder()
.ConfigureWebJobs(config =>
{
config.AddTimers();
config.AddAzureStorageCoreServices();
})
.ConfigureLogging((context, b) =>
{
b.AddConsole();
}
)
.Build();
builder.Run();
}
}
Then create a new file, like SayHelloWebJob.cs, and code in it:
public class SayHelloWebJob
{
public void ProcessCollateFiles([TimerTrigger("0 */1 * * * *", RunOnStartup = false)]TimerInfo timerInfo,TextWriter writer)
{
writer.WriteLine("hi, it is a testing running");
Console.WriteLine("test");
}
}
Note that in the appsettings.json file, add your storage connection string, like below:
{
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xxx;EndpointSuffix=core.windows.net"
}
Then run the project, you can see the function is triggered as per 1 minute:
2.For settings.job, eg. if you're just creating a console project, and does not use the webjobs sdk. Since you're not using webjobs sdk, you cannot use the timerTrigger attribute. At this moment, you can include the settings.job file(in it's property, set "Copy to Output Directory" as "copy if newer") in this project and configure the scheduled timer like you did in your post. After publish as webjob(from visual studio, when publish, select "Webjob run mode" as "run on demand"), it can run as per the schedule you defined in settings.job.
I have been struggling with the same problem. Here is my understanding after longer research.
There are two kind of Webjobs:
triggered
continuous
Triggered one has to be triggered manually or can be triggered by App Service as per CRON expression schedule provided in setting.job. These jobs are not present in memory once not running.
Continuous one runs always, so the process exists in memory all the time. You can schedule it using Webjobs SKD TimeTrigger attribute.
You will also notice difference between these two Webjob types in the Dashboard.
For triggered Webjobs you will see on top level jobs runs then functions invoked and eventually invocation details.
For continuous Webjobs this will be functions invoked and eventually invocation details. Job runs are missing as this is just one long running job.
Check App Service / Process explorer under Kudu w3wp process to see Webjobs processes running.
Note that continuous and triggered Webjobs have to be started in different way in Main method where you provide the configuration. All comes configured when Webjob of specific type is added via Visual Studio.
This is based on WebJobs 2.x.
My recommendation is
for periodical (e.g. once per few hours, days) jobs use triggered
ones, job when not running will not consume resources,
for more frequent jobs use continuous ones with TimeTrigger
attribute, it will consume resources all the time but will not need
extra time for start-up.

Why does my Time trigger webjob keep running?

I have a Webjob that I want to be time triggered:
public class ArchiveFunctions
{
private readonly IOrderArchiver _orderArchiver;
public ArchiveFunctions(IOrderArchiver orderArchiver)
{
_orderArchiver = orderArchiver;
}
public async Task Archive([TimerTrigger("0 */5 * * * *")] TimerInfo timer, TextWriter log)
{
log.WriteLine("Hello world");
}
}
My program.cs:
public static void Main()
{
var config = new JobHostConfiguration
{
JobActivator = new AutofacJobActivator(RegisterComponents())
};
config.UseTimers();
var host = new JobHost(config);
// The following code ensures that the WebJob will be running continuously
host.RunAndBlock();
}
my publish-setting.json:
{
"$schema": "http://schemastore.org/schemas/json/webjob-publish-settings.json",
"webJobName": "OrdersArchiving",
"runMode": "OnDemand"
}
Here is what it looks like on azure portal:
My problem is that the job runs, I have the hello world, but the job keeps in run state and it get to a time out error message:
[02/05/2018 15:34:05 > f0ea5f: ERR ] Command 'cmd /c ""Ores.Contr ...' was aborted due to no output nor CPU activity for 121 seconds. You can increase the SCM_COMMAND_IDLE_TIMEOUT app setting (or WEBJOBS_IDLE_TIMEOUT if this is a WebJob) if needed.
What can I do to fix this?
I have a wild guess RunAndBlock could be a problem.. but I do not see a solution..
Thanks!
Edit:
I have tested Rob Reagan answer, it does help with the error, thank you!
On my same service, I have one other time triggerd job (was done in core, while mine is not).
You can see the Webjob.Missions is 'triggered', and status update on last time it ran. You can see as well the schedule on it.
I would like to have the same for mine 'OrdersArchiving'.
How can I achieve that?
Thanks!
Change your run mode to continuous and not triggered. The TimerTrigger will handle executing the method you've placed it on.
Also, make sure that you're not using a Free tier for hosting your WebJob. After twenty minutes of inactivity, the app will be paused and will await a new HTTP request to wake it up.
Also, make sure you've enabled Always On on your Web App settings to prevent the same thing from happening to a higher service tier web app.
Edit
Tom asked how to invoke methods on a schedule for a Triggered WebJob. There are two options to do so:
Set the job up as triggered and use a settings.json file to set up the schedule. You can read about it here.
Invoke a method via HTTP using an Azure Scheduler. The Azure Scheduler is a separate Azure service that you can provision. It has a free tier which may be sufficient for your use. Please see David Ebbo's post on this here.

Azure triggered webjob started processing 2 messages at a time

I have an Azure webjob that is triggered by a queue (inherited, not originally written by me). The queue usually only has one item placed on it at a time, but on the first of every month has many items.
On these occasions it has always processed one queue item at a time, finishing processing before picking up the next.
I noticed, however, that as of a couple of months ago it started processing two files at any one time, which is causing problems.
Whilst i could refactor the code to allow for this, I really don't have the time, and the return would be minimal. I simply want it to process one item at a time again, but i cant find anything that may have caused this to change.
Are there any settings in the azure portal I should be aware of? I don't believe any code relating to the trigger itself has changed.
Thank you in advance
Sure, this can be done. Note that a WebJob can be triggered by either a Service Bus Queue or an Azure Storage Queue. Here's info for both.
For Azure Storage Queues
By default, a QueueTrigger will grab 16 messages at a time and process them in parallel. If you don't want this, you need to set the JobHostConfiguration instance's BatchSize property to 1 in your WebJob's static void Main method. Example:
static void Main(string[] args)
{
JobHostConfiguration config = new JobHostConfiguration();
config.Queues.BatchSize = 8;
JobHost host = new JobHost(config);
host.RunAndBlock();
}
For Service Bus Queues
Similarly, you'll set properties in the JobHostConfiguration. If you're using Service Bus, there's a little more setup. Example:
static void Main(string[] args)
{
JobHostConfiguration config = new JobHostConfiguration();
ServiceBusConfiguration serviceBusConfig = new ServiceBusConfiguration();
serviceBusConfig.MessageOptions.MaxConcurrentCalls = 1;
config.UseServiceBus(serviceBusConfig);
JobHost host = new JobHost(config);
host.RunAndBlock();
}
In my case, the problem was that someone had scaled out the Azure app service to 2 instances. However, I'm marking Rob's answer as the most helpful.

Azure WebJob won't run locally in debugger

My Azure WebJob used to run in the VS2015 Debugger, but I found it gradually became very intermittent and now won't run at all. It works fine it I deploy it to Azure. The job is marked as RunOnStartUp.
public class Program
{
static void Main()
{
var config = new JobHostConfiguration();
config.UseTimers();
var host = new JobHost(config);
host.RunAndBlock();
}
}
public class TestJob : BaseJob
{
public static async Task StartupJob([TimerTrigger("05:00:00", RunOnStartup = true)] TimerInfo timerInfo, TextWriter log)
{
log.WriteLine("StartupJob");
await Jobs.Test(some params);
log.WriteLine("Sorted");
}
}
What do I need to do to get it running in the Debugger?
I'm guessing you use the same storage account for your job in Azure and when you debug it locally? If that's the case - the TimeTrigger runs as a singleton which means it needs to acquire a lock to be able to execute. If your webjob is already running in Azure your local version, which you're trying to debug, is not able to acquire the lock.
To avoid this just use different storage accounts for "live" Azure version and local local development.
I would also recommend to enable "development settings" - config.UseDevelopmentSettings(); - when you debug locally. If you enable it you'll see the messages "unable to acquire lock for function..." (or something similar).
See Jason Haley's comment in this thread:
(total hack but should work) rename the function while debugging so
the lock listener blob name will be different.
This hack worked for me. Maybe to make it less hacky, you could use the Disable-attribute to create a timer-triggered function that would only be enabled in your local environment:
Create "MyFunction", which handles the logic. This is the one that will run in your Azure app. Note RunOnStartup=false as recommended by the docs.
[FunctionName("MyFunction")]
public async Task RunJob(
[TimerTrigger("0 0 0 * * *", RunOnStartup = false)] TimerInfo timer)
{
//logic here
}
Create "MyFunction-Local" with the Disable attribute and a different method name. All this does is call the method above.
[FunctionName("MyFunction-Local")]
[Disable("Disable_MyFunction")]
public async Task RunJobLocal(
[TimerTrigger("0 0 0 * * *", RunOnStartup = true)] TimerInfo timer)
{
RunJob(timer);
}
In your local app configuration, set {"Disable_MyFunction" = false}, whereas for the app running in Azure, set this to true.

Resources