I have a Azure Scheduled Web Job on Azure WebApp. Inside web job, it downloads the excel file from azure blob and read (25 - 30 k records in a file), then it iterate through loop to insert's records in database. Now, when we schedule (Schedule time: 18 hrs) first time the job ran successfully and processed all records from file without any interruptions (between this time site was accessing). But, next time it fails (Aborted) without any reason (No one was accessed site. may be in idle state).
WebJob Abort Status
There is no any indications when i looked into logs. However, first schedule ran successfully and shows success status later all are in Aborted status, Why?
Success
Also done with below settings in webapp:
stopping_wait_time - 3600
WEBJOBS_IDLE_TIMEOUT - 3600
SCM_COMMAND_IDLE_TIMEOUT - 121
Per Azure's documentation,
Always On. By default, web apps are unloaded if they are idle for some
period of time. This lets the system conserve resources. In Basic or
Standard mode, you can enable Always On to keep the app loaded all the
time. If your app runs continuous web jobs, you should enable Always
On, or the web jobs may not run reliably.
So there is no promise from Azure Web Apps for your WebJob to continue running more than 20 minutes. If you need to rely on that you'll need to enable Always On. To enable it. See below.
In the Azure Portal, navigate to your App Services.
Click Application Settings in the SETTINGS menu.
Turn Always On to on, then click Save.
Here the picture:
If you are using the free tier of Azure, maybe this thread will help you: How can I keep my Azure WebJob running without "Always On".
Related
I have a very simple asp.net core hosted blazor wasm application on a S1 App Service with 2 slots: Integration and production. When I just publish into my integration slot that takes around 50 seconds. But when I swap the integration with the production slot it takes over 7 minutes. Both apps slots are after that very slow, often taking over a minute before they react again. During the swap both are not responding at all.
There are only 2 settings and the connection string to change. And I don't have any manual warmup.
Is the swap functionality just not really meant the be used on such a low configuration or can I adjust something in my configuration to speed things up?
Adding some information here regarding Azure App service deployment slot swap might be helpful:
Some apps might require custom warm-up actions before the swap. The applicationInitialization configuration element in web.config lets you specify custom initialization actions. The swap operation waits for this custom warm-up to finish before swapping with the target slot.
During the swap operation the Web App’s worker process may get restarted in order for some settings to take effect. Even though the swap does not proceed until the restarted worker process comes back online on every VM instance, it may still not be enough for application to be completely ready to take on production traffic.
Try enabling Application Initialization Module to completely warm up your application prior to swapping it into production.
More detailed explanation of that process is available at How to warm up Azure Web App during deployment slots swap
To add to this if the swap operation takes a long time to complete, you can also get information on the swap operation in the activity log. On your app's resource page in the portal, in the left pane, select Activity log. A swap operation appears in the log query as Swap Web App Slots. You can expand it and select one of the sub operations or errors to see the details.
Please refer to below links for more details on this:
https://learn.microsoft.com/en-us/azure/app-service/deploy-staging-slots#what-happens-during-a-swap
https://ruslany.net/2017/11/most-common-deployment-slot-swap-failures-and-how-to-fix-them/
https://ruslany.net/2019/06/azure-app-service-deployment-slots-tips-and-tricks/
I have been running my web jobs for a few months now and the history includes hundreds of thousands of instances when some of them ran, mainly TimerTriggers. When I go in the portal to the "Functions" view of web jogs logs I have noticed that my app service plan shoots up to 100% CPU while I am sat on that page. The page constantly says "Indexing...."
When I close the "Functions" view down the CPU goes straight back down to a few percent, it's normal range.
I assume it must be down to the fact that it has been running for so long and the number of records to search through is so vast. I cannot see any option to archive or remove old records of when jobs ran.
Is there a way I can reduce the history of the jobs? Or is there another explanation?
I'm not familiar with Azure Web Jobs, but I am familiar with Azure Functions which is built on top of Web Jobs, so this might work.
In Azure Functions, each execution is stored in Azure Storage Table. There, you can see all of the parameters that were passed in, as well as the result. I could go into the Storage Table and truncate the records I do not need, so you might be able to do the same with Web Jobs.
Here is how to access this information:
Table Storage in markheath.net/post/three-ways-view-error-logs-azure-functions
Based on your description, I checked my webjob and found the related logs for azure webjobs dashboard as follows:
For Invocation Log Recently executed functions, you could find them as follows:
Note: The list records for Invocation Log are under azure-webjobs-dashboard\functions\recent\flat, the detailed invocation logs are under azure-webjobs-dashboard\functions\instances.
We have a WebJob in our Azure website that was supposed to listen to some messages on Azure Service Bus (ASB), so based on that we were expecting to run a process continuously so that it can listen to bus messages and process them as soon as they arrive. so with this background we chose “Run continuously” as WebJobs’s schedule at deployment time.
After some time, it appears that the WebJob does not work! and as usual we checked the Azure portal in that environment and checked the status of the job and noticed that it is running!
But the observation was, as soon as we open the WebJob tab in the azure portal we could see that it says “Web Job Starting” and after a few seconds it says “Web job Running”!
Any ideas about this behaviour and how to fix it?
There is a setting in the Configure tab called “ALWAYS ON“ which it has a very bold message on top of it that says “Should be turned on if you have a job that runs continuously”.
Here is how this setting looks like:
I also have a blog post about this issue:
https://koukia.ca/microsoft-windows-azure-webjobs-and-how-to-keep-them-awake-16283c28f19f#.nentaowwo
Can anybody explain the difference between Azure Web Jobs and Azure Scheduler
Azure Web Jobs
Only available on Azure Websites
It is used to run code at particular intervals. E.g. a console application every day
Used to trigger and run workloads.
Mainly recommended for workloads that either scale with the website or are relatively small.
Can be persistently running if "Always On" selected, otherwise you will get the 20 min timeout.
The code that needs to be run and schedule are defined together.
Azure Scheduler
Is not tied to Websites or Cloud Services
It allows you to call a website or add a message to a storage queue
Used for triggering events or triggering small workloads (e.g. add to queue), usually to trigger larger workloads
Mainly recommended for triggering more complex workloads.
This is only a trigger, and a separate function listening to trigger events (e.g. queue's) needs to be coded separately.
For many instances I prefer to use the scheduler to push to a storage queue and a worker role on each instance takes off the queue. This keeps tasks controlled granularly and can also move up or down in scale outside of your website.
With WebJobs they scale up and down with your site and hence your background tasks can become over taxed if your website is experiencing low traffic and scaled down.
Azure Scheduler - Provides a way to easily schedule http calls in a well-defined schedule, like every hour, every Friday at 9:00 am, Once a day, ...
Azure WebJobs - Provides a way to run small to medium work load (in the form of a script: .exe, .cmd, .sh, .js, ...) at the same context of an Azure Website (but can be hosted even with an empty website).
While a WebJob can run continuously (with a process that has a while loop) and Azure will make sure this WebJob is always running (with "Always On" set).
There is also an integration between Azure scheduler and Azure WebJobs where you have a WebJob that is running some finite work and the schduler is responsible for scheduling this work (invoking the WebJob).
So in summary, the scheduler is about scheduling work and WebJobs is about running work load.
A very simple question:
Why would someone use the Azure Scheduler if Azure WebJobs are free?
I couldnt find any topic regarding "azure webjobs vs azure scheduler"
The main difference is that the webjob contains everything that the scheduler can do:
Scheduler can make HTTP calls
WebJob can do that and more (run SQL commands, etc)
The actual scheduling bits of WebJobs are built on top of the scheduler. When you set up a Web Job on a schedule under the hood it uses the scheduler to kick it off. WebJobs provides a nice little location to host the code that gets executed. In fact, if you create WebJobs for a web site look in the Scheduler on the portal and you'll see them listed there as well.
Also note that the scheduler could call out to other systems not running Azure. If you have something running in a Cloud Service that needs to be called regularly, or even if something was hosted elsewhere (another provider or on premises) the scheduler is where you can set that up.
Regarding the cost aspect, there is a free tier to the scheduler as well: http://www.windowsazure.com/en-us/pricing/details/scheduler/.
It's 2016. The below answers are no longer accurate.
WebJobs now also has a built-in scheduler and the schedule can be defined by a cron expression.
When publishing to Azure, you can choose if you want to have the WebJob sparked off by the Scheduler or by the WebJob internal scheduler.
Important Note: The Azure Scheduler has limits to frequency of either 1hr or 1min depending on if paid or not. For the internal scheduler however, your App Service requires Always On to keep on running and firing off the job. This Always On status may affect your pricing.
Continuous jobs are monitored, and if they exit they are re-executed. In this way they act more like "services" in your local machine. There is a module that monitors and keeps your app working. Always-ON is a feature that will help your site stay alive and hence, your webjobs to continuously run.
Scheduler is used to trigger the webjobs. It uses the scheduler user account (not the back-end account). This way you can move out of the free tier for scheduler, sign up to higher tiers to suit your needs. But essentially, all the scheduler is doing is hitting an https endpoint (which is public, but required your auth).
Triggered jobs (scheduled and on demand) are invoked by an https call. These calls are load balanced - much in the same way that a web app with many instances is load balanced. Continuous jobs run concurrently by default, but can be set to be a singleton.
For Webjobs starting from version 2, there is no reason to use an Azure Scheduler anymore. As a matter of fact, the Azure Portal already flags this functionality as (Legacy).
From WebJob SDK v2 additional triggers have been introduced and one of them is TimerTrigger, which works with CRON expressions to schedule executions. This execution mode does not need any additional Azure construct, you just need the webapp to be set as AlwaysOn to guarantee the webjob to run.
Another azure service that works with TimerTriggers is Azure Functions, which is built on top of the WebJob SDK that allows a serverless execution.