We recently implemented Auto Heal on our Azure account. It restarts itself every once in a while, maybe once per day.
We also noticed that some of our long-running cron jobs (web jobs) were not finishing.
Would an Auto Heal restart also kill any running web jobs? Could that explain why the crons are sometimes failing to finish?
There are two main processes for an Azure web app. There is the w3wp.exe process, which runs the website itself. The second main process is the w3wp.exe scm process, which runs KUDU.
All webjobs run on the w3wp.exe scm process (KUDU). Auto heal only restarts the site w3wp.exe process meaning your web jobs should not be impacted by auto heal. The exception here is that if you web job is pulling data from the main site when the site is recycling, this could cause your web job to fail due to the site being offline for the recycle.
You can observe your site processes by navigating to (replace sitename with your web app name) https://sitename.scm.azurewebsites.net/ProcessExplorer/
One item that you should ensure for long running web jobs is that you have 'always on' enabled. You can turn this feature on in the 'Application settings' blade for your web app.
You can also refer the link Proactive Auto Heal to know more about this feature.
Related
I've got a React frontend using a NodeJS backend application running as an App Service in Azure. And I'm troubleshooting that it doesn't work properly to wake up / load the backend application after it has been unloaded due to inactivity.
According to this GitHub issue, the App Service is not unloaded before 20 minutes of inactivity.
Since it would be very ineffective to have to wait 20 minutes for the app to be unloaded over and over while troubleshooting my problem, I would like to manually trigger the App Service to be unloaded. Either using the CLI, an API, a menu option in Azure or something else.
How can I manually trigger Azure to unload an App Service?
When you kill w3wp.exe, I think this is the same function as clicking the recovery on the iis physical server. The process is killed and a new w3wp.exe process will be regenerated when the website is revisited, which can be distinguished by pid.
Tips:
It's just that the test effect is similar, not that the recycle on iis is also a kill process to achieve this function.
Step 1. Open SCM site.
Step 2. Click Process explorer.
Step 3. Right click w3wp.exe, not scm.
Step 4. Then click Kill.
Show test results:
I am just wondering whether Azure web jobs are restarted when I restart my app service?
When you hit the restart in azure portal, both the web app and web jobs get restarted. You can take a look at the official doc here:
Note that when you hit Restart, both the main and SCM sites get restarted.
The SCM site is where Kudu runs, including WebJobs.
You should also keep in mind that when you click Stop in the azure portal, only the main site is stopped, while the SCM site(where web jobs running) keeps running.
Yes, if we restart the webapp, the webjob would also get restarted.
If you go to Kudu console of the webapp (webappname.scm.azurewebsites.net) and click on Process Explorer you will notice that the webjob is running under the w3wp.exe_scm which is the w3wp process for the kudu site. Also, the site is running under w3wp.exe.
If you restart the webapp, process ID for both the w3wp.exe would change.
When I log into the server and open dashboard, it shows recurring jobs with the next execution in the past. Within seconds it triggers processes them automatically, but if I don't log in and open the dashboard, the recurring jobs are not triggered.
It appears that Hangfire is running as expected and processing jobs.
Why will it not trigger without me looking at it?
It's a usual problem if a Hangfire server runs in a web application. The jobs are not triggered because the web server has stopped the app due to user inactivity.
By default, Hangfire Server instance in a web application will not be
started until the first user hits your site. Even more, there are some
events that will bring your web application down after some time (I’m
talking about Idle Timeout and different app pool recycling events).
In these cases your recurring tasks and delayed jobs will not be
enqueued, and enqueued jobs will not be processed.
Read the docs to prevent it: Making ASP.NET application always running.
Another solution is to migrate your Hangfire server from a web application to a console app or a windows service.
In our current setup there are 2 slots for an Azure App service, the question that I have is, when the slot is stopped would it also shut down the webjob that is deployed on that slot or would the web job continue to run?
It is possible to run an Azure WebJob even when the Azure App Service Web App is not running.
Though you can force the WebJob not to run or stop when the Web App is running using WEBJOBS_STOPPED setting in your Web App's "Application Settings"
Take a look at this article about this subject for more details:
https://blogs.msdn.microsoft.com/benjaminperkins/2017/03/01/failed-to-run-webjob/
The following picture shows a running WebJob when the web app is not running:
https://github.com/projectkudu/kudu/wiki/Full-stopping-a-Web-App
I have actually deleted some running jobs, but even though my jobs were shown as deleted in the UI, they are silently running and consuming the memory. I observed this from availability and Performance > Memory Analysis dashboard under Memory Drill Down > Memory Usage (App) > Physical Memory Usage section. Shocked to see the jobs still running after they are deleted. Luckily this solution helped me kill all the processes.
Right now I have a web job that is Continuos with "Always On". This webjob listens for queue messages and performs actions. Everything works fine but I have to pay for every minute the web job "listens" to commands, isn't it?
I have seen that you can also run a web job on demand but I haven't found any example code on how to execute an on demand web job.
Background: I am running a web crawler inside a web job that receives the URL that should be crawled from a message queue. I would like to minimize the costs.
How can I execute an on demand web job from my ASP.NET website?
Thanks
Since your site is already running on a Standard Instance and the WebJob is running in the same Web Hosting Plan at your site, you are not paying anything additional as both the WebJob (and its host site) and the site itself are running on the same VMs.
You can enable (or leave) Always On enabled and not incur any additional costs. Turning off Always On will not save you any money and as you are aware, it is required for a Continuous WebJob to run properly.
Hope that helps.