I have SharePoint list. I have written a windows service, which runs every 5 minutes and read new/modified list items from SharePoint and insert into SQL database.
Which one of the below service will be best and cost effective way to run long running service on azure?
VM
Scheduler
Web app/ web job
Worker role/ Web role
Batch
Thanks,
Kannan Eswar.
I'd suggest using either a WebJobs [1] or a Function App [2].
The downside to creating an entire VM to host the service is you'll need to pay to keep the VM running. WebJobs and Functions will both be cheaper and can both be scheduled to execute your task every 5 minutes. If there is not much code involved for the task, I'd look at creating a function in a Function App.
[1] https://learn.microsoft.com/en-us/azure/app-service-web/web-sites-create-web-jobs
[2] https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-timer
Related
I am very new in Azure. I have got a requirement to extract all ADO Workitems for all projects under the domain. I did it using WIQL and personal access token in C#. But the extraction takes around 1 hr to fetch everything for the last few years. Thats fine..
But the trouble is I have no idea how to make it schedule to run early morning every day (Outside business hours). Earlier we have got a dedicated server and thus I made the app a console app and used Task scheduler to run that every morning. But in Azure, please suggest a best and easy solution.
Developed in .NET Core 3.0
Azure webjobs will solve this problem you can enable the trigger to run automatically. Follow below article for step by step:
Micorsoft Document Reference
Using CRON jobs you step the schedule your task early morning
For this requirement, you can create a azure timer trigger function with cron expression which you want. Do your task in function code, the function will be executed according to the cron expression.
When you create the function app, please choose app service plan but not consumption plan because function in consumption plan can just run maximum 10 minutes and you mentioned your task will take about 1 hour. So choose app service plan and set the value of property functionTimeout as -1 in the host.json of your function.
By the way, you'd better also enable "Always on" of your function app because you need the function long running.
I am writing an Azure hosted MVC website for a gym booking system. I need to be able to maintain membership expiry, suspensions as well as gym class attendence (i.e. logging to the database if a session has been missed). Each of these tasks requires a "c# service function" to be run that will go through the database, perform some checks and update records as and when required.
I need this to run pretty regularly to ensure that missed sessions are logged asap. Should I be developing this as an Azure WebJob and running it continuously? Should i be doing it in another manner? If I could get some suggestions on routes to take that would be massively appreciated.
Thanks
You have a few options: Web Jobs, Scheduler, and Worker Roles.
Web Jobs are a nice addon to an existing azure web app and have the benefit of no additional cost. Web Jobs use Scheduler under the covers if you choose to schedule the Web Job to run at an interval other than continuously. Here is a nice answer that describes the differences between the two.
Worker Roles would be the next logical step up from a Web Job. Worker Roles are dedicated Cloud Service VMs that can provide more dedicated power and offer greater scaling capabilities. Worker Roles can also do much more than just run jobs.
For the application you have described, if you are already running on Azure App Services (Web App) it sounds like a continuously running Web Job would be the correct choice.
Ok Azure Experts,
I have a task that only needs to run once every week - this is a long
running task that can take 2-3 days to run.
I have set up a worker role to scale based on a queue. On the day
that we want the task to start - we populate the queue (using a Web
Job).
During the rest of the time, when the queue is empty, I want the
worker roles to shut down - but I cannot scale down to 0 instances.
Originally, we wanted to do this with a Web Job, but the website shuts down from time to time - abruptly turning off my webjob - is this supposed to happen? Even with Keep-Alive turned on? Also, you cannot stop a triggered Web Job from running - so if we want the process to stop - we need to turn off the Web site - not ideal.
How do I scale my instances down to zero?
* Alternatives solutions are also welcome.
Trying to minimize cost here - why pay for a worker role that isn't doing anything?
It is not possible to scale-down a Worker Role to 0 instances at this time. Even if you STOP the worker role, you're still incurring charges for STOPPED instances.
However, behavior that you're looking for, is possible with Virtual Machines. If you shutdown (STOP & DEALLOCATE) a virtual machine, you're not paying fees for that machine.
Now, the only challenge is to stop/start the VM based on a queue count. I don't recall if Azure portal's native scaling supports scaling down to 0 instances for VMs. However, if you use AzureWatch, you should be able to get this done without any issues. Disclaimer: I am affiliated with AzureWatch.
HTH
Can anybody explain the difference between Azure Web Jobs and Azure Scheduler
Azure Web Jobs
Only available on Azure Websites
It is used to run code at particular intervals. E.g. a console application every day
Used to trigger and run workloads.
Mainly recommended for workloads that either scale with the website or are relatively small.
Can be persistently running if "Always On" selected, otherwise you will get the 20 min timeout.
The code that needs to be run and schedule are defined together.
Azure Scheduler
Is not tied to Websites or Cloud Services
It allows you to call a website or add a message to a storage queue
Used for triggering events or triggering small workloads (e.g. add to queue), usually to trigger larger workloads
Mainly recommended for triggering more complex workloads.
This is only a trigger, and a separate function listening to trigger events (e.g. queue's) needs to be coded separately.
For many instances I prefer to use the scheduler to push to a storage queue and a worker role on each instance takes off the queue. This keeps tasks controlled granularly and can also move up or down in scale outside of your website.
With WebJobs they scale up and down with your site and hence your background tasks can become over taxed if your website is experiencing low traffic and scaled down.
Azure Scheduler - Provides a way to easily schedule http calls in a well-defined schedule, like every hour, every Friday at 9:00 am, Once a day, ...
Azure WebJobs - Provides a way to run small to medium work load (in the form of a script: .exe, .cmd, .sh, .js, ...) at the same context of an Azure Website (but can be hosted even with an empty website).
While a WebJob can run continuously (with a process that has a while loop) and Azure will make sure this WebJob is always running (with "Always On" set).
There is also an integration between Azure scheduler and Azure WebJobs where you have a WebJob that is running some finite work and the schduler is responsible for scheduling this work (invoking the WebJob).
So in summary, the scheduler is about scheduling work and WebJobs is about running work load.
Is there a way to use the windows scheduled task to kick off a url or a exe on a schedule?
Can I write a program as an exe then create a Azure VM then RDP into the Azure VM and hook it up to windows task scheduler?
Azure does have a scheduler now.
It allows invoking a Web Service over HTTP/s and post a message to a Windows Azure Storage Queue. It's very new but it can be free if you do not need the scheduler to be executed often. Otherwise it's a small monthly fee which come with scheduled task that can be up to every minute.
Things got much easier lately, please see this link https://azure.microsoft.com/en-us/services/scheduler/ to create a scheduled job in the new Azure Scheduler. There is a basic Free tier as well as some paid options but I think this is exactly what many of us were looking for. It is an excellent solution for triggering URLs with GET,POST,PUT,DELETE requests.
Just follow the simple instructions. Starting by selecting "Scheduler" from the Azure dashboard app menu:
Today the scheduler has been Azure Logic Apps:
https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-overview
If you are looking for something like a cron job (which is a job, that is being run at specific time again and again), then check out Azure Functions:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview
Google Azure Storage Queues. They allow you to schedule jobs that will run at a later date. You can even specify when the job should run.