Does having many triggers (blob, servicebus,timer) in a single webjob will reduce the performance of webjob?
Is there any way to improve performance of webjob with many triggers?
Can a heavy weight webjob be divided into smaller weight webjob?
Regard to Azure WebJobs as a feature of Azure App Service to run background job as the offical document said below.
WebJobs is a feature of Azure App Service that enables you to run a program or script in the same context as a web app, API app, or mobile app. There is no additional cost to use WebJobs.
Althought it said no additional cost, WebJob as a simple and useful feature was founded before Azure published other similar and more powerful services, which like Functions be introduced in the same doc as below.
Azure Functions provides another way to run programs and scripts. For a comparison between WebJobs and Functions, see Choose between Flow, Logic Apps, Functions, and WebJobs.
In the reference document above, the Summary section recommend its best application scenario.
Summary
Azure Functions offers greater developer productivity, more programming language options, more development environment options, more Azure service integration options, and more pricing options. For most scenarios, it's the best choice.
Here are two scenarios for which WebJobs may be the best choice:
You need more control over the code that listens for events, the JobHost object. Functions offers a limited number of ways to customize JobHost behavior in the host.json file. Sometimes you need to do things that can't be specified by a string in a JSON file. For example, only the WebJobs SDK lets you configure a custom retry policy for Azure Storage.
You have an App Service app for which you want to run code snippets, and you want to manage them together in the same DevOps environment.
For other scenarios where you want to run code snippets for integrating Azure or third-party services, choose Azure Functions over WebJobs with the WebJobs SDK.
Meanwhile, per my experience on Azure, WebJobs and Functions are only suitable for some simple and light-weight task job. For high performance requirement, Azure Batch service is a good choice to get the balance between cost and ease of use.
1. Does having many triggers (blob, servicebus,timer) in a single webjob will reduce the performance of webjob?
Provided that your webjob is not singleton. You can have multiple functions in your webjob with multiple triggers and the performance will not reduce. (provided your webapp plan is beefy enough to handle all the load.)
2.Is there any way to improve performance of webjob with many triggers? -
the best way would be to divide the webjob into smaller webjobs and each webjob having a single trigger. And scale your webjobs out (add more instances) based on load. Also in case your webjob executes within 5 mins you can also choose to use Azure Function App. Which gives a much better option. Alternatively, assuming the webjob execution takes more than 5 mins you can have your webjob exe as a docker image and provision them using logic app on demand using ACI. In this scenario you will be configuring the triggers in logic app.
1. Can a heavy weight webjob be divided into smaller weight webjob? - yes, see my previous answers.
Related
When hosting an Azure Function in an App Service Plan, are there any significant differences compared with using an App Service (EDIT: for a restful API) associated with the same App Service Plan? I assume the only difference is that the Function offers additional out of the box triggers. Any differences I'm missing that would lead to preferring one over the other?
I'd also like to confirm that hosting Azure Functions in an App Service Plan can actually limit scalability if scaling is not configured on the App Service Plan. As I understand it, Functions automatically scale as needed when using Consumption or Premium hosting without additional configuration.
When hosting an Azure Function in an App Service Plan, are there any significant differences compared with using an App Service associated with the same App Service Plan? I assume the only difference is that the Function offers additional out of the box triggers. Any differences I'm missing that would lead to preferring one over the other?
Well, an Azure Function is a different beast than an App Service. An Azure function is triggered by an external event or a timer. It then executes the code of the function. When hosted on a consumption plan this execution is allowed to run for 5 or 10 minutes max. When you need a longer execution time you need to run it on an App Service Plan.
An App Service can host any app you've created. Like a website (that runs continuously and doesn't need to be triggered before it starts doing something) or an api for example.
I'd also like to confirm that hosting Azure Functions in an App Service Plan can actually limit scalability if scaling is not configured on the App Service Plan. As I understand it, Functions automatically scale as needed when using Consumption or Premium hosting without additional configuration.
Correct, when hosting Azure Functions in an App Service Plan you are responsible for making sure the app service is scaled to allow the function to perform well under load. Thats why the consumption plan is designed to handle this so the developer can focus on the functionality and does not need to worry about the infrastructure.
So, for integration scenario's azure functions are a very natural fit. For web sites an App Service might be the best solution.
To address your comment:
I should have mentioned that this question was in the context of hosting a restful API and not a UI application. In this scenario, I'm not seeing much difference between a Function and App Service, but please correct me if I'm missing something
A couple of things: For one, there is a certain sweet spot. If traffic is heavy enough a consumption plan based azure function might be more costly than having a dedicated app service plan. That depends of course on a lot of factors (CPU usage, request duration etc.). Also, you won't be able to use things like Asp.Net Core Middleware out of the box.
Finally, I'd argue that if your api is becoming large enough managing a single asp.net core solution may be easier than having to manage a lot of azure functions with small functions or one azure function project with lots and lots of functions, but hey, that's just my opinion (haven't actually dealt with it to be honest)
Some resources to consider:
https://www.taztopia.com/single-post/2019/07/28/azure-function-vs-web-app-aka-serverless-vs-paas
https://dasith.me/2018/01/20/using-azure-functions-httptrigger-as-web-api/
The main difference is in how you pay for it:
Azure Functions Consumption plan you pay per execution
Azure Functions in an App Service (dedicated plan) you pay for the allocated hardware per minute.
You also have control of which VNET your functions run in when you use an app service plan. You may have security requirements that make this important.
You are correct that if you run it in an app service that is not configured to scale, then throughput will be limited by the allocated hardware.
For details see: https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale
If you are having limited and predictable workload then deploy az function in AppService plan with supports VNET integration for private compute otherwise go for Premium plan which will provide autoscaling capability of your compute environment.
How does a Azure function differ from a ASP.NET Core Worker Service?
Does both of these cover the same use cases?
How do I decide which one to use?
Update:
Azure function is designed for azure. So when you use azure, or other azure services which are integrated with azure function, you should consider azure function, which could simplify your code and logic by the built-in features of azure function. But it will also have charges for azure function. And yes, if you don't use azure, just use asp.net core worker services.
Azure function are totally different from ASP.NET Core Worker Service.
The benefit of Azure function is that it supports a lot of triggers like azure blob trigger / azure event hub trigger, and is integrated with other azure services.
With these built-in features, it's easy to create a proper azure function to complete a proper task. For example, if you upload an image to azure blob storage, and then want to resize the image, you can easily create a blob trigger azure function with less code.
Worker services are the perfect use case for any background processing. And if you want to operation some azure services like azure blob / azure event hub etc. you may achieve this but need to do a lot of work.
At last, it depends on your use case to choose which one should be used, and select the simple / efficient one.
I think the answer Fred was looking for is, why would someone use Azure functions over Asp.net Core hosted service? And the answer isn't only because you have Azure account so everything is easy and integrated. It depends. I.e Traffic, cost, expertise, etc.
Actually, there are guidelines set by Microsoft when to use an Azure Functions versus let's say over a IHosted service running in a linux container hosted in AKS for example. and I'll post the link, but it's basically if your workload is mission critical or not. If you want the best performance, lowest cost, reliability a hosted dotnet core service running in Linux container would be an excellent alternative.
Here is the mission critical guidelines from Microsoft, hope this helps!
Design recommendations
Azure Functions:
Consider Azure Functions for simple business process scenarios which don't have the same stringent business requirements as business-critical system flows.
Low-critical scenarios can also be hosted as separate containers within AKS to drive consistency, provided affinity and anti-affinity requirements are fully considered when collocating containers on nodes.
Excerpt taken from here:
https://learn.microsoft.com/en-us/azure/architecture/framework/mission-critical/mission-critical-application-platform
We currently have a window service which send some notification emails to users after doing some processing on database(SQL database). Runs once in day.
We want to move this on azure cloud. One alternate is to put it on Azure VM as is. but I am finding some other best possible solution for that.
I study about recurring and on demand Web jobs but I am not sure is this is best solution.
Also is there any possibility to update configuration of service code in App.config without re-deploy the code of service on cloud. I means we can manage configuration from Azure portal.
Thanks in advance.
Update 11/4/2016
Since this was written, there are 2 additional features available in Azure that are both excellent choices depending on what functionality you need:
Azure Functions (which was based on the WebJobs described below): Serverless code that can be trigger/invoked in various ways, and has scaling support.
Azure Service Fabric: Microservice platform, with support for actor model, stateful and stateless services.
You've got 3 basic options:
Windows service running on VM
WebJob
Cloud service
There's a lot of information out there on the tradeoffs between these choices, but here's a brief summary.
VM - Advantages: you can move your service basically as it is without having to change much or any of your code. They also have the easiest connectivity with other resources in Azure (blob storage, virtual networks, etc). The disadvantage is you're giving up all the of PaaS advantages and are still stuck managing your own VM infrastructure
WebJob - Advantages: Multiple invocation options (queues, blobs, manually, queue receive loops, continuous while-loop style, etc), scheduled (would cover your case). Easy to deploy (can go with website, as a console app, automatically through Kudu), has some built in logging in Azure portal - and yes, to answer your question, you can alter the configuration in the portal itself for connection strings and app settings.
Disadvantages - you'll need to update code, you don't have access to underlying resources (if you need that), and more of something to keep in mind than a disadvantage - it uses the same resources as the webapp it's deployed with.
Web Jobs are the newest of the options, but at the same time appear to have active development going on to increase the functionality and usefulness.
Cloud Service - like a managed VM, has some deployment options, access to underlying VM if needed. Would require some code changes from your existing service.
There's nothing you've mentioned in your use case that makes me think a Web Job shouldn't be first thing you try.
(Edit: Troy Hunt has a great and relatively recent blog post illustrating most of the points I've mentioned about Web Jobs above: http://www.troyhunt.com/2015/01/azure-webjobs-are-awesome-and-you.html)
I'm trying to figure out a solution for recurring data aggregation of several thousand remote XML and JSON data files, by using Azure queues and WebJobs to fetch the data.
Basically, an input endpoint URL of some sort would be called (with a data URL as parameter) on an Azure website/app. It should trigger a WebJobs background job (or can it continuously running and checking the queue periodically for new work), fetch the data URL and then callback an external endpoint URL on completion.
Now the main concern is the volume and its performance/scaling/pricing overhead. There will be around 10,000 URLs to be fetched every 10-60 minutes (most URLs will be fetched once every 60 minutes). With regards to this scenario of recurring high-volume background jobs, I have a couple of questions:
Is Azure WebJobs (or Workers?) the right option for background processing at this volume, and be able to scale accordingly?
For this sort of volume, which Azure website tier will be most suitable (comparison at http://azure.microsoft.com/en-us/pricing/details/app-service/)? Or would only a Cloud or VM(s) work at this scale?
Any suggestions or tips are appreciated.
Yes, Azure WebJobs is an ideal solution to this. Azure WebJobs will scale with your Web App (formerly Websites). So, if you increase your web app instances, you will also increase your web job instances. There are ways to prevent this but that's the default behavior. You could also setup autoscale to automatically scale your web app based on CPU or other performance rules you specify.
It is also possible to scale your web job independently of your web front end (WFE) by deploying the web job to a web app separate from the web app where your WFE is deployed. This has the benefit of not taking up machine resources (CPU, RAM) that your WFE is using while giving you flexibility to scale your web job instances to the appropriate level. Not saying this is what you should do. You will have to do some load testing to determine if this strategy is right (or necessary) for your situation.
You should consider at least the Basic tier for your web app. That would allow you to scale out to 3 instances if you needed to and also removes the CPU and Network I/O limits that the Free and Shared plans have.
As for the queue, I would definitely suggest using the WebJobs SDK and let the JobHost (from the SDK) invoke your web job function for you instead of polling the queue. This is a really slick solution and frees you from having to write the infrastructure code to retrieve messages from the queue, manage message visibility, delete the message, etc. For a working example of this and a quick start on building your web job like this, take a look at the sample code the Azure WebJobs SDK Queues template punches out for you.
A very simple question:
Why would someone use the Azure Scheduler if Azure WebJobs are free?
I couldnt find any topic regarding "azure webjobs vs azure scheduler"
The main difference is that the webjob contains everything that the scheduler can do:
Scheduler can make HTTP calls
WebJob can do that and more (run SQL commands, etc)
The actual scheduling bits of WebJobs are built on top of the scheduler. When you set up a Web Job on a schedule under the hood it uses the scheduler to kick it off. WebJobs provides a nice little location to host the code that gets executed. In fact, if you create WebJobs for a web site look in the Scheduler on the portal and you'll see them listed there as well.
Also note that the scheduler could call out to other systems not running Azure. If you have something running in a Cloud Service that needs to be called regularly, or even if something was hosted elsewhere (another provider or on premises) the scheduler is where you can set that up.
Regarding the cost aspect, there is a free tier to the scheduler as well: http://www.windowsazure.com/en-us/pricing/details/scheduler/.
It's 2016. The below answers are no longer accurate.
WebJobs now also has a built-in scheduler and the schedule can be defined by a cron expression.
When publishing to Azure, you can choose if you want to have the WebJob sparked off by the Scheduler or by the WebJob internal scheduler.
Important Note: The Azure Scheduler has limits to frequency of either 1hr or 1min depending on if paid or not. For the internal scheduler however, your App Service requires Always On to keep on running and firing off the job. This Always On status may affect your pricing.
Continuous jobs are monitored, and if they exit they are re-executed. In this way they act more like "services" in your local machine. There is a module that monitors and keeps your app working. Always-ON is a feature that will help your site stay alive and hence, your webjobs to continuously run.
Scheduler is used to trigger the webjobs. It uses the scheduler user account (not the back-end account). This way you can move out of the free tier for scheduler, sign up to higher tiers to suit your needs. But essentially, all the scheduler is doing is hitting an https endpoint (which is public, but required your auth).
Triggered jobs (scheduled and on demand) are invoked by an https call. These calls are load balanced - much in the same way that a web app with many instances is load balanced. Continuous jobs run concurrently by default, but can be set to be a singleton.
For Webjobs starting from version 2, there is no reason to use an Azure Scheduler anymore. As a matter of fact, the Azure Portal already flags this functionality as (Legacy).
From WebJob SDK v2 additional triggers have been introduced and one of them is TimerTrigger, which works with CRON expressions to schedule executions. This execution mode does not need any additional Azure construct, you just need the webapp to be set as AlwaysOn to guarantee the webjob to run.
Another azure service that works with TimerTriggers is Azure Functions, which is built on top of the WebJob SDK that allows a serverless execution.