I am working on a HttpTrigger based azure function and trying to figure out the scaling and cold start issues.
While looking into scaling, I found that the azure function documentation states that the
"instances of the Azure Functions host are dynamically added and removed based on the number of incoming events"
which has confused me as to how does the number of events determine the scaling out of function instances, as different function can be of different sizes in terms of how much compute power or memory they require to execute.
And where exactly can I find this "number of events" that supposedly triggers a new instance to be added?
You won't find a specific "number of events," it's based on a wide variety of factors that Microsoft measures to determine the load of the currently running instances. Functions that are grouped together in a single project and deployed as a single Function App on Azure scale together. If you need functions that consume different levels of resources to scale independently, then be sure to deploy them as separate Function Apps (in the C#/VS world, that means different Projects).
If you have cold start issues, then the Premium plan can come into play. You pay for at least one instance to always be on and "pre-warmed" so that you never have cold starts. The plan will then scale from there based on the previously mentioned "events" that Azure measures to determine if scaling out is needed. MS has said that scaling out tends to be faster on the Premium Plan. You also get a longer default function runtime on Premium if that is necessary (30min vs 5min).
Related
I'm currently using an Azure Premium Service App (P3v3). I found that my function needs some boost, so I decided to look at how I can configure scaling.
I found two available options:
Set FUNCTIONS_WORKER_PROCESS_COUNT in the configuration section
Configure the number of instances in Scale-Out menu
But what is the difference between instances and workers? Haven't found any information about that or how it can affect costs.
what is the difference between instances (Scale-Out menu) and workers (FUNCTIONS_WORKER_PROCESS_COUNT)?
These are Microsoft Documentations to understand Workers and Instances in terms of Cost Management (Billing) and Functionality:
MSFT Sources of FUNCTIONS_WORKER_PROCESS_COUNT:
Azure Functions - Functions App Settings - functions_worker_process_count
Azure Functions - Best Practices - FUNCTIONS_WORKER_PROCESS_COUNT
MSFT Sources of Azure Functions Premium Plan Scale-Out Instances:
Azure Functions Hosting options Information
Azure Functions Premium Plan Instances Billing
AFAIK, FUNCTIONS_WORKER_PROCESS_COUNT limits the max no. of worker processes per Function host instance. These instances are kind of separate VMs where the FUNCTIONS_WORKER_PROCESS_COUNT limit is applied to each of them individually.
For example, If the FUNCTIONS_WORKER_PROCESS_COUNT is set to 10, it means 10 Individual Functions concurrently run by each host instance.
Multiple Workers means Multiple Process Ids of the Same Function App which is a logical collection of Functions.
One Worker Process can host all functions of one Function App where the Single Host has the default value as 1 to the FUNCTIONS_WORKER_PROCESS_COUNT and the Function Host means it is the Physical/Virtual Host where Function App runs as Windows/Linux Process.
Refer here for more information on mechanism of FUNCTIONS_WORKER_PROCESS_COUNT.
how it can affect costs.
As this Microsoft Azure Services Pricing Calculator Says, you'll be charged per instance when you scale-out.
When it comes to Cost Management, the incremental nature of the Scale-Out methodology is extremely beneficial.
Cost increases should be somewhat predictable because the components are identical.
Scaling out also allows you to respond more quickly to changes in demand.
In most cases, services can be quickly added or deleted to satisfy resource requirements. By just using (and paying for) the resources required at the time, this flexibility and speed effectively minimize spending.
Refer to this article for more information on cost management and the benefits of Scale-Up and Scale-out.
When would I prefer Azure Functions to Azure Container Instances, considering they both offer the possibility to perform run-once tasks and they bill on consumption?
Also, reading this Microsoft Learn Module:
Serverless compute can be thought of as a function as a service (FaaS), or a microservice that is hosted on a cloud platform.
Azure Functions is a platform that allows you to run plain code (instead of containers). The strength of Azure Functions is the rich set of bindings (input- and output bindings) it supports. If you want to execute a piece of code when something happen (e. g. a blob was added to a storage Account, a timer gets triggered, ....) then I definitely would go with Azure Functions.
If you want to run some container-based workload for a short period of time and you don't have an orchestrator (like Azure Kubernetes Services) in place - Azure Container Instances makes sense.
Take a look at this from Microsoft doc
Source: https://learn.microsoft.com/en-us/dotnet/architecture/modernize-with-azure-containers/modernize-existing-apps-to-cloud-optimized/choosing-azure-compute-options-for-container-based-applications
If you would like to simplify application development model where your application architecture has microservices that are more granular, such that various functionalities are reduced typically to a single function then, Azure functions can be considered for usage.
In case, the solution needs some extension to existing azure application with event trigger based use cases , the azure functions can be better choice . Here, the specific code (function) shall be invoked only for specific event or trigger as per requirement and the function instances are created and destroyed on demand (compute on demand - function as a service (FaaS) ).
More often, the event driven architecture is seen in IoT where typically you can define a specific trigger that causes execution of Azure function. Accordingly, Azure functions have its place in IoT ecosystem as well.
If the solution has fast bursting and scaling requirement, then container Instances can be used whereas if the requirement is predictable scaling then, VMs can be used.
Azure function avoids allocation of extra resources (VMs) and also the cost is considered only when the function is processing work. Here, we need not take care of infrastructure such as where the code is going to execute, server configuration, memory etc. For ACI, the cost is per-second where it is accounted based on the time the container runs - CaaS(Container As A Service).
ACI enables for quickly spawning a container for performing the operation and deletion of it when done where the cost is only for few hours of usage rather than a dedicated VM which would be of high cost. ACI enables one to run a container by avoiding dependency on orchestrators like Kubernetes in scenarios where we would not need orchestration functions like service discovery, mesh and co-ordination features.
The key difference is that, in case of Azure function, the function is the unit of work whereas in container instance, the entire container contains the unit of work. So, Azure functions start and end based on event triggers whereas the microservices in containers shall get executed the entire time.
The processing / execution time also plays a critical role where if the event handler function consumes processing time of 10 minutes or more to execute, it is better to host in VM because the maximum timeout that is configurable for functions is 10 minutes.
There are typical solutions that utilize both the functionalities such that Azure function shall be triggered for minimal processing / decision making and inturn can invoke container instance for specific burst processing / complete processing.
Also, ACI along with AKS form a powerful deployment model for microservices where AKS can be for typical deployment of microservices and ACIs for handling the burst workloads thereby reducing the challenges in management of scaling and ensuring effective utilization of the per second usage cost model.
The Consumption Plan for Azure Functions is quite ideal, with its pleasant pricing and automatic scaling. However, I haven't found much information about High Availability with such a plan.
Let's consider a scenario. Imagine that, based on the load, there is currently one instance of the function app running. Then there is a problem in that data center. The consumption plan only scales out based on load. I can find no guarantees that a new instance will be added in this scenario, let alone that downtime will be prevented.
I'm aware that we could use Azure Front Door, with two separate function apps behind it. However, it appears that we must manage those function apps separately. That is a hassle. Swapping slots twice, remembering to change app settings in two places... That's no good.
What I'd like to achieve is something like Azure SQL in its Premium or Business-Critical tier, preferably with zone-redundant configuration. The diagram here shows how that works: https://learn.microsoft.com/en-us/azure/sql-database/sql-database-high-availability#zone-redundant-configuration
In simple terms, there is a primary replica with automatic failover to a secondary replica in the same data center, and also automatic failover to secondary replicas in two different data centers within the availability zone.
Notice how there is no manual management of the secondary replicas, since they are simply replicated from the primary.
How much of this can we achieve with Azure Functions, and how?
I have an azure function that reads jobs from a storage queue. It then executes these jobs and grabs more. I have been getting more jobs for it to run lately and noticed that the queue is building up.
What can I do from an Azure Perspective to get better performance out of this? Each job runs in its own little world so adding a new instance or adding threads or attaching to a "better" machine would all work fine.
Things come to mind with the information provided:
For more pure power: Host your Azure Function in a dedicated App Service plan instead of using the consumption plan. You can scale up (better hardware) or out (more hardware). Be aware that this could also be worse in theory. I would give it a try. Or try the "premium consumption plan" mentioned by Ken.
More parallelism: If your queue builds up even though you are not using most of your resources. Try playing with the configuration parameters batchSize and newBatchThreshold.
Changed execution logic: Depending where most of your time is spent during function execution, durable functions might help. Based on your comments you might also try to cache the external data using static or Azure Redis Cache.
Look at the most common performance considerations
Premium plan (Preview)
Azure Functions Premium plan provides customers the same features and scaling mechanism used on the Consumption plan (based on number of events) with enhanced performance and VNET access. Azure Functions Premium Functions plan is billed on a per second basis based on the number of vCPU-s and GB-s your premium functions consume.
In order to use the Azure Functions Premium Plan private preview your subscription needs to be added to an allowlist. Please apply for access via http://aka.ms/functionspremium.
More Info:
https://github.com/Azure/Azure-Functions/blob/master/functions-premium-plan/overview.md
I've created Azure function using Visual Studio as library project, have single project with 12 functions
e.g.
As of now my function hosted in Dedicated AppService Plan and one of function (EventHub listener) processes millions of message per-day, hence it utilizes 90%+ CPU all the time.
So we are planning to scale or Separate App Service plan for that single function. I see following possible solutions
Change dedicated app service plant to consumption. (not sure about pricing impact or SLA).
Create a new project, move single function in that project and deploy that function in a separate app service plan. (required code and CI/CD pipeline changes.)
Deploy same function dlls in 2 AppService Plan and remove functions according to scale. (not sure this is possible with Azure DevOps or not).
would like to know the better approach, if customer does not agree on consumption plan.
The advantage of a dedicated plan over a consumption plan is predictability. Running a workload on a consumption plan will usually work out faster and cheaper than running the same workload on a dedicated plan, but using a dedicated plan does prevent unexpected spikes in usage from leading to unexpected spikes in cost and also prevents cold start issues.
If you switch to the consumption model (option 1), you're paying for individual function execution so whether you run one or two function apps makes no difference from that point of view. You should decide how to split your functions based on how independent they are. If you'll only be deploying them as a single unit then stick to one function app, but if you may want to update the listener separately, put it in a separate project.
If you split into two apps running on separate dedicated service plans, creating a new project (option 2) is certainly the cleanest way to go about it. However, deploying the same project to both apps (option 3) is technically possible - you just need to disable the functions that shouldn't run on each app using app settings. Create settings named AzureWebJobs.MyFunction.Disabled with value true (see https://learn.microsoft.com/en-gb/azure/azure-functions/disable-function#functions-2x---all-languages).
wont help you, it scales all the functions in the function app
that will work
its the same as version, just with 2 app service plans, not one.
Better approach is - isolate this function into a single Azure Function App and then you can scale it independently.