I have an Azure Function that calls a web service hosted on a VM. Because of this dependency it is important that no more than 8 instances of the Azure Function are ever running concurrently (the VM only has 8 cores and each call to the web service uses a single core). If more than 8 instances are spawned then the web service calls will start to backup, items from the queue that is triggering the Azure Function will start timing out, and messages will be dropped. However, I also want to utilize the available resources as much as possible to maximize the processing of the queue so I'd like there to always be 8 instances of the Azure function running whenever there are 8 or more items in the queue.
In order to accomplish the required throttling I have set the Azure Function plan to run on an App Service plan instead of a Consumption plan and I have set the following values in the host.json for the Azure Function service:
{
"queues": {
"batchSize": 1,
"newBatchThreshold": 7
}
}
Theoretically this makes it so that as long as 7 or fewer instances of the function are running then 1 more message will be dequeued until there are 8 messages being processed. I have this running right now and it seems to be working, but I can't find anywhere in the host.json documentation or the WebJobs SDK documentation that says whether or not it is okay for batchSize to be less than newBatchThreshold. I only know that the recommendation is for newBatchThreshold to be half of batchSize (which is clearly not what I am doing).
My question is: Is this configuration okay? Or is there a better way to accomplish my throttling goals?
Yes, that is perfectly ok. Each flag has a precise (albeit kind of confusing) semantic, and any combination is valid and will honor the documented semantic.
Be aware that when using Consumption Azure Functions, you may end up getting scaled to multiple instances, each of which having those limits. If you want to avoid that try setting WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT to 1, per https://github.com/Azure/azure-webjobs-sdk-script/wiki/Configuration-Settings
Related
I have an Azure Function App inside an App Service Plan - Elastic Premium. The Function App is inside a VNet and it is a durable function app with some functions triggered by non-HTTP triggers:
I noticed the the Service Plan is not able to scale out properly, indeed the number of minimum instances is 4 and maximum instances 100
but checking the number of instances in the last 24 hours in:
"Function App -> Diagnose and solve problems -> Availability and performance -> HTTP Functions scaling"
the number of instances is never higher than 5/6 instances:
which is not good because if I check in:
"Function App -> Diagnose and solve problems -> Availability and performance -> Linux Load Average"
I have the following message:
"High Load Average Detected. Shows Load Average information if system has Load Average > 4 times # of Cpu's in the timeframe. For more information see Additional Investigation notes below."
and also checking the CPU and memory usage from the metrics I have some high spikes
so this means that my App Service plan is not able to scale out properly.
One of my colleague, suggested to verify that:
"Function App -> Configuration -> Function runtime settings -> Runtime Scale Monitoring"
is set to "on":
because if it is set to "off", may be that the VNet blocks Azure from diagnostic our app and as a result, Azure is not spawning more instances because is not seeing in real-time what CPU Load is.
But I didn't understand how this setting can help me to scale out.
Do you know what the "Runtime Scale Monitoring" is used for and why this can help me to scale out?
And also, do you think is more a problem related to scaling up instead of scaling out?
I assume that you are not using HTTP triggers, but instead something like ServiceBus. Also I don't think there is any "scaling up" in Consumption or Premium plans.
https://learn.microsoft.com/en-us/azure/azure-functions/functions-networking-options?tabs=azure-cli#premium-plan-with-virtual-network-triggers
When you enable virtual network trigger support, only the trigger types shown in the previous table scale dynamically with your application. You can still use triggers that aren't in the table, but they're not scaled beyond their pre-warmed instance count. For the complete list of triggers, see Triggers and bindings.
My understanding is that the setting will allow the scale controller to gain insight into what is triggering your function. So it is not scaling horizontally by looking at CPU usage, but rather by looking at the executed triggers to check if the messages are being processed fast enough.
https://learn.microsoft.com/en-us/azure/azure-functions/event-driven-scaling#runtime-scaling
For example, when you're using an Azure Queue storage trigger, it scales based on the queue length and the age of the oldest queue message.
If you disable the setting, the Scale Controller shown in the image will not have access to the queue length and you will not observe any scaling.
FUNCTIONS_WORKER_PROCESS_COUNT - This will Scale-up the Function Workers but not the hosts running.
It means if you set the value to X, then each host instance runs the X individual function worker processes concurrently.
This setting will scale up the host instance to run as many as Function Worker processes concurrently based on specified value.
Runtime Scale Monitoring requires the Contributor Access Level on both App Service Plan and Function App Level to enable the setting for Scaling Up/down operation performing.
I believe the above settings should need the pre-warmed instances to be at least set to 1.
For more information on how runtime scale controller works and cost optimization by enabling above settings, I have found the useful information in SO 1 and2 provided by the users #HariKrishna and #HuryShen.
Updated Answer:
In Azure Functions, an orchestrator function code can be either HTTP, Timer or any other Code Events, Sub-orchestrations, and activity functions.
To deal with the multiple orchestrator functions in parallel, you can use the below setting under extensions > durableTask > maxConcurrentOrchestratorFunctions to X Value in the host.json file. Refer to the JonasW Blog Article regarding how scaling happens in Azure Durable Functions.
I have a use case to implement multiple BlobTriggers in Azure Functions (using the Linux Consumption Plan). For example in Azure Storage I would have 5 different clients with a directory structure like:
client1/file.txt
client2.file.txt
client3/file.txt
client4/file.txt
client5/file.txt
It's possible for both client1/file.txt and client2/file.txt to be dropped off at the same time in Azure Storage. To prevent race conditions and exceeding the 1.5 GB memory limit, I would like the BlobTrigger for client1/file.txt to wait for the BlobTrigger for client2/file.txt to finish or vice versa (the order doesn't matter here, just that both of them eventually execute).
Do I have to set up a queue process separately? Can I use the preview setting WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUTto achieve this easily?
Edit: Would using durable functions be a better solution?
You should be able to do this by making sure the MAX SCALE OUT value is set to 1, this way it will only process 1 file at a time. You can also change your consumption\pricing model from consumption to app service plan one. This way you can use the tier you want, then you can have more memory available as well (depending on the tier you choose).
Is it possible to fix a cap on the number of servers on which the azure functions scale? I have a consumption plan and basically I would like to set a cap on the number of resources that azure functions can use.
The only solutions I found are:
set a cap for daily GB/sec threshold, after which the functions are stopped until the following day, which is definitely something I do not want because I need to use some functions for online tasks.
In the host.json, changing parameters for http.maxConcurrentRequests and http.maxOutstandingRequests, which will affect the number of concurrent functions running. Is this the thing should I look into? isn't this setting per-server level? my fear is that this won't end up capping resources, but insted will let Azure create just more and more servers, in order to comply with request loads
You can use the WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT app setting: The maximum number of instances that the function app can scale out to. Default is no limit.
Note: This setting is a preview feature - and only reliable if set to a value <= 5
Ref: https://learn.microsoft.com/en-us/azure/azure-functions/functions-app-settings#websitemaxdynamicapplicationscaleout
One thing to note is that timer-triggered functions are automatically singletons. In my case that was sufficient, as I can wake-up such function every minute and process specific amount of data. Even if the function takes longer than expected, there's no risk a second one will be run concurrently.
More info: https://stackoverflow.com/a/53919048/4619705
We are using azure function apps with consumption plan(integrated with service bus queues).
One of the app containing 4 functions and rest are having 1 to 2 functions.
We have set WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT setting for each app to control the instances.
Apps which contains 4 and 2 functions are having 190 instances to process 2.9k calls(WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT has been set to 100),
Even though this setting is preview feature it's working fine for some apps(3 to 9 instances to process 9.3k calls).
What I'm not able to understand is what may be the reason for having 190 instances? whether it is due to many functions in single app or it may due to any memory leak.
Is there anyone faced the same problem with azure function apps before.
My application host.json settings are :
"serviceBus": {
"maxConcurrentCalls": 5,
"prefetchCount": 50,
"autoRenewTimeout": "00:05:00"
}
Thanks.
For consumption plans, Azure Portal now has dedicated slider for Scale Out:
It maybe a known issue currently. For more information please refer to Azure function Host Configuration Settings.
Set a maximum number of instances that a function app can scale to. This limit is not yet fully supported - it does work to limit your scale out, but there are some cases where it might not be completely foolproof. We're working on improving this.
Is it possible to limit the maximum number of Functions that run in parallel?
I read the documentation and came across this:
When multiple triggering events occur faster than a single-threaded function runtime can process them, the runtime may invoke the function multiple times in parallel.
If a function app is using the Consumption hosting plan, the function app could scale out automatically. Each instance of the function app, whether the app runs on the Consumption hosting plan or a regular App Service hosting plan, might process concurrent function invocations in parallel using multiple threads.
The maximum number of concurrent function invocations in each function app instance varies based on the type of trigger being used as well as the resources used by other functions within the function app.
https://learn.microsoft.com/en-gb/azure/azure-functions/functions-reference#parallel-execution
I am using a Function on an App Service plan with an Event Hub input binding and only have a single Function within my Function App. If I can't limit it, does anyone know what the maximum number of concurrent function invocations will be for this kind of setup?
There isn't a way to specify a maximum concurrency for Event Hubs triggered functions, but you can control batch size and fetching options as described here.
The maximum number of concurrent invocations may also vary depending on your workload and resource utilization.
If concurrency limits are needed, this is (currently) something you'd need to handle, and the following posts discuss some patterns you may find useful:
Throttling Azure Storage Queue processing in Azure Function App
Limiting the number of concurrent jobs on Azure Functions queue
Just for reference, I came across here in my search for throttling. You can use the [Singleton] attribute on your function ensuring only one-at-a-time execution. Maybe not really what you were looking for and a very rigorous way of throttling, but still, it is an option.
https://learn.microsoft.com/en-us/azure/app-service/webjobs-sdk-how-to#singleton-attribute
Microsoft has added a new setting which can be used to limit concurrency of function execution. The setting is WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT and can be used to limit how many function instances will execute in parallel. However, according to Microsoft, it isn't fully implemented yet.
https://github.com/Azure/azure-functions-host/wiki/Configuration-Settings
For those who are still interested:
https://learn.microsoft.com/en-us/azure/azure-functions/event-driven-scaling#limit-scale-out
There's a way to limit the number of parallel execution by setting functionAppScaleLimit parameter.