Is it possible to limit the maximum number of Functions that run in parallel?
I read the documentation and came across this:
When multiple triggering events occur faster than a single-threaded function runtime can process them, the runtime may invoke the function multiple times in parallel.
If a function app is using the Consumption hosting plan, the function app could scale out automatically. Each instance of the function app, whether the app runs on the Consumption hosting plan or a regular App Service hosting plan, might process concurrent function invocations in parallel using multiple threads.
The maximum number of concurrent function invocations in each function app instance varies based on the type of trigger being used as well as the resources used by other functions within the function app.
https://learn.microsoft.com/en-gb/azure/azure-functions/functions-reference#parallel-execution
I am using a Function on an App Service plan with an Event Hub input binding and only have a single Function within my Function App. If I can't limit it, does anyone know what the maximum number of concurrent function invocations will be for this kind of setup?
There isn't a way to specify a maximum concurrency for Event Hubs triggered functions, but you can control batch size and fetching options as described here.
The maximum number of concurrent invocations may also vary depending on your workload and resource utilization.
If concurrency limits are needed, this is (currently) something you'd need to handle, and the following posts discuss some patterns you may find useful:
Throttling Azure Storage Queue processing in Azure Function App
Limiting the number of concurrent jobs on Azure Functions queue
Just for reference, I came across here in my search for throttling. You can use the [Singleton] attribute on your function ensuring only one-at-a-time execution. Maybe not really what you were looking for and a very rigorous way of throttling, but still, it is an option.
https://learn.microsoft.com/en-us/azure/app-service/webjobs-sdk-how-to#singleton-attribute
Microsoft has added a new setting which can be used to limit concurrency of function execution. The setting is WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT and can be used to limit how many function instances will execute in parallel. However, according to Microsoft, it isn't fully implemented yet.
https://github.com/Azure/azure-functions-host/wiki/Configuration-Settings
For those who are still interested:
https://learn.microsoft.com/en-us/azure/azure-functions/event-driven-scaling#limit-scale-out
There's a way to limit the number of parallel execution by setting functionAppScaleLimit parameter.
Related
I have a Python Azure Function (Linux Consumption Plan) that is being set up to run multiple HttpTriggers at various times throughout the day. It's possible for more than one of these triggers to execute at or around the same time as the other triggers. To avoid exceeding the 1.5 GB memory limit, I'd like to make sure only one function invocation is allowed to run at a time. Is there any way to achieve this?
Edit: After doing a little research, would this setting allow me to avoid concurrent executions of my HttpTriggers: https://learn.microsoft.com/en-us/azure/azure-functions/functions-app-settings#website_max_dynamic_application_scale_out?
If I set WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT to 1, would that mean only one invocation could run at a time and the other HttpTriggers would wait?
Check out the full description of WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT here. It says that:
WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT sets a maximum number of instances that a function app can scale to.
This limit is not yet fully supported - it does work to limit your
scale out, but there are some cases where it might not be completely
foolproof. We're working on improving this.
I believe this is not what you are looking for.
Logically, this can be made possible if we choose timer trigger times in such a way that they never collide within a day (24hrs). Please note, this depends on the business requirements of function app HttpTriggers - what is the required frequency of them.
Another solution can be to have separate consumption plans for different HttpTriggers. Infact, in this case, you will get a monthly free grant of 1 million requests and 4,00,000 GB-s of resource consumption per month per subscription in pay-as-you-go pricing across all function apps in that subscription.
As mentioned above, there can be different solutions for this, you need to choose as per what suits you the best.
I'm running an Azure Function under the Consumption Plan. The Host Health Monitor is shutting it down every week or two due to hitting the Threads thresholds. I've read most of what I can find about this (e.g., starting with https://aka.ms/functions-thresholds) and understand what the health monitor is doing. Most of the articles I've read discuss the Connections threshold, not Threads. What I'm unable to figure out is how to troubleshoot this deeper and determine which part of my code is causing the threading issue. I'm not using any explicit TPL code. I'm doing a lot of very basic/standard async await operations against a REST API using a static HttpClient and standard crud operations against CosmosDB using a static DocumentClient. The only two Task-related methods I'm calling are async and await. I'm not using any retry patterns in this function. Application Insights doesn't give me much in the way of telling me what calls are creating threads, what threads still open or abandoned, etc.
Any suggestions on how I can troubleshoot things deeper? I've opened a support ticket with Microsoft and am waiting on their help as well.
Thanks in advance, Tom
I believe there are a couple of things to consider like
The number of functions in your function app
The number of async operations (2 in your case)
The number of parallel executions at a moment
Functions in the Consumption Plan scale based on the number of events (uses heuristics) that are triggered (doc) up to 200 instances without a limit on concurrent executions in a single instance.
So, even if you have async/await code, multiple long-running concurrent executions could cause problems I believe. Also, what triggers you use and at what rate they are triggered would influence the scale out.
To workaround the problem you could
Split the function app into multiple function apps if possible
Use Durable Functions which should allow you to scale your long running operations, if any
Is it possible to fix a cap on the number of servers on which the azure functions scale? I have a consumption plan and basically I would like to set a cap on the number of resources that azure functions can use.
The only solutions I found are:
set a cap for daily GB/sec threshold, after which the functions are stopped until the following day, which is definitely something I do not want because I need to use some functions for online tasks.
In the host.json, changing parameters for http.maxConcurrentRequests and http.maxOutstandingRequests, which will affect the number of concurrent functions running. Is this the thing should I look into? isn't this setting per-server level? my fear is that this won't end up capping resources, but insted will let Azure create just more and more servers, in order to comply with request loads
You can use the WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT app setting: The maximum number of instances that the function app can scale out to. Default is no limit.
Note: This setting is a preview feature - and only reliable if set to a value <= 5
Ref: https://learn.microsoft.com/en-us/azure/azure-functions/functions-app-settings#websitemaxdynamicapplicationscaleout
One thing to note is that timer-triggered functions are automatically singletons. In my case that was sufficient, as I can wake-up such function every minute and process specific amount of data. Even if the function takes longer than expected, there's no risk a second one will be run concurrently.
More info: https://stackoverflow.com/a/53919048/4619705
I have a service bus trigger function that when receiving a message from the queue will do a simple db call, and then send out emails/sms. Can I run > 1000 calls in my service bus queue to trigger a function to run simultaneously without the run time being affected?
My concern is that I queue up 1000+ messages to trigger my function all at the same time, say 5:00PM to send out emails/sms. If they end up running later because there is so many running threads the users receiving the emails/sms don't get them until 1 hour after the designated time!
Is this a concern and if so is there a remedy?
FYI - I know I can make the function run asynchronously, would that make any difference in this scenario?
1000 messages is not a big number. If your e-mail/sms service can handle them fast, the whole batch will be gone relatively quickly. Few things to know though:
Functions won't scale to 1000 parallel executions in this case. They will start with 1 instance doing ~16 parallel calls at the same time, and then observe how fast the processing goes, then maybe add a second instance, wait again etc.
The exact scaling behavior is not publicly described and can change over time. Thus, YMMV, and you need to test against your specific scenario.
Yes, make the functions async whenever you can. I don't expect a huge boost in processing speed just because of that, but it certainly won't hurt.
Bottom line: your scenario doesn't sound like a problem for Functions, but if you need very short latency, you'll have to run a test before relying on it.
I'm assuming you are talking about an Azure Service Bus Binding to an Azure Function. There should be no issue with >1000 Azure Functions firing at the same time. They are a Serverless runtime and should be able to scale greatly if you are running under a consumption model. If you are running the functions in a service plan, you may be limited by the service plan.
In your scenario you are probably more likely to overwhelm the downstream dependencies: the database and SMS sending system, before you overwhelm the Azure Functions infrastructure.
The best thing to do is to do some load testing, and monitor the exceptions coming out of the connections to the database and SMS systems.
We're trying to test scalability of Azure functions (it's a bear). We came across this https://azure.microsoft.com/en-in/documentation/articles/functions-reference/#parallel-execution
If a function app is using the Dynamic Service Plan, the function app
could scale out automatically up to 10 concurrent instances. Each
instance of the function app, whether the app runs on the Dynamic
Service Plan or a regular App Service Plan
Does this mean that the maximum scalability of a single function is just 10? we've never been able to get over 10 units running... (previous question on the algorithm to determine adding another consumption unit, this to determine the upper end of scalability).
Thanks
UPDATE: There is no official maximum number of instances. We see customers who are able to scale out to hundreds. The number you achieve depends mostly on your workload, but partly on the region you're running in (some regions have more capacity than others). The 10 instance limit mentioned in previous versions of the docs has been removed.
You can find more information about our consumption plan and scaling here: https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale#how-the-consumption-plan-works
Also note that each instance in Azure Functions can run multiple function executions in parallel. For example, if you have a function app which has a single function that runs quickly, you could expect to see dozens or even hundreds of concurrent executions on a single instance. This is unlike other services such as AWS Lambda which only execute a single function at a time per instance. New instances are added only when the system decides that the current number of instances is insufficient to handle the current load (more details on that in my answer to your other question).