I'm developing an Azure Function that executes several operations in Dynamics 365 CRM.
I don't fully understand how Azure Functions concurrency works.
I have a Consumption Plan, my Azure Function has a function inside that is triggered by a Service Bus message.
When I tested it the first time, the service bus received around 200 messages and the app started processing a lot of messages at the same time, making a huge load of requests to dynamics 365 that couldn't handle them.
So in the Azure Portal I managed to set the max number of instances to 1, but still the function was processing many messages at one time.
What's the best way to set a limit to that?
Using maxConcurrentCalls in host.json?
Using maxConcurrentSessions in host.json?
Using WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT in the app configs?
Also, what's the difference between setting maxConcurrentCalls at 10, and 1 function instance or setting it at 5 with 2 function instances?
maxConcurrentCalls is the attribute configured in host.json for the Azure Functions Service Bus Trigger.
By default, the runtime of Functions processes multiple messages concurrently (default value - 16). Set maxConcurrentCalls to 1 for setting up to process only a single queue or topic message at a time by the runtime.
Also, maxConcurrentCalls is the max no. of concurrent calls to the callback that should be initiate per scaled instance.
maxConcurrentSessions - Maximum No. of Sessions handled concurrently per scaled instance.
This setting only applies for functions that receive a single message at a time.
For the requirement of only one message need to be processed at a time per instance than you can use above configuration in the host.json.
If your requirement is Singleton support for Functions to ensure only one function running at a time, then you need to configure this.
WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT
This setting has no default limit, which states the max no. of instances that the app can scale out to.
Not to Scale out the function and to run only in one instance, set WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT to 1
In addition to this setting, you need to set maxConcurrentCalls to 1.
Few References for more information:
Azure Function to process only single message at one time
Azure Functions - WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT and batchSize - how can I get the desired concurrency
azure servicebus maxConcurrentCalls totally ignored
Official documentation of host.json settings in Azure Function Service Bus Trigger explains about maxConcurrentCalls, maxConcurrentSessions
Official Doc explains about WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT
Related
Per this documentation, I am using the Azure Function consumption plan and am trying to limit the parallelism of one of the queue-triggered functions so that only one instance runs at a time:
{
"queues": {
"batchSize": 1
}
}
The queue is part of a Microsoft Storage Account, and isn't a service bus.
However, my problem is that the function is still being run in parallel if there are multiple items on the queue at once. I read in the fine print of the documentation above:
If you want to avoid parallel execution for messages received on one queue, you can set batchSize to 1. However, this setting eliminates concurrency only so long as your function app runs on a single virtual machine (VM). If the function app scales out to multiple VMs, each VM could run one instance of each queue-triggered function.
Since I am using a consumption plan, how do I know if the function app is running on a single VM or on multiple? How can I successfully limit this function's batch size to one?
In consumption plan, a single function app only scales up to a maximum of 200 instances. A single instance may process more than one message or request at a time though, so there isn't a set limit on number of concurrent executions.
Also, when you're using a Consumption plan, instances of the Azure Functions host are dynamically added and removed based on the number of incoming events.
As you want to limit the parallelism of one of the queue-triggered functions, so I suggest that you could use Azure App Service Plan to achieve it.
For more details, you could refer to this article.
I have an Azure function that is triggered every 1 second.
Every time it is triggered, it reads a batch of messages from a Service Bus queue and processes them.
It runs on App service plan which scales on the active messages in the queue.
However as the service plan scales out I do not see any increase in the throughput of the function.
Does the Time triggered Azure function scale as the increase in the instances of the App service plan?
No, Timer triggers are singletons. That means that at any given moment only one instance will be firing Function calls on timer.
Obviously, different Functions have independent invocations.
To scale processing of Service Bus messages you should use Service Bus Trigger directly, which can manage scaling for you.
So we just found out how we can set our Azure function that is listening to a service bus queue to run as Single Threaded which is great for the issue we were dealing with on one of our queues. However we have another queue in the same service bus where it is not hyper critical to process messages synchronously.
By editing our host.json we were able to set the MaxConcurrentCalls to 1 but obviously this now cascades down to both queue listener functions in the same project. Is there a way to separate these settings or would we have to have 2 separate function apps altogether?
Unfortunately today the host settings are shared for all functions in the app - so you would need to separate these two functions into two separate apps.
How does the concept of storage queue polling apply when an Azure Function is hosted under the consumption plan?
I get the principal of polling with classic hosted WebJob functions and I understand that the maximum polling interval of 1 minute can be overridden. However in the case of consumption plan hosting there is no app-level memory resident process, therefore I assume that Azure internals spin up a FunctionApp via some other trigger beyond my control.
The motivation for this question is that I am trying to understand typical E2E function invocation propagation delays when an Azure hosted WebApp adds a message to a storage queue. In my case the WebApp, StorageQueue and pre-compiled function DLL will run in the same Azure region.
I need to cap Azure Function invocation delays to under 10 seconds with an average of <3 seconds.
Unfortunately this isn't possible on the consumption plan with the current polling model, as we poll your trigger resource every 10s to determine if there are new events requiring a function instance to be loaded/started.
If your function app runs frequently enough that it always has active instances (a new queue message every 5 min, for example) you can get the invocation delays that you want, as the instances themselves handle the polling.
The worst case (no function instances running) is ~10s polling + ~5s instance startup time to process a new event.
The Problem:
So we are building a newsletter system for our app that must have a capacity to send 20k-40k emails up to several times a day.
Tools of Preference:
Amazon SES - for pricing and scalability
Azure Functions - for serverless compute to send emails
Limitations of Amazon SES:
Amazon SES Throttling having Max Send Rate - Amazon SES throttles sending via their services by imposing a max send rate. Right now, being out of the sandbox SES environment, our capacity is 14 emails/sec with 50K daily emails cap. But this limit can be increased via a support ticket.
Limitations of Azure Functions:
On a Consumption Plan, there's no way to limit the scale as to how many instances of your Azure Function execute. Currently the scaling is handled internally by Azure, and thus the function can execute between just a few to hundreds of instances.
From reading other post on Azure Functions, there seems to be "warm-up" period for Azure Functions, meaning the function may not execute as soon as it is triggered via one of the documented triggers.
Limitations of Azure Functions with SES:
The obvious problem would be Amazon SES throttling sending emails from Azure functions because the scaled execution of Azure Function that sends out an email will be much higher than allowed send rate for SES.
Due to "warm-up" period of Azure Function messages may end up being piled up in a queue before Azure Function actually starts processing them at a scale and sending out the email, thus there's a very high probability of hitting that send/rate limit.
Question:
How can we utilize sending emails via Azure Functions while still being under X emails/second limit of SES? Is there a way to limit how many times an Azure Function can execute per time frame? So like let's says we don't want more than 30 instances of Azure Function running per/second?
Other thoughts:
Amazon SES might not like continuous throttling of SES for a customer if the customer's implementation is constantly hitting up that throttling limit. Amazon SES folks, can you please comment?
Azure Functions - as per documentation, the scaling of Azure Functions on a Consumption Plan is handled internally. But isn't there a way to put a manual "cap" on scaling? This seems like such a common requirement from a customer's point of view. The problem is not that Azure Functions can't handle the load, the problem is that other components of the system that interface with Azure Functions can't handle the load at the massive scale at which Azure Functions can handle it.
Thank you for your help.
If I understand your problem correctly, the easiest method is a custom queue throttling solution.
Basically your AF just retrieve the calls for all the mailing requests, and then queue them into a queue system (say ServiceBus / EventHub / IoTHub) then you can have another Azure function that runs by x minute interval, which will pull a maximum of y messages and push it to SES. Your control point becomes that clock function, and since the queue system will help you to ensure you know your message delivery status (has sent to SES yet) and you can pop the queue once done, it would allow you to ensure the job is eventually processed.
You should be able to set the maxConcurrentCalls to 1 within the host.json file for the function; this will ensure that only 1 function execution is occurring at any given time and should throttle your processing rate to something more agreeable from AWS perspective in terms of sends per second:
host.json
{
// The unique ID for this job host. Can be a lower case GUID
// with dashes removed. When running in Azure Functions, the id can be omitted, and one gets generated automatically.
"id": "9f4ea53c5136457d883d685e57164f08",
// Configuration settings for 'serviceBus' triggers. (Optional)
"serviceBus": {
// The maximum number of concurrent calls to the callback the message
// pump should initiate. The default is 16.
"maxConcurrentCalls": 1,
...