I know you can add spending limit for your azure functions per month, But I need to find a way to limit number of executions for an Azure function per day. The Function I am developing is calling a 3rd party API where we have a limit of 25,000 calls per day. When we reach that limit we get a response "LIMIT_REACH". I want to be able to pause the azure function execution until 12AM the next day. I am using a storage Queue to trigger the Azure Function. I know an option is in the function.json. I can update ["disabled": false] But i will need to set it through programatically. Then I will have to trigger a process to turn on the function again.
Why not keep a flag, or a "next valid execution time" in TableStorage when you have hit the LIMIT_REACH response. Each time the function triggers, interrogate that time and either execute or abort. Update the flag / next execution time when you are able to re-hit that 3rd party API.
I've made an ask out to the Azure Functions team to introduce a "pause" button for Azure functions. You can see the discussion and possible implementations over here = https://github.com/Azure/azure-functions-host/issues/7888
Unfortunately there are not any apis to programmatically enable/disable an Azure function at present.
However, you could achieve this in a few ways:
First, upon receiving LIMIT_REACH, have the queue function modify its own function.json to set disabled true - this will trigger a restart after all currently executing functions finish.
Then, at the time you wish to re-enable processing, run a different function to update disabled: true to false:
Use a timer trigger with a schedule to run at midnight daily (0 0 0 * * *)
or
Use another queue and set the visibility time to schedule when the message becomes visible, upon which time you re-enable the function.
Why don't you use RateLimiter as a tool to limit Function executions? There are a lot of framework that do that. As an example, here is the one:
https://github.com/David-Desmaisons/RateLimiter
I hope that it'll help you!!!
Related
I have a firebase function that runs every 2 minutes. The problem is that sometimes it takes over 540sec. to finish. Hence two executions of the function occur which messes up things.
Is there a way to ensure that the function does not fire till a previous instance finishes?
I tried to handle it using a flag stored in firestore which was set to true when function would start running, and false when function would finish. However sometimes function execution times out hence the flag is never set to false, thereby stopping all future executions.
So how do I make sure that only one execution of the function is running at a time?
You can limit the number of instances using the runWith method and using the maxInstances parameter. Read more here.
By the way, why are your functions taking too long to execute? are you terminating them correctly? You can post relevant part of you code so we can see why or you can learn about how to terminate your function here
Is there a way to ensure that the function does not fire till a previous instance finishes?
No. You'll have to store some value in a database as you are doing now and terminate the function if an instance is active.
However sometimes function execution times out hence the flag is never set to false, thereby stopping all future executions.
Checkout Cloud Functions V2 (beta) or Cloud Run itself that can run up to 1 hour.
Also, if you know a function execution is going to take more than 540 seconds every time, it might be best to increase the interval between 2 invocations.
Lets say I want to have a TimerTrigger function app that executes every 10 seconds and prints an increasing count(1...2...3...),
how can I achieve this WITHOUT using environment variable?
You're already using an Azure Storage account for your function. Create a table within that storage account, and increment the counter there. This has the added benefit of persisting across function restarts.
Since you're using a TimerTrigger, it's implicit that there will only ever be one instance of the function running. If this were not the case, you could end up in a race condition with two or more instances interleaving to incorrectly increment your counter.
I suggest you look into Durable Functions. This is an extension for Azure Functions that allow state in your (orchestrator) functions.
In your case, you can have a single HTTP triggered starter function that starts a long running orchestrator function. The HTTP function passes the initial count value to the orchestrator function. You can use the Timer functionality of Durable Functions to have the orchestrator wait for the specified amount of time before continuing/restarting. After the timer expires, the count value is incremented and you can restart the orchestrator function with this new count value by calling the ContinueAsNew method.
This periodic work example is almost what you need I think. You still need to add the initial count to be read as the input, and increment it before the ContinueAsNew method is called.
If you need more details about Durable Functions, I have quite some videos that explain the concepts.
I have one Azure function that need to execute on multiple (n) dynamically configured CRON schedules with different sets of parameters. For example, Function1 needs to execute on the first of the month with parameter X=1. Function1 needs to also run quarterly at 5 pm Friday with X=2, etc... The schedules is perfect for CRON Strings. The schedules and parameters are configurable in an app as are the numbers scheduled instances. Scheduler https://azure.microsoft.com/en-us/services/scheduler/ has been deprecated so I can’t use that. Logic apps do not use CRON. is there an approach someone can suggest? So far all I have is one function that polls the various configurations, evaluates the CRON strings and kicks off Function1 as appropriate. I wanted to avoid polling.
You can implement is with azure function timer trigger and do not need to use azure scheduler.
Azure function timer trigger also run with cron expression, you can refer to this tutorial to know how to create a function with timer trigger. And this tutorial will help you know about the cron expression in azure function timer trigger.
By the way, azure function timer trigger can just schedule one cron expression, so if you want your function to execute on multiple cron expression, you need to create multiple azure functions with different cron expression.
Update:
Create multiple timer trigger functions with different cron expression, show as below example.
One function with its cron expression and with the X = 1.
Another function with its cron expression and with the X = 2.
You can create n timer trigger functions in your function app(with different cron expressions and different parameter(X) in it). And when you create this function app, you can choose "Consumption plan" for it, then you do not need to worry about how many functions in this function app and do not worry about resource management. You just need to pay for the time that your code runs.
I would like to have a function called on a timer (every X minutes) but I want to ensure that only one instance of this function is running at a time. The work that is happening in the function shouldn't take long, but if for some reason it takes longer than the scheduled timer (X minutes) I don't want another instance to start and the processes to step on each other.
The simplest way that I can think of would be to set a maximum execution time on the function to also be X minutes. I would want to know how to accomplish this in both the App Service and Consumption plans, even if they are different approaches. I also want to be able to set this on an individual function level.
This type of feature is normally built-in to a FaaS environment, but I am having the hardest time google-binging it. Is this possible in the function.json? Or also are there different ways to make sure that this runs only once?
(PS. I know I could this in my own code by wrapping the work in a thread with a timeout. But I was hoping for something more idiomatic.)
Timer functions already have this behavior - they take out a blob lease from the AzureWebJobsStorage storage account to ensure that only one instance is executing the timer function. Also, the timer will not execute while a previous scheduled execution is in flight.
Another roll-your-own possibility is to handle this with storage queues and visibility timeout - when the queue has finished processing, push a new queue message with visibility timeout to match the desired schedule.
I want to mention that the functionTimeout host.json property will add a timeout to all of your functions, but has the side effect that your function will fail with a timeout error and that function instance will restart, so I wouldn't rely on it in this case.
You can specify 'functionTimeout' property in host.json
https://github.com/Azure/azure-webjobs-sdk-script/wiki/host.json
// Value indicating the timeout duration for all functions.
// In Dynamic SKUs, the valid range is from 1 second to 10 minutes and the default value is 5 minutes.
// In Paid SKUs there is no limit and the default value is null (indicating no timeut).
"functionTimeout": "00:05:00"
There is a new Azure Functions plan called Premium (in public preview as of May 2019) that allows for unlimited execution duration:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale
It will probably end up the goto plan for most Enterprise scenarios.
I have one capture application, that performs the MP4 capture. i need to schedule this capture application to capture video of every 30 minutes (or some dynamic value).
I read the MSDN article for IReferenceClock::AdviseTime from article i am not sure but i assume will trigger event when end-time elapses. but it not seems to work . please advice me if my understanding is incorrect about it. Or is any other batter way to repeat schedule in 30 minutes to capture video??
Thanks
IReferenceClock::AdviseTime is what let's close schedule setting an event. Filters might take advantage of this internally as a part of streaming operation. For you, however, this methods is of no use. There is a number of ways to trigger an action every 30 minutes. On a running application you would typically use SetTimer + WM_TIMER API. If you want your app started every 30 minutes, Task Scheduler is here for you.