I have a firebase function that runs every 2 minutes. The problem is that sometimes it takes over 540sec. to finish. Hence two executions of the function occur which messes up things.
Is there a way to ensure that the function does not fire till a previous instance finishes?
I tried to handle it using a flag stored in firestore which was set to true when function would start running, and false when function would finish. However sometimes function execution times out hence the flag is never set to false, thereby stopping all future executions.
So how do I make sure that only one execution of the function is running at a time?
You can limit the number of instances using the runWith method and using the maxInstances parameter. Read more here.
By the way, why are your functions taking too long to execute? are you terminating them correctly? You can post relevant part of you code so we can see why or you can learn about how to terminate your function here
Is there a way to ensure that the function does not fire till a previous instance finishes?
No. You'll have to store some value in a database as you are doing now and terminate the function if an instance is active.
However sometimes function execution times out hence the flag is never set to false, thereby stopping all future executions.
Checkout Cloud Functions V2 (beta) or Cloud Run itself that can run up to 1 hour.
Also, if you know a function execution is going to take more than 540 seconds every time, it might be best to increase the interval between 2 invocations.
Related
Lets say I want to have a TimerTrigger function app that executes every 10 seconds and prints an increasing count(1...2...3...),
how can I achieve this WITHOUT using environment variable?
You're already using an Azure Storage account for your function. Create a table within that storage account, and increment the counter there. This has the added benefit of persisting across function restarts.
Since you're using a TimerTrigger, it's implicit that there will only ever be one instance of the function running. If this were not the case, you could end up in a race condition with two or more instances interleaving to incorrectly increment your counter.
I suggest you look into Durable Functions. This is an extension for Azure Functions that allow state in your (orchestrator) functions.
In your case, you can have a single HTTP triggered starter function that starts a long running orchestrator function. The HTTP function passes the initial count value to the orchestrator function. You can use the Timer functionality of Durable Functions to have the orchestrator wait for the specified amount of time before continuing/restarting. After the timer expires, the count value is incremented and you can restart the orchestrator function with this new count value by calling the ContinueAsNew method.
This periodic work example is almost what you need I think. You still need to add the initial count to be read as the input, and increment it before the ContinueAsNew method is called.
If you need more details about Durable Functions, I have quite some videos that explain the concepts.
Can the execution of an expressJS method be delayed for 30 days or more just by using setTimeout ?
Let's say I want to create an endpoint /sendMessage that send a message to my other app after a timeout of 30 days. Will my expressJS method execution will last long time enough to fire this message after this delay ?
If your server runs continuously for 30 days or more, then setTimeout() will work for that. But, it is probably not smart to rely on that fact that your server never, ever has to restart.
There are 3rd party programs/modules designed explicitly for this. If you don't want to use one of them, then what I have done in the past is I write each future firing time into a JSON file and I set a timer for it with setTimeout(). If the timer successfully fires, then I remove that time from the JSON file.
So, at any point in time, the JSON file always contains a list of times in the future that I want timers to fire for. Any timer that fires is immediately removed from the JSON file.
Anytime my server starts up, I read the times from the JSON file and reconfigure the setTimeout() for each one.
This way, even if my server restarts, I won't lose any of the timers.
In case you were wondering, the way nodejs creates timers, it does not cost you anything to have a bunch of future timers configured. Nodejs keeps the timers in a sorted linked list and the event loop just checks the time for the next timer to fire - the one at the front of the sorted list (the rest of the timers are not looked at until they get to the front of the sorted list). This means the only time it costs anything to have lots of future timers is when inserting a new timer into the sorted list and there is no regular cost in the event loop to having lots of pending timers present.
I'm trying to build a workflow system, which will process a series of tasks & delays. Delay can be changed or removed from a running workflow.
What is the better way to making longer delay inside a series of tasks? (Like 3-4 months). Right now two ways are pocking around my head:
Pre-calculating & saving delay time. Setup a scheduler that will check delay repeatedly after a specific interval(1 minute maybe). This will make a lot of database queries, but the delay can be changed instantly.
Schedule a job for a delay. This can reduce a lot of database queries &, but the problem is maintaining & changing delay in these long-running jobs. Also, these jobs need to survive a server crash or restart.
Right now I'm not sure how to do it in a better way and still studying about it. If anyone has a similar experience, please share.
You can store the tasks into the database, like :
{
_id: String,
status: Enum,
executionTime: timestamp,
}
When you declare a new task, push a new entry into the DB.
At your server start, or when a new task is declared, create a setTimeout that will wake up your node.js when it's necessary.
Optimization
To avoid having X setTimeout, with X the number of task to execute. Keep only one setTimeout, with the time to wait equals to the closest task to execute.
For example, you have three task, one must run in 1 hour, one in 2 hour and one in 3 hour. Use a setTimeout of 1 hour. When it get triggered, it execute the task 1 and then look at the remaining tasks to re-run.
I would like to have a function called on a timer (every X minutes) but I want to ensure that only one instance of this function is running at a time. The work that is happening in the function shouldn't take long, but if for some reason it takes longer than the scheduled timer (X minutes) I don't want another instance to start and the processes to step on each other.
The simplest way that I can think of would be to set a maximum execution time on the function to also be X minutes. I would want to know how to accomplish this in both the App Service and Consumption plans, even if they are different approaches. I also want to be able to set this on an individual function level.
This type of feature is normally built-in to a FaaS environment, but I am having the hardest time google-binging it. Is this possible in the function.json? Or also are there different ways to make sure that this runs only once?
(PS. I know I could this in my own code by wrapping the work in a thread with a timeout. But I was hoping for something more idiomatic.)
Timer functions already have this behavior - they take out a blob lease from the AzureWebJobsStorage storage account to ensure that only one instance is executing the timer function. Also, the timer will not execute while a previous scheduled execution is in flight.
Another roll-your-own possibility is to handle this with storage queues and visibility timeout - when the queue has finished processing, push a new queue message with visibility timeout to match the desired schedule.
I want to mention that the functionTimeout host.json property will add a timeout to all of your functions, but has the side effect that your function will fail with a timeout error and that function instance will restart, so I wouldn't rely on it in this case.
You can specify 'functionTimeout' property in host.json
https://github.com/Azure/azure-webjobs-sdk-script/wiki/host.json
// Value indicating the timeout duration for all functions.
// In Dynamic SKUs, the valid range is from 1 second to 10 minutes and the default value is 5 minutes.
// In Paid SKUs there is no limit and the default value is null (indicating no timeut).
"functionTimeout": "00:05:00"
There is a new Azure Functions plan called Premium (in public preview as of May 2019) that allows for unlimited execution duration:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale
It will probably end up the goto plan for most Enterprise scenarios.
I have an issue with a timer function not always running as desired in Matlab.
I am trying to run an external simulation then fire a watcher function which checks the sim has exited correctly after a set time, if not it will force kill the task so it can be started again.
However the function works correctly for the first 2 batches of simulations then ceases to run. I am restarting the timer and using single shot. So each time the sim exits the timer is restarted. The timer callback is the function to kill the sim and the docs say 'Callback functions execute code during some event' which suggests the function should run regardless of any other matlab code. Is this correct or do callbacks work in some other way?
I initially thought this to be an issue with the execution queue holding the matlab worker for the sim, however it doesn't explain why it works for the first 100 or so times.
To be able to solve my issue I need more information on how the matlab workers work with regards to timer call backs. I am aware that unless specified matlab works on a single execution queue but I do not understand how callbacks function in this environment.
Any information regarding when and how timer call backs execute (ie do they fire exactly when timed to do so or do they simply get added to the queue at that time?) and any way of forcing a function to run regardless of the current worker would be greatly appreciated.
I have tried using batch() instead of a timer call back but the shell commands do not seem to work in a batch function (I have no idea why).
Many thanks