I setup an Azure function which is set to run with a timer trigger every 10 minutes (CRON schedule is set to 0 */10 * * * *) . The function is running and everything works well, but suddenly after 3 days, the function is no longer invoked. When I restarted the service, it returns to normal and runs every 10 minutes. This problem happened 3 times.
Is there any explanation for it ?
Are you running on consumption plan or a dedicated web app?
I'm assuming this is dedicated web app, as consumption has a built in timeout of 5 min to avoid runaway functions.
You can set a timeout value for functions in host.json which will kill these runaway functions and restart automatically. You can also add verbose logging settings in host.json to help determine why these functions aren't completing.
Related
I have created a web scraping python script and its running fine on my local system it takes 30 mins.
But when I tried to put the script on GCP cloud function it threw timeout after 60004 ms.
2022-03-16T11:41:01.420461007Zget_training_databudock8slftb Function execution took 60004 ms, finished with status: 'timeout'
Function execution took 60004 ms, finished with status: 'timeout'
To complete this I used the following services.
Cloud scheduler -> Pub/Sub -> cloud function
Could you please suggest which GCP service should I pick to run the python script with cost-effective and which runs daily?
Function execution time is limited by the timeout duration, which you can specify at function deployment time. By default, a function times out after 1 minute or 60000ms which you're experiencing, but you can extend this period up to a maximum of 9 minutes.
You can extend it by editing your deployed function and set the timeout to 540 seconds.
For more information, you may also refer on this documentation.
If your scraper took 30mins on your local, then maybe you need to optimize it first and logically set a timer before 9mins and create another schedule by using Google Cloud Client Libraries for google-cloud-scheduler to logically continue where you end and start on it until it finishes the scraping.
You can now use 2nd generation Cloud Functions.
If you create a 2nd gen Cloud Function and use an https trigger, you can set the max timeout to 3600 seconds = 1 hour instead of 10 minutes.
For info on 2nd generation Cloud Functions:
https://cloud.google.com/functions/docs/2nd-gen/overview
See also here on how to combine with Cloud Scheduler:
Do Google Cloud background functions have max timeout?
I have a timer trigger Function App ("version": "2.0") in azure which runs every 5 min. Cron Expression Used- 0 */5 * * * *
Its working as expected but sometimes it suddenly stops running. If I disable the function app and re-enable it, its starts working again.
If you see the screenshot below, It stopped working from 2021-04-14 16:54:59.998 to 2021-04-14 20:55:12.139
Any Help will be appreciated.
There could be different reasons for this issue and I will suggest you to review the below document to troubleshoot the issue and see if you are able to find the root cause.
Timer triggered function app uses TimerTriggerAttribute. This attribute consists of the Singleton Lock feature which ensures that only a single instance of the function is running at any given time. If any process runs longer than the scheduled timer, the new incoming process waits for the older process to finish and then uses the same instance. If you are using the same storage account across different timer trigger functions then this could be one of the reasons as mentioned here.
The other reason could be a restart and I will suggest you to check the Web App Restart detection section.
https://github.com/Azure/azure-functions-host/wiki/Investigating-and-reporting-issues-with-timer-triggered-functions-not-firing
https://github.com/Azure/azure-webjobs-sdk-extensions/wiki/TimerTrigger#troubleshooting
I need to develop a process (e.g. Azure fucntion app) that will load a file from FTP once every week, and perform ETL and update to other service for a long time (100mins).
My question is that will Timer Trigger Azure Function app with COMSUMPTION plan works in this scenario, given that the max running time of Azure function app is 10 mins.
Update
My theory of using Timer trigger function with Comumption plan is that if the timer is set to wake up every 4 mins from certain period (e.g. 5am - 10am Monday only), and within the function, a status tells whether or not an existing processing is in progress. If it is, the process continues its on-going job, otherwise, the function exits.
Is it doable or any flaw?
I'm not sure what is your exact scenario, but I would consider one of the following options:
Option 1
Use durable functions. (Here is a C# example)
It will allow you to start your process and while you wait for different tasks to complete, your function won't actually be running.
Option2
In case durable functions doesn't suit your needs, you can try to use a combination of a timer triggered function and ACI with your logic.
In a nutshell, your flow should looks something like this:
Timer function is triggered
Call an API to create the ACI
End of timer function.
The service in the ACI starts his job
After the service is done, it calls an API to remove it's own ACI.
But in anyway, durable functions usually do the trick.
Let me know if something is unclear.
Good luck. :)
With Consumptions plan, the azure function can run for max 10 minutes, still, you need to configure in host.json
You can go for the App Service Plan which has no time limit. Again you need to configure function timeout property in host.json
for more seed the following tutorial
https://sps-cloud-architect.blogspot.com/2019/12/azure-data-load-etl-process-using-azure.html
We have a couple of functions in a function app. Two of them are triggered by a timer, do some processing and write to queues to trigger other functions.
They normally work very well until recently where the timer trigger just stopped triggering. We fixed this by restarting the application which resolved the issue. The problem is that we were completely unaware of the trigger stopping as there were no failures and the function app is not constantly 'looked at' by our people.
I'd like to configure automatic monitoring and alerting for this special case. I configured Application Insights for the function app and tried to write an alert which watches the count metric of the functions which are triggered by a timer. If the metric is below the set threshold (below 1 in the last 5 minutes) the alert should be triggered.
I tested this by just stopping the function app. My reasoning behind this was that a function app that does not run should fullfill this condition and should trigger an alert within a reasonable time frame. Unfortunately this was not the case. Apparently a non-existing count is not measured and the alert will never be triggered.
Did someone else experience a similar problem and has a way to work around this?
I've added Application Insights alert:
Type: Custom log search
Search query:
requests | where cloud_RoleName =~ '<FUNCTION_APP_NAME_HERE>' and name == '<FUNCTION_NAME_HEER>'
Alert logic: Number of results less than 1
Evaluated based on: Over last N hours, Run every M hours
Alert fires if there are no launches over last N hours.
We have an Azure WebJob which is scheduled to run at 8:00 AM UTC daily(CRON - 0 00 08 * * *).Most of the days it is triggering correctly but on some days it is triggering twice (Second run is occurring ~10 secs after first run ). I can see in the web job history that when it triggered twice,first run's trigger property(from WebJob History JSON) is showing as "External - " and second run trigger property is showing as "Schedule - 0 0 8 * * *" but we don't have any external services triggering this WebJob.
When i checked the Job scheduler log for more details, "Web Job Invoked" status is only present for those days when the webjob got triggered twice.
Your problem appears to be that you appear to have two different things triggering your WebJob:
You probably have a settings.job (wiki) with a cron expression.
You may also have an Azure Scheduler Job Collection hitting your WebJob from the outside (possibly with a different schedule, which is why they don't always match).
Suggestion is to get rid of #2, and only keep the internal WebJobs scheduling via settings.job.