Our function app has TimerTrigger-function with CRON expression */10 * * * * * (every 10 seconds), and periodically it seems like trigger stops to fire, only manual app restarting helps. We're sure that function wasn't fired because it should insert records into DB.
We're using .net6 (In-Process) and runtime v4, App Service Plan with 'Always On' enabled and disabled logging sampling.
Sample InvocationId: 74089037-5f8a-4f30-beca-0261aba58400
You can see gaps in a timeline chart from Application Insights (all executions for last 48 hours).
Related
I have a v2 function app written in C# that is deployed to azure. I have application insights monitoring set up to monitor it. I'm looking at the logs to try and diagnose some performance issues and I'm noticing a bunch of messages like this:
Host started (xyz ms)
I see one of these messages every hour + 4 minutes.
7/9/2019, 8:27:04 AM - TRACE
7/9/2019, 7:23:03 AM - TRACE
7/9/2019, 6:19:02 AM - TRACE
7/9/2019, 5:15:03 AM - TRACE
etc.
I have a function that runs on a trigger that I'm using to keep the function alive so I can avoid cold starts, which end up in really slow function calls when it first starts.
[FunctionName("KeepAlive")]
public void Run([TimerTrigger("30 */4 * * * *", RunOnStartup=true)]TimerInfo myTimer, ILogger log)
{
log.LogInformation("Keep Alive");
}
I thought that with this function running every 4 minutes it would prevent my function app from shutting down, but for some reason it is restarting every hour + four minutes. What am I doing wrong?
From the back-end logs of 9th and 10th July, there were no restarts.
All these functions and rest of the function executed successfully without a single failure.
Sta*****Function
Mo*****st
Physical*******List
We have detected that you are running with the default setting of logging sampling enabled for Application Insights. This could cause missing execution logs from your monitor logs.
Enable the application insights logging sampling might lead to:
Timer Trigger executions missing from your monitor logs
Other data log missing
You may just need to adjust the sampling settings to fit your particular monitoring scenario.
Please review this guidance to configure sampling.
Also runOnStartup is enabled. We recommend against setting runOnStartup to true in production.
The function will be invoked when the runtime starts. This might lead to unscheduled executions in the execution list below.
Please check here to disable runOnStartup configuration.
I have run into a situation similar to
Azure Function timer is running twice and when I log onto the Azure portal
but I the solution there hasn't worked for me so far.
My function (written in Node) runs on a timer (every 15 minutes). In Application Insights and in the Azure portal Monitor logs, I see a single invocation every 15 minutes, as expected (presumably due to sampling). However, the output is occurring twice (I'm writing a record to a Cosmos collection). When I watch the live logs for my function (in the portal, navigate to the function code and pull up Logs from the bottom of the screen) I can see that it is running twice, very close together. Sample logs below. The intervals overlap, so I don't think it's a CRON issue (but I'm no CRON expert).
I did consider runOnStartup. I'm not sure if that defaults to false so I set it to false and restarted. Same problem after restart. (I'm still suspicious of this because a similar function app on a one minute timer runs as expected).
I have also tried running the function from the portal manually. When I do this, it only runs once.
Providing the information requested on the Azure functions github wiki:
I am using Consumption Plan
Not sure how much info is needed - we use Azure DevOps pipelines and Kudu.
v2 (Preview) runtime
Not setting WEBSITE_TIME_ZONE
My CRON expression: 0 */15 * * * * My expectation about what it means: function should run every 15 minutes.
N/A
Providing two from the close-together starts -
Invocation id: 4e142315-60e3-420d-b71a-9990683ba5aa
Invocation datetime: 2019-04-18T17:45:00.0044464+00:00
Region: East US
Invocation id: 4c6f4e7a-1e9d-4278-b3c6-0a2b5310199c
Invocation datetime: 2019-04-18T17:45:00.0131739+00:00
Region: East US
Sample logging showing two overlapping invocations (actual log text edited slightly for simplicity):
2019-04-18T17:45:00.004 [Information] Executing 'Functions.unassignDriverPermits' (Reason='Timer fired at 2019-04-18T17:45:00.0044464+00:00', Id=4e142315-60e3-420d-b71a-9990683ba5aa)
2019-04-18T17:45:00.013 [Information] Executing 'Functions.unassignDriverPermits' (Reason='Timer fired at 2019-04-18T17:45:00.0131739+00:00', Id=4c6f4e7a-1e9d-4278-b3c6-0a2b5310199c)
2019-04-18T17:45:33.577 [Information] Log a thing about something
2019-04-18T17:45:33.577 [Information] Log a thing about id 0
2019-04-18T17:45:33.586 [Information] Executed 'Functions.unassignDriverPermits' (Succeeded, Id=4e142315-60e3-420d-b71a-9990683ba5aa)
2019-04-18T17:45:33.281 [Information] Log a thing about something
2019-04-18T17:45:33.282 [Information] Log a thing about id 0
2019-04-18T17:45:33.294 [Information] Executed 'Functions.unassignDriverPermits' (Succeeded, Id=4c6f4e7a-1e9d-4278-b3c6-0a2b5310199c)
I once had a similar issue with my function that ran daily at 12:00. RunOnStartup was set to true. The function was running on a consumption plan, and by running only daily the function deallocates and is restarted when triggered (in cold start). Perhaps this was the reason for the double invocation: once for the trigger and once because it was started). It was fixed by setting RunOnStartup to false.
Make sure the RunOnStartup is set to false.
Looks like the 2 invocation ids are from 2 different apps (prod/dev may be?)
Are you able to filter your logs based on App name to verify if it is the same trigger firing twice?
I have long running web job in AppService (around 1h).
AppService has "Always on" turned on.
It is initialized with:
var host = new JobHost(config);
host.Call(typeof(Functions).GetMethod("SyncUsers"));
host.Start();
Actual methods SyncUsers wrapped with attributes:
[Timeout("00:59:00", ThrowOnTimeout = true)]
[NoAutomaticTrigger]
Schedule is set with settings file settings.job:
{
"is_singleton": true,
"schedule": "0 0 */4 * * *"
}
Main issue is that in WebJobs Dashboard I see status "Never finished" in 90% of cases (or failed with exception - OK situation). Running time for such jobs is different: from 5 min to 30 mins. Logs just stopped at some moment without any exception or detailed message.
Another thing is that I can see that multiple jobs are running in the same time. So looks like singleton and schedule don't work (since job should run every 4h).
Also some jobs that have this status, displayed without running time, like this: "1 hour ago ( running time)" and I am not able to see logs or download them.
Anybody had such experience?
Thank you
Looks like you're trying to run an executable that never ends as a triggered WebJob, which has no chance of working. You need to either:
Use a continuous WebJob and rely on the WebJobs SDK for your timer
Use a plain console app deployed as a Scheduled WebJob. No need to use the SDK here. Just do what you need to do from your Main() and let it end.
I'd suggest #2 unless you have a specific need to use the WebJobs SDK.
I have a function App. type is Timetrigger. I have given the time trigger expression 0 0 * * * * , as my requirement it to run with 1 Hour interval. I have refereed the TimeTrigger Cheat Sheet
But Unfortunetly its triggering in each 5 min. Somehow it's not working. Help me regarding this.
target--> Run the function App in the interval of 1 Hour.
If you have changed your code, I suggest you could right click project>rebuild your project to try again. Or you could try to use expression like '0 0 */1 * * *'.
The result:
But not sure why its getting triggered 5 min while putting the code in Azure Portal.
The code in portal and Visual Studio is different. You could try to right click project>publish>choose Azure function service to publish your function to portal. Then check the schedule in function.json:
In Portal, you could also click '+' to create TimeTrigger in Azure function service directly:
I got a conclusion out of the Issue, Although you will update the Time trigger schedule in the Code level, this will not impact the Running schedule. What ever is there in the configuration ( go to Integrate tab of the function App & check the timer value). Code value changes not impacting the config values. Need to change the config values manually.
You can set the trigger time as a config value, something like this:
[FunctionName(nameof(TimerFunction))]
public static async Task Run(
[TimerTrigger("%schedule%")]
TimerInfo timerInfo,
TraceWriter log)
{}
and then define schedule in the Application Settings of your Function App in Azure portal:
schedule 0 */5 * * * * (in this case every five minutes)
I'm using crontab.guru for choosing triggering intervals
I have a timer function that runs every 12 hours that refreshes an oAuth token. For some reason the function run over and over many times resulting in about 3500 calls every 12 hours. Not sure if it is relevant but I have implemented the function in c# in Visual Studio and deployed to azure. My function does have an output binding to a azure blob for writing the refreshed token information. I thought originally that the problem was because I implemented the function as Async but the problem persists despite removing async.
Cron expression is wrong. I was using * * */12 * * * instead of 0 0 */12 * * *