Let's assume a situation that there is a celery beat task that is scheduled to run every 15 minutes. Now a "celery beat process" schedules it as per the system clock and the "celery workers" consume from the queue in which "celery beat" produces those tasks. Now there can be a delay in the time that the worker picks up the task to be executed and the time that the task was intended to run. How do I get this trigger time?
For example let's say beat schedules a task every 15 minutes 8:00 AM, 8:15 AM, 8:30 AM, and so on. The worker picks up the 8:30 task at about 8:51 PM and thus if I try to get the time in the task code I will get 8:51 AM instead of 8:30 AM. Is there any parameter that can allow me to get this 8:30 AM trigger time from within the task?
Related
I’m considering pg-boss for running and distributing event-based jobs between the instances of the same service. One of my use cases, apart from event-based, is scheduled jobs. Some of them can take a while and continue running until it’s time to trigger the next invocation - e.g. a job is set to run every 5 minutes but it can take e.g. 8 to complete. In such case I need the system to realize that the previous run is still in progress and not trigger the same job while the previous invocation of it is still in progress, using the example of every 5 minutes and a job taking 8 minutes - I’d like sth like the following to happen:
13:00 job triggered
13:05 job still runs, system sees it and doesn’t trigger once more even though it’s time
13:08 job done
13:10 next job run triggered
Is there an elegant way to achieve it with pg-boss without implementing my own locking mechanism?
I have a Java/Camel application that has a timer.
I have the following expression for the timer:
0+0,30+0-17,19-23+++?
The timer runs every 30 minutes from 12AM to 5PM and then 7PM to 11PM
IT wants to do maintenance on the server every monday from 12:30AM to 3:30AM, so they want me to not run the job during this time.
How can I change this expression to prevent the job from running only on mondays during this time?
In my ANYLOGIC model there are certain services( some delays the agent from 10 to 15 minutes, others 4 to 8 hrs), using certain resources from a resource pool.
The resources (pool) are available as per a well defined time ( Available: entire week except sunday , 10 am to 1:30 pm and then 2:00 Pm to 6 pm.).
I can see that once a service starts it continues till it finishes itself even after the resource availability time is over.
For example:
A resource is available :entire week except sunday , 10 am to 1:30 pm and then 2:00 Pm to 6 pm.
A service( of 8 hrs delay) starts from 12:30 pm....once it starts, it gets continue till it get finished. Practically it shall release resource from 1:30 to 2 pm and also if the task is not over then it shall not continue beyond 6 pm as well, and shall stop the same and start next day(or next availability).
but it does continue once it starts till it gets finished.
kindly suggest the specific area to be targeted to code or any other option is available?
Define your resource-pool downtimes using a Downtime block. Tick it's "may preempt other tasks" as below:
NOTE: play around with preemption as it interacts with Seize-preemption, resource pool preemption and priorities. Start simple and add complexity only when you fully understand how things work under the hood
I have a cron job that should repeat every 3 hours and must land on 9:30 in US/NY time.
If I start the job at 0:30, and run it every 3 hours, will it run at 9:30 on daylight savings days?
The documentation is a bit vague on this: https://cloud.google.com/scheduler/docs/configuring/cron-job-schedules#daylight_savings_time. It uses wall clock time to start, but does it use wall-clock time for each offset as well?
Alternatively, could I start it using 9:30 as the trigger time, and then every 3 hours until 6:30am the next day?
As you mentioned, Cloud scheduler uses wall clock time, is expected that the executions changes according the wall time.
To run a job from 9:30am to 6:30am (next day) you need 2 Scheduler jobs
1st will be run every 3 hours starting 9:30 to 23:30
30 9/3 * * *
2nd will be run every 3 hours starting 0:30 to 6:30
30 0-6/3 * * *
I use this page when I have doubts about cron format
The best way to completely avoid any kind of issue with DST changes is to switch to a timezone unaffected by those, as the documentation states:
If your job requires a very specific cadence, you may want to consider choosing time zones that do not observe daylight savings time.
A good one would be UTC for example, and as the difference between NY(GMT -4) and UTC(GMT) is 4h you just need to add 4 to your desired starting time.
With that you are assuring that the scheduled hours do not get modified and whatever you are expecting to be run is done as it should.
Regarding the start time of the trigger, it does not matter whether it is 0:30 or 9:30, once you create it, it will run every X hours/mins/seconds as you will have it specified beforehand.
I want to limit the korn job to run between 10 pm to 12 am.Currently it was taking more time to complete than the time frame. Even if its not finished , it as to terminate.
You can write a cron job, that runs on 12 am. It can look for your job and kill it.
Without a cron job you can do the same with timeout.
Another approach is having your script to look at the clock and call some abort() function a few seconds before 12 am.
This solution relies on the assumption, that your script is in some long loop, and you can add a function call at a central point (or some central points) where the script will be every few seconds. Using timeout seems to be better, but now you can stop your script in a controlled way (and timeout is unaware of changes in the clock, the DST or rdate).