I need to run 2 tasks on below condition
1st task will run every alternate Thursday at 3 AM
2nd task will run every day at 12 AM except on the days 1st job will run.
I'm new to cron job expressions but tried few approaches. I'm using Spark, Scala for creating tasks and Airflow for scheduling the task.
schedule_interval= '0 0 3 1-5,15-19,29-31 * THU'
I've tried above expression for first task in Airflow but it will fail if alternate Thursday will occur on 6-14, 20-28 For example in January 2022 Thursdays will come on 6th, 13th, 20th and 27th.
The other issue will come for example if alternate Thursday comes between 29-31 and next Thursday comes between 1-5 job will run for 2 consecutive Thursday.
I resolved the above use-case using below mentioned piece of code
def is_alternate_thursday(starting_day, date_to_check):
set_day = THURSDAY
if (date_to_check < starting_day) or (date_to_check.weekday() != set_day):
return False
base_week = starting_day.isocalendar()[1]
checked_week = date_to_check.isocalendar()[1]
return (checked_week - base_week) % 2 == 0
Based on the flag now we can run the task.
Related
I am trying to run a pipeline on a schedule where it runs on a certain weekday where n number of weekdays have already been elapsed? For example, I want to run it on Monday(weekday, may 16 2022) where we have already passed 10 weekdays in the month of May. Similarly for other months, I would like to run it on a Monday where we have crossed 10 weekdays before that Monday and so on.
I tried various ways but nothing worked the way I want. Is there a way to keep a count of weekdays while scheduling the airflow pipeline?
We have jobs that are scheduled to run 1 time per day - every day
We do maintenance every 3rd Sunday of the month.
Up until now every month we have manually adjusted the cron to make the job run a little later in the morning then after maintenance we reset to the desired schedule
I am trying to change cron so that we
run at 7:00am every day EXCEPT the third Sunday of the month
run at 9:00am only on the third Sunday of the month
the second item I am able to handle
0 13 15-21 * 0
however, the first has me stumped. I thought this would do the job but it will only execute this if the day is between 1-14 or 22-31 but what if the 15th is not Sunday - then it won't run.
0 11 1-14,22-31 * *
How do I tell cron to run a schedule EXCEPT the third Sunday of the month?
There is a large base of guidance on how to limit when a cron runs to a specific window but I haven't found much for how to EXCLUDE a cron from a specific window
******** UPDATE ********
I think I may have come up with an answer - not sure if it is the most efficient but
0 11 1-14,22-31 * 0
0 13 15-21 * 0
0 11 1-14,22-31 * 1-6
The above will
run at 11:00 UTC on Sunday if date is between 1-14 or 22-31
run at 13:00 UTC on Sunday if date is between 15-21 (3rd Sunday)
run at 11:00 UTC Monday through Saturday all month
If a cron job has different timing than others, then it best to just define it by itself rather than trying to combine, unless you put some code in your script to do what you actually want. Doing something in cron on some nth day of the month is a pretty well known problem. Most crontab man pages have this note:
Note: The day of a command's execution can be specified in the following two fields — 'day of month', and 'day of week'. If both fields are restricted (i.e., do not contain the "*" character), the command will be run when either field matches the crent time. For example,
"30 4 1,15 * 5" would cause a command to be run at 4:30 am on the 1st and 15th of each month, plus every Friday.
So it does OR between the day of the week and the day of the month, not an AND. I don't who ever thought this was helpful, but that's the way it is. You can see solutions at:
Run every 2nd and 4th Saturday of the month
you need something like (this assumes cron runs /bin/sh):
[ `date +\%w` -eq 6 ] && <command>
on your cron job line, the above is would restrict to running only on Saturday.
I have to schedule a job to run every 9th of the month. and if 9th falls in weekend then in should run on next monday
for example
feb 9th is sunday so it should run on feb 10th i.e. next working day.
This kind of logic won't be possible entirely from your crontab schedule. I'll also point out the possibility that Monday might not necessarily be the "next working day" (think of holidays for example) in which case you might want to account for that as well.
If you will always choose Monday when the 9th is on a weekend (regardless of that Monday being a real working day, you can schedule your cron job to run on the 9th, 10th, and 11th of every month. For example, this would run on the 9th/10th/11th at 8am every month:
* 8 9-11 * 0 your_script
After that, the logic to determine if tasks need to run will have to be handled within the script itself. If you're using Python for example:
import datetime
today = datetime.date.today()
weekday = today.weekday()
if weekday is 5 or weekday is 6:
print("it's the weekend, do nothing")
elif today.day is 9 or (today.day is not 9 and weekday is 0):
print("run the script on the 9th or the first Monday after the 9th")
else:
print("this should not happen")
(Note that the script would run on every single Monday with this logic, but if you set up the cron job only to run on the 9th/10th/11th of the month then it will work just fine)
I really hope this helps!
Goal
Create a Cron expression that will run a task at 2 pm and 4 am every day to run a Splunk alert
Except for only run the 2 pm task on Thursday (don't run the task a 4 am on Thursday).
Question
Is this an expression that can be represented in a single expression? (if so how).
Edited:
Agree with Simon, you can configure 2 separate cron schedules:
1st expression(skipping Thursday) - “At minute 0 past hour 4 and 14 on every day-of-week from Monday through Wednesday and every day-of-week from Friday through Sunday.”
0 4,14 * * 1-3,5-7
Cron expression for Thursday:
0 14 * * 4
You can't express that in Cron.
Suggest you go with 2 separate from expressions, both times on all days expect Thursday, and then a separate Cron task just for the 2pm task on Thursday
I'm trying to come up with a CRON expression that will allow me to schedule a quartz trigger to run on every Monday in a month except the first one.
References:
http://www.quartz-scheduler.org/documentation/quartz-2.x/tutorials/crontrigger.html
https://docs.oracle.com/cd/E12058_01/doc/doc.1014/e12030/cron_expressions.htm
CRON allows you to specify the nth occurrence of a day of the week easily. An expression for First Monday of the month would be:
0 5 0 ? * 2#1
Where the 2#1 represents the first Monday of the month (2 = day of the week, 1 being the nth occurrence)
However, if I try to do something like
0 5 0 ? * 2#2-2#5
OR
0 5 0 ? * 2#2,2#3,2#4,2#5
It complains with the message
Support for specifying multiple "nth" days is not implemented.
Does anyone know how to achieve this in CRON?
Where cron doesn't give you the expressiveness you desire(a), it's a simple matter to change the command itself to only execute under certain conditions.
For your particular case, you know that the first Monday of a month is between the first and seventh inclusive and subsequent Mondays must be on the eighth or later.
So use cron to select all Mondays but slightly modify the command to exclude the first one in the month:
# mm hh dom mon dow command
0 1 * * 1 [[ $(date +%u) -gt 7 ]] && doSomething
That job will run at 1am every Monday but the actual payload doSomething will only be executed if the day of the month is greater than seven.
Some people often opt for putting the test into the script itself (assuming it even is a script) but I'm not a big fan of that, preferring to keep all scheduling information in the crontab file itself.
(a) Don't be mistaken into thinking you can combine the day of week 1 and day of month 8-31 to do this. As per the man page, those conditions are OR'ed (meaning either will allow the job to run):
Commands are executed by cron when the minute, hour, and month of year fields match the current time, and when at least one of the two day fields (day of month, or day of week) match the current time
Combining those two will run the job on the first Monday and every single day from the eighth onwards.