how schedule a job in airflow for first 4 business day (not on weekends) of the month - cron

job should run on first 4 business day(not on weekends) at 3 am
ex : 00 03 1-6 * 1-5 ==> jobs are running every day-of-week from Monday through Friday
1 - fri 03:00
4 - mon 03:00
5 - tue 03:00
6- wed 03:00

This is currently not possible with a single cron expression. You could configure "the first Monday-Friday of the month" using the # symbol:
0 3 * * MON#1,TUE#1,WED#1,THU#1,FRI#1
However, sometimes Friday is part of the 4 business days (e.g. Wednesday 1 Thursday 2 Friday 3 Saturday 4 Sunday 5 Monday 6 Tuesday 7), so you cannot simply exclude that.
Within Airflow, you currently have 2 options to work around this:
Implement a timetable
Run your DAG with the expression shown above, and start your DAG with a task to skip all remaining tasks in case the DAG run's date is a fifth working day.

Related

Cron periodicity works unclear with month days

I have expression 0 0 6/2 * *
it will run anything every second day starting from 6 day of month and in next month it can be earlier than 6 day? Example cron runs:
2022-06-06
2022-06-08
2022-06-10
...
2022-06-26
2022-06-28
2022-06-30
2022-07-02
2022-07-04
2022-07-06
2022-07-08
...
Does it work like this or cron run only from 6 day every month?
From here
At 12:00 AM, every 2 days, starting on day 6 of the month
So no, it will start on 6th day of month and then every 2 days

Cron job one schedule for 3rd Sunday but different schedule every other day

We have jobs that are scheduled to run 1 time per day - every day
We do maintenance every 3rd Sunday of the month.
Up until now every month we have manually adjusted the cron to make the job run a little later in the morning then after maintenance we reset to the desired schedule
I am trying to change cron so that we
run at 7:00am every day EXCEPT the third Sunday of the month
run at 9:00am only on the third Sunday of the month
the second item I am able to handle
0 13 15-21 * 0
however, the first has me stumped. I thought this would do the job but it will only execute this if the day is between 1-14 or 22-31 but what if the 15th is not Sunday - then it won't run.
0 11 1-14,22-31 * *
How do I tell cron to run a schedule EXCEPT the third Sunday of the month?
There is a large base of guidance on how to limit when a cron runs to a specific window but I haven't found much for how to EXCLUDE a cron from a specific window
******** UPDATE ********
I think I may have come up with an answer - not sure if it is the most efficient but
0 11 1-14,22-31 * 0
0 13 15-21 * 0
0 11 1-14,22-31 * 1-6
The above will
run at 11:00 UTC on Sunday if date is between 1-14 or 22-31
run at 13:00 UTC on Sunday if date is between 15-21 (3rd Sunday)
run at 11:00 UTC Monday through Saturday all month
If a cron job has different timing than others, then it best to just define it by itself rather than trying to combine, unless you put some code in your script to do what you actually want. Doing something in cron on some nth day of the month is a pretty well known problem. Most crontab man pages have this note:
Note: The day of a command's execution can be specified in the following two fields — 'day of month', and 'day of week'. If both fields are restricted (i.e., do not contain the "*" character), the command will be run when either field matches the crent time. For example,
"30 4 1,15 * 5" would cause a command to be run at 4:30 am on the 1st and 15th of each month, plus every Friday.
So it does OR between the day of the week and the day of the month, not an AND. I don't who ever thought this was helpful, but that's the way it is. You can see solutions at:
Run every 2nd and 4th Saturday of the month
you need something like (this assumes cron runs /bin/sh):
[ `date +\%w` -eq 6 ] && <command>
on your cron job line, the above is would restrict to running only on Saturday.

How to write a cronTab expression to run a script every second Monday in a month at specified time?

My instance is scheduled to run every 7 days in a month. I have two programs P1 and P2 of which P1 runs every Monday at 4PM and P2 runs every second Monday day at 00:10 midnight.
My instance crontab expression (to turn it on every 7th day is):
0 0 * * MON
My crontab expression for P1 is : (to run P1 at 16:00 hours every 7th Day)
30 16 * * MON
Now I am trying to figure out how to write crontab expression to run every second monday.
Any advise for the same?
Here is how my structure looks:
Today is 7th May : Friday
Next:
Machine On time: 10th May 00:00 (Monday)
P1 On time: 10th May 16:30 hrs
P2 On time: NA
Next:
Machine On time: 17th May 00:00 (Monday)
P1 On time: 17th May 16:30 hrs
P2 On time: 17th May 00:10 hrs
Next:
Machine On time: 24th May 00:00 (Monday)
P1 On time: 24th May 16:30 hrs
P2 On time: NA
and so on.
You can do something like this - 0 0 12 ? * 1#2. This cron job will run on the second Monday of the Month, at noon.
You can do something similar to this.
Or if you want to explore some more types, here are a few useful resources where you can find your answer -
https://crontab.cronhub.io/
https://www.freeformatter.com/cron-expression-generator-quartz.html
https://crontab.guru/

Does Cron have a way to include all except one case

Goal
Create a Cron expression that will run a task at 2 pm and 4 am every day to run a Splunk alert
Except for only run the 2 pm task on Thursday (don't run the task a 4 am on Thursday).
Question
Is this an expression that can be represented in a single expression? (if so how).
Edited:
Agree with Simon, you can configure 2 separate cron schedules:
1st expression(skipping Thursday) - “At minute 0 past hour 4 and 14 on every day-of-week from Monday through Wednesday and every day-of-week from Friday through Sunday.”
0 4,14 * * 1-3,5-7
Cron expression for Thursday:
0 14 * * 4
You can't express that in Cron.
Suggest you go with 2 separate from expressions, both times on all days expect Thursday, and then a separate Cron task just for the 2pm task on Thursday

Is there a Cron expression to schedule task after every 5 months across year boundaries?

I have an agenda job that I want to schedule for every five months. Suppose I started that job on Jan 20th, so now the schedule should be Jan 20th 2019, June 20th 2019, Nov 20th 2019, April 20th 2020 and so on.
Agenda uses cron for scheduling.
The problem with 00 00 20 1,6,11 * is that it will never run in April, this will run
at 2019-06-01 00:00:00
then at 2019-11-01 00:00:00
then at 2020-01-01 00:00:00
then at 2020-06-01 00:00:00
then at 2020-11-01 00:00:00.
Another expression that I used is 00 00 20 */5 *. The next run times are
at 2019-06-20 00:00:00
then at 2019-11-20 00:00:00
then at 2020-01-20 00:00:00
then at 2020-06-20 00:00:00
then at 2020-11-20 00:00:00
but they are not the month that I want it to run, i.e., at a regular interval of five months.
I couldn't find a way to start on the exact date in the month when it's started. If it is ok it's the first (like example below), you can use this:
Contab guru
“At 00:00 on day-of-month 1 in every 5th month.”
Otherwise play with the values to suit your needs.
Cron can't do this directly, but you can move that logic into a script that you call every month. Your cron job could look like
0 0 20 * * monthcheck 2019 1 5
and monthcheck is a script somewhere in your path with this content:
#!/usr/bin/env bash
baseyear=$1
basemonth=$2
interval=$3
read -r year month <<< "$(date '+%Y %-m')"
if (( (12*(year-baseyear) + (month-basemonth)) % interval == 0 )); then
echo "Run the script"
fi
This takes a base date specified by year and month (baseyear and basemonth, supplied as arguments with 2019 and 1 in the crontab entry) plus an interval in months (interval, value 5 in the example).
It then reads the current year and month into year and month, checks how many months have passed since the basedate, and if that difference is a multiple of interval (modulo division by the interval is 0), then it runs your script (just an echo command in this example).
The formatting string for date uses %-m to avoid zero padding for the month (gets 1 instead of 01). This feature might not be present in every date – I'm using GNU date here.

Resources