Given an input date, I want to write a bash function that will output the previous business day.
By this I mean the preceding weekday (Monday through Friday);
I don't need it to take holidays into account.
So, for example, given "Jan 2, 2018" the result should be "Jan 1, 2018"
(even though that is a holiday),
but given "Jan 1, 2018" the result should be "Dec 29, 2017"
(because Dec 30 and 31 were Saturday and Sunday).
I don't require any particular format;
just something that is human-readable and acceptable to date -d.
I have tried the following but the input date does not seem to be correctly taken into account:
function get_previous_busday()
{
DAY_OF_WEEK=`$1 +%w`
if [ $DAY_OF_WEEK -eq 0 ] ; then
LOOKBACK=-2
elif [ $DAY_OF_WEEK -eq 1 ] ; then
LOOKBACK=-3
else
LOOKBACK=-1
fi
PREVDATE=date -d "$1 $LOOKBACK day"
}
I want to apply it for today:
PREVDATE=$(get_previous_busday $(date))
echo $PREVDATE
and for yesterday:
PREVDATE=$(get_previous_busday (date -d "$(date) -1 day"))
echo $PREVDATE
But it is not working:
main.sh: line 3: Fri: command not found
main.sh: line 4: [: -eq: unary operator expected
main.sh: line 6: [: -eq: unary operator expected
main.sh: line 11: -d: command not found
main.sh: command substitution: line 20: syntax error near unexpected token `date'
main.sh: command substitution: line 20: `get_previous_busday (date -d "$(date) -1 day"))'
A function to do what you want is:
get_previous_busday() {
if [ "$1" = "" ]
then
printf 'Usage: get_previous_busday (base_date)\n' >&2
return 1
fi
base_date="$1"
if ! day_of_week="$(date -d "$base_date" +%u)"
then
printf 'Apparently "%s" was not a valid date.\n' "$base_date" >&2
return 2
fi
case "$day_of_week" in
(0|7) # Sunday should be 7, but apparently some people
# expect it to be 0.
offset=-2 # Subtract 2 from Sunday to get Friday.
;;
(1) offset=-3 # Subtract 3 from Monday to get Friday.
;;
(*) offset=-1 # For all other days, just go back one day.
esac
if ! prev_date="$(date -d "$base_date $offset day")"
then
printf 'Error calculating $(date -d "%s").\n' "$base_date $offset day"
return 3
fi
printf '%s\n' "$prev_date"
}
For example,
$ get_previous_busday
Usage: get_previous_date (base_date)
$ get_previous_busday foo
date: invalid date ‘foo’
Apparently "foo" was not a valid date.
$ get_previous_busday today
Fri, Nov 30, 2018 1:52:15 AM
$ get_previous_busday "$(date)"
Fri, Nov 30, 2018 1:52:51 AM
$ PREVDATE=$(get_previous_busday $(date))
$ echo "$PREVDATE"
Fri, Nov 30, 2018 12:00:00 AM
$ get_previous_busday "$PREVDATE"
Thu, Nov 29, 2018 12:00:00 AM
$ PREVPREVDATE=$(get_previous_busday "$PREVDATE")
$ printf '%s\n' "$PREVPREVDATE"
Thu, Nov 29, 2018 12:00:00 AM
$ get_previous_busday "$PREVPREVDATE"
Wed, Nov 28, 2018 12:00:00 AM
There are a couple of ways to get the offset needed to get back to a business day.
You can write a case statement:
case $dow in
0|7) backday=2;; # For Sunday (either named 0 or 7) go back 2 days
1) backday=3;; # For monday go back three (3) days.
*) backday=1;; # For the rest, just one day.
esac
You can use math:
backday=$(( ((dow%7)>1) ? 1 : (dow%7)+2 ))
Or a lookup array:
a=(0 1 2 3 4 5 6 7)
b=(2 3 1 1 1 1 1 2)
backday=${b[dow]}
For any alternative, you may use this (without error detection) function,
and some tests:
#!/bin/bash
get_previous_busday() { dow=$(date -d "$*" '+%w')
backday=$(( ((dow%7)>1) ? 1 : (dow%7)+2 ))
prev_date="$(date -d "$* -$backday day")"
printf '%s\n' "$prev_date"
}
get_previous_busday "$(date)"
get_previous_busday $(date -d "-1 day")
get_previous_busday $(date -d "-2 day")
get_previous_busday $(date -d "-3 day")
get_previous_busday $(date -d "-4 day")
Will print:
Fri Nov 30 10:03:45 UTC 2018
Fri Nov 30 10:03:45 UTC 2018
Fri Nov 30 10:03:45 UTC 2018
Thu Nov 29 10:03:45 UTC 2018
Wed Nov 28 10:03:45 UTC 2018
Related
I have a little problem here. I am trying to write a script shell that shows background processes, but at a certain date. The date is a positional parameter. I've searched on Internet about the date validation, but all the info I found got me confused. I'm fairly new at this so a little help would be appreciated.
How do I verify if the date parameter is in a valid format in my shell script?
you can use regex :
#! /bin/bash
isDateInvalid()
{
DATE="${1}"
# Autorized separator char ['space', '/', '.', '_', '-']
SEPAR="([ \/._-])?"
# Date format day[01..31], month[01,03,05,07,08,10,12], year[1900..2099]
DATE_1="((([123][0]|[012][1-9])|3[1])${SEPAR}(0[13578]|1[02])${SEPAR}(19|20)[0-9][0-9])"
# Date format day[01..30], month[04,06,09,11], year[1900..2099]
DATE_2="(([123][0]|[012][1-9])${SEPAR}(0[469]|11)${SEPAR}(19|20)[0-9][0-9])"
# Date format day[01..28], month[02], year[1900..2099]
DATE_3="(([12][0]|[01][1-9]|2[1-8])${SEPAR}02${SEPAR}(19|20)[0-9][0-9])"
# Date format day[29], month[02], year[1904..2096]
DATE_4="(29${SEPAR}02${SEPAR}(19|20(0[48]|[2468][048]|[13579][26])))"
# Match the date in the Regex
if ! [[ "${DATE}" =~ "^(${DATE_1}|${DATE_2}|${DATE_3}|${DATE_4})$" ]]
then
echo -e "ERROR - '${DATE}' invalid!"
else
echo "${DATE} is valid"
fi
}
echo
echo "Exp 1: "`isDateInvalid '12/13/3550'`
echo "Exp 2: "`isDateInvalid '12/11/20322'`
echo "Exp 3: "`isDateInvalid '12 01 2000'`
echo "Exp 4: "`isDateInvalid '28-02-2014'`
echo "Exp 5: "`isDateInvalid '12_02_2002'`
echo "Exp 6: "`isDateInvalid '12.10.2099'`
echo "Exp 7: "`isDateInvalid '31/11/2020'`
If I'm understanding correctly, you want to input a date and filter background processes by that input and you are looking for a way to validate the input using the shell?
If you can break out the year/month/day into a slash-separated format, you can use the date command to test if the date stamp is valid. This runs under bash, other shells may be different
bad
$ echo -n "year :"; read year; echo -n "month :"; read month; echo -n "day :"; read day; date -d "$year/$month/$day" || echo "Not a valid date"
year :9999
month :32
day :32
date: invalid date ‘9999/32/32’
Not a valid date
good
$ echo -n "year :"; read year; echo -n "month :"; read month; echo -n "day :"; read day; date -d "$year/$month/$day" || echo "Not a valid date"
year :2020
month :3
day :23
Mon Mar 23 00:00:00 CDT 2020
This question already has answers here:
Grep error due to expanding variables with spaces
(3 answers)
Closed 5 years ago.
Solution: My date variables were in the wrong format (day number and day of the week were flipped). I changed this, then used the if statement proposed by #PesaThe below instead of my test.
Original Post:
I am writing a bash script to run as part of my servers' daily maintenance tasking. This particular job is to search for entries in input_file matching yesterday's and today's time stamps. Here are my date variables.
today=$(date "+%a, %b %d %Y")
yesterday=$(date --date=yesterday "+%a, %b %d %Y")
Here are the declarations, which are exactly as they should be:
declare -- adminLogLoc="/opt/sc/admin/logs/"
declare -- adminLog="/opt/sc/admin/logs/201801.log"
declare -- today="Tue, Jan 02 2018"
declare -- yesterday="Mon, Jan 01 2018"
declare -- report="/maintenance/daily/2018-01-02_2.2.txt"
Here are some actual log entries like those I need output. These were found with grep $today $adminLog | grep error
Tue, 02 Jan 2018 14:38:50 +0000||error|WARNING|13|Query #2464 used to generate source data is inactive.
Tue, 02 Jan 2018 14:38:50 +0000||error|WARNING|13|Query #2468 used to generate source data is inactive.
Tue, 02 Jan 2018 14:38:50 +0000||error|WARNING|13|Query #2470 used to generate source data is inactive.
Tue, 02 Jan 2018 14:38:50 +0000||error|WARNING|13|Query #2474 used to generate source data is inactive.
Here is the if statement I am trying to run:
# Check for errors yesterday
if [ $(grep $yesterday $adminLog|grep "error") != "" ]; then
echo "No errors were found for $yesterday." >> $report
else
$(grep $yesterday $adminLog|grep "error") >> $report
fi
# Check for errors today (at the time the report is made, there
# probably won't be many, if any at all)
if [ $(grep $today $adminLog|grep "error") != "" ]; then
echo "No errors were found for $today." >> $report
else
$(grep $today $adminLog|grep "error") >> $report
fi
I have tried this several ways, such as putting double quotes around the variables in the test, so on. When I run the grep search in the command line after setting the variables, it works perfectly, but when I run it in the test brackets, grep uses each term (i.e. Tue, Jan... so on) as individual arguments. I have also tried
grep $yesterday $adminLog 2> /dev/null | grep -q error
if [ $? = "0" ] ; then
with no luck.
How can I get this test to work so I can input the specified entry into my log file? Thank you.
Could you please try following script and let me know if this helps you. This snippet will help to simply print the yesterday's and today's logs in case you want to take them into a output file or so, we could adjust it accordingly too then.
#!/bin/bash
today=$(date "+%a, %b %d %Y")
yesterday=$(date --date=yesterday "+%a, %b %d %Y")
todays=$(grep "$today" Input_file)
yesterdays=$(grep "$yesterday" Input_file)
if [[ -n $todays ]]
then
echo "$todays"
else
echo "no logs found for todays date."
fi
if [[ -n $yesterdays ]]
then
echo "$yesterdays"
else
echo "NO logs found for yesterday's date."
fi
This loop should print the dates of each day from 9/28 through 10/14, but it seems to get confused after the end of September. However, it eventually moves onto October. Is there something wrong with the syntax or date formatting or incrementation?
Here is the code in test2.sh:
#!/bin/sh
# this is meant to run on the data-science server, so uses GNU syntax
BASEDIR=$(dirname $0)
cd $BASEDIR
start_date=$(date -d 2015-09-28 +"%y%m%d")
end_date=$(date -d 2015-10-14 +"%y%m%d")
dateTs=$start_date
while [ $dateTs -le $end_date ]
do
date=$(date -d $dateTs +%Y-%m-%d)
printf '%s\n' $date
dateTs=$(($dateTs+1))
done
This is the result of running sh test2.sh:
2015-09-28
2015-09-29
2015-09-30
date: invalid date ‘150931’
date: invalid date ‘150932’
date: invalid date ‘150933’
... [leaving out a bunch more of these] ...
date: invalid date ‘150997’
date: invalid date ‘150998’
date: invalid date ‘150999’
date: invalid date ‘151000’
2015-10-01
2015-10-02
2015-10-03
2015-10-04
2015-10-05
2015-10-06
2015-10-07
2015-10-08
2015-10-09
2015-10-10
2015-10-11
2015-10-12
2015-10-13
2015-10-14
... I'd love to know what I'm doing wrong.
You can't use ordinary arithmetic on date strings, because most numbers are not valid dates. When you go past the end of a month, the day part will be outside the possible range of dates. And when you go past the end of the year, you'll have invalid months.
The date command allows you to specify increments. Use:
start_date=2015-09-28
end_date=2015-10-14
date=$start_date
while [[ $date <= $end_date ]]
do
printf '%s\n' $date
date=$(date -d "$date + 1 day" +%Y-%m-%d)
done
I have a date in this format: "27 JUN 2011" and I want to convert it to 20110627
Is it possible to do in bash?
#since this was yesterday
date -dyesterday +%Y%m%d
#more precise, and more recommended
date -d'27 JUN 2011' +%Y%m%d
#assuming this is similar to yesterdays `date` question from you
#http://stackoverflow.com/q/6497525/638649
date -d'last-monday' +%Y%m%d
#going on #seth's comment you could do this
DATE="27 jun 2011"; date -d"$DATE" +%Y%m%d
#or a method to read it from stdin
read -p " Get date >> " DATE; printf " AS YYYYMMDD format >> %s" `date
-d"$DATE" +%Y%m%d`
#which then outputs the following:
#Get date >> 27 june 2011
#AS YYYYMMDD format >> 20110627
#if you really want to use awk
echo "27 june 2011" | awk '{print "date -d\""$1FS$2FS$3"\" +%Y%m%d"}' | bash
#note | bash just redirects awk's output to the shell to be executed
#FS is field separator, in this case you can use $0 to print the line
#But this is useful if you have more than one date on a line
More on Dates
note this only works on GNU date
I have read that:
Solaris version of date, which is unable
to support -d can be resolve with
replacing sunfreeware.com version of
date
On OSX, I'm using -f to specify the input format, -j to not attempt to set any date, and an output format specifier. For example:
$ date -j -f "%m/%d/%y %H:%M:%S %p" "8/22/15 8:15:00 am" +"%m%d%y"
082215
Your example:
$ date -j -f "%d %b %Y" "27 JUN 2011" +%Y%m%d
20110627
date -d "25 JUN 2011" +%Y%m%d
outputs
20110625
If you would like a bash function that works both on Mac OS X and Linux:
#
# Convert one date format to another
#
# Usage: convert_date_format <input_format> <date> <output_format>
#
# Example: convert_date_format '%b %d %T %Y %Z' 'Dec 10 17:30:05 2017 GMT' '%Y-%m-%d'
convert_date_format() {
local INPUT_FORMAT="$1"
local INPUT_DATE="$2"
local OUTPUT_FORMAT="$3"
local UNAME=$(uname)
if [[ "$UNAME" == "Darwin" ]]; then
# Mac OS X
date -j -f "$INPUT_FORMAT" "$INPUT_DATE" +"$OUTPUT_FORMAT"
elif [[ "$UNAME" == "Linux" ]]; then
# Linux
date -d "$INPUT_DATE" +"$OUTPUT_FORMAT"
else
# Unsupported system
echo "Unsupported system"
fi
}
# Example: 'Dec 10 17:30:05 2017 GMT' => '2017-12-10'
convert_date_format '%b %d %T %Y %Z' 'Dec 10 17:30:05 2017 GMT' '%Y-%m-%d'
Just with bash:
convert_date () {
local months=( JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC )
local i
for (( i=0; i<11; i++ )); do
[[ $2 = ${months[$i]} ]] && break
done
printf "%4d%02d%02d\n" $3 $(( i+1 )) $1
}
And invoke it like this
d=$( convert_date 27 JUN 2011 )
Or if the "old" date string is stored in a variable
d_old="27 JUN 2011"
d=$( convert_date $d_old ) # not quoted
It's enough to do:
data=`date`
datatime=`date -d "${data}" '+%Y%m%d'`
echo $datatime
20190206
If you want to add also the time you can use in that way
data=`date`
datatime=`date -d "${data}" '+%Y%m%d %T'`
echo $data
Wed Feb 6 03:57:15 EST 2019
echo $datatime
20190206 03:57:15
Maybe something changed since 2011 but this worked for me:
$ date +"%Y%m%d"
20150330
No need for the -d to get the same appearing result.
I'm trying to split a large log file, containing log entries for months at a time, and I'm trying to split it up into logfiles by date. There are thousands of line as follows:
Sep 4 11:45 kernel: Entry
Sep 5 08:44 syslog: Entry
I'm trying to split it up so that the files, logfile.20090904 and logfile.20090905 contain the entries.
I've created a program to read each line, and send it to the appropriate file, but it runs pretty slow (especially since I have to turn a month name to a number). I've thought about doing a grep for every day, which would require finding the first date in the file, but that seems slow as well.
Is there a more optimal solution? Maybe I'm missing a command line program that would work better.
Here is my current solution:
#! /bin/bash
cat $FILE | while read line; do
dts="${line:0:6}"
dt="`date -d "$dts" +'%Y%m%d'`"
# Note that I could do some caching here of the date, assuming
# that dates are together.
echo $line >> $FILE.$dt 2> /dev/null
done
#OP try not to use bash's while read loop to iterate a big file. Its tried and proven that its slow, and furthermore, you are calling external date command for every line of the file you read. Here's a more efficient way, using only gawk
gawk 'BEGIN{
m=split("Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec",mth,"|")
}
{
for(i=1;i<=m;i++){ if ( mth[i]==$1){ month = i } }
tt="2009 "month" "$2" 00 00 00"
date= strftime("%Y%m%d",mktime(tt))
print $0 > FILENAME"."date
}
' logfile
output
$ more logfile
Sep 4 11:45 kernel: Entry
Sep 5 08:44 syslog: Entry
$ ./shell.sh
$ ls -1 logfile.*
logfile.20090904
logfile.20090905
$ more logfile.20090904
Sep 4 11:45 kernel: Entry
$ more logfile.20090905
Sep 5 08:44 syslog: Entry
The quickest thing given what you've already done would be to simply name the files "Sep 4" and so on, then rename them all at the end - that way all you have to do is read a certain number of characters, no extra processing.
If for some reason you don't want to do that, but you know the dates are in order, you could cache the previous date in both forms, and do a string comparison to find out whether you need to run date again or just use the old cached date.
Finally, if speed really keeps being an issue, you could try perl or python instead of bash. You're not doing anything too crazy here, though (besides starting a subshell and date process every line, which we already figured out how to avoid), so I don't know how much it'll help.
A skeleton of script:
BIG_FILE=big.txt
# remove $BIG_FILE when the script exits
trap "rm -f $BIG_FILE" EXIT
cat $FILES > $BIG_FILE || { echo "cat failed"; exit 1 }
# sort file by date in place
sort -M $BIG_FILE -o $BIG_FILE || { echo "sort failed"; exit 1 }
while read line;
# extract date part from line ...
DATE_STR=${line:0:12}
# a new date - create a new file
if (( $DATE_STR != $PREV_DATE_STR)); then
# close file descriptor of "dated" file
exec 5>&-
PREV_DATE_STR=$DATE_STR
# open file of a "dated" file for write
FILE_NAME= ... set to file name ...
exec 5>$FILE_NAME || { echo "exec failed"; exit 1 }
fi
echo -- $line >&5 || { echo "print failed"; exit 1 }
done < $BIG_FILE
This script executes the inner loop 365 or 366 times, once for each day of the year, instead of iterating over each line of the log file:
#!/bin/bash
month=0
months=(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec)
for eom in 31 29 31 30 31 30 31 31 30 31 30 31
do
(( month++ ))
echo "Month $month"
if (( month == 2 )) # see what day February ends on
then
eom=$(date -d "3/1 - 1 day" +%-d)
fi
for (( day=1; day<=eom; day++ ))
do
grep "^${months[$month - 1]} $day " dates.log > temp.out
if [[ -s temp.out ]]
then
mv temp.out file.$(date -d $month/$day +"%Y%m%d")
else
rm temp.out
fi
# instead of creating a temp file and renaming or removing it,
# you could go ahead and let grep create empty files and let find
# delete them at the end, so instead of the grep and if/then/else
# immediately above, do this:
# grep --color=never "^${months[$month - 1]} $day " dates.log > file.$(date -d $month/$day +"%Y%m%d")
done
done
# if you let grep create empty files, then do this:
# find -type f -name "file.2009*" -empty -delete