I have a date in this format: "27 JUN 2011" and I want to convert it to 20110627
Is it possible to do in bash?
#since this was yesterday
date -dyesterday +%Y%m%d
#more precise, and more recommended
date -d'27 JUN 2011' +%Y%m%d
#assuming this is similar to yesterdays `date` question from you
#http://stackoverflow.com/q/6497525/638649
date -d'last-monday' +%Y%m%d
#going on #seth's comment you could do this
DATE="27 jun 2011"; date -d"$DATE" +%Y%m%d
#or a method to read it from stdin
read -p " Get date >> " DATE; printf " AS YYYYMMDD format >> %s" `date
-d"$DATE" +%Y%m%d`
#which then outputs the following:
#Get date >> 27 june 2011
#AS YYYYMMDD format >> 20110627
#if you really want to use awk
echo "27 june 2011" | awk '{print "date -d\""$1FS$2FS$3"\" +%Y%m%d"}' | bash
#note | bash just redirects awk's output to the shell to be executed
#FS is field separator, in this case you can use $0 to print the line
#But this is useful if you have more than one date on a line
More on Dates
note this only works on GNU date
I have read that:
Solaris version of date, which is unable
to support -d can be resolve with
replacing sunfreeware.com version of
date
On OSX, I'm using -f to specify the input format, -j to not attempt to set any date, and an output format specifier. For example:
$ date -j -f "%m/%d/%y %H:%M:%S %p" "8/22/15 8:15:00 am" +"%m%d%y"
082215
Your example:
$ date -j -f "%d %b %Y" "27 JUN 2011" +%Y%m%d
20110627
date -d "25 JUN 2011" +%Y%m%d
outputs
20110625
If you would like a bash function that works both on Mac OS X and Linux:
#
# Convert one date format to another
#
# Usage: convert_date_format <input_format> <date> <output_format>
#
# Example: convert_date_format '%b %d %T %Y %Z' 'Dec 10 17:30:05 2017 GMT' '%Y-%m-%d'
convert_date_format() {
local INPUT_FORMAT="$1"
local INPUT_DATE="$2"
local OUTPUT_FORMAT="$3"
local UNAME=$(uname)
if [[ "$UNAME" == "Darwin" ]]; then
# Mac OS X
date -j -f "$INPUT_FORMAT" "$INPUT_DATE" +"$OUTPUT_FORMAT"
elif [[ "$UNAME" == "Linux" ]]; then
# Linux
date -d "$INPUT_DATE" +"$OUTPUT_FORMAT"
else
# Unsupported system
echo "Unsupported system"
fi
}
# Example: 'Dec 10 17:30:05 2017 GMT' => '2017-12-10'
convert_date_format '%b %d %T %Y %Z' 'Dec 10 17:30:05 2017 GMT' '%Y-%m-%d'
Just with bash:
convert_date () {
local months=( JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC )
local i
for (( i=0; i<11; i++ )); do
[[ $2 = ${months[$i]} ]] && break
done
printf "%4d%02d%02d\n" $3 $(( i+1 )) $1
}
And invoke it like this
d=$( convert_date 27 JUN 2011 )
Or if the "old" date string is stored in a variable
d_old="27 JUN 2011"
d=$( convert_date $d_old ) # not quoted
It's enough to do:
data=`date`
datatime=`date -d "${data}" '+%Y%m%d'`
echo $datatime
20190206
If you want to add also the time you can use in that way
data=`date`
datatime=`date -d "${data}" '+%Y%m%d %T'`
echo $data
Wed Feb 6 03:57:15 EST 2019
echo $datatime
20190206 03:57:15
Maybe something changed since 2011 but this worked for me:
$ date +"%Y%m%d"
20150330
No need for the -d to get the same appearing result.
Related
I have a little problem here. I am trying to write a script shell that shows background processes, but at a certain date. The date is a positional parameter. I've searched on Internet about the date validation, but all the info I found got me confused. I'm fairly new at this so a little help would be appreciated.
How do I verify if the date parameter is in a valid format in my shell script?
you can use regex :
#! /bin/bash
isDateInvalid()
{
DATE="${1}"
# Autorized separator char ['space', '/', '.', '_', '-']
SEPAR="([ \/._-])?"
# Date format day[01..31], month[01,03,05,07,08,10,12], year[1900..2099]
DATE_1="((([123][0]|[012][1-9])|3[1])${SEPAR}(0[13578]|1[02])${SEPAR}(19|20)[0-9][0-9])"
# Date format day[01..30], month[04,06,09,11], year[1900..2099]
DATE_2="(([123][0]|[012][1-9])${SEPAR}(0[469]|11)${SEPAR}(19|20)[0-9][0-9])"
# Date format day[01..28], month[02], year[1900..2099]
DATE_3="(([12][0]|[01][1-9]|2[1-8])${SEPAR}02${SEPAR}(19|20)[0-9][0-9])"
# Date format day[29], month[02], year[1904..2096]
DATE_4="(29${SEPAR}02${SEPAR}(19|20(0[48]|[2468][048]|[13579][26])))"
# Match the date in the Regex
if ! [[ "${DATE}" =~ "^(${DATE_1}|${DATE_2}|${DATE_3}|${DATE_4})$" ]]
then
echo -e "ERROR - '${DATE}' invalid!"
else
echo "${DATE} is valid"
fi
}
echo
echo "Exp 1: "`isDateInvalid '12/13/3550'`
echo "Exp 2: "`isDateInvalid '12/11/20322'`
echo "Exp 3: "`isDateInvalid '12 01 2000'`
echo "Exp 4: "`isDateInvalid '28-02-2014'`
echo "Exp 5: "`isDateInvalid '12_02_2002'`
echo "Exp 6: "`isDateInvalid '12.10.2099'`
echo "Exp 7: "`isDateInvalid '31/11/2020'`
If I'm understanding correctly, you want to input a date and filter background processes by that input and you are looking for a way to validate the input using the shell?
If you can break out the year/month/day into a slash-separated format, you can use the date command to test if the date stamp is valid. This runs under bash, other shells may be different
bad
$ echo -n "year :"; read year; echo -n "month :"; read month; echo -n "day :"; read day; date -d "$year/$month/$day" || echo "Not a valid date"
year :9999
month :32
day :32
date: invalid date ‘9999/32/32’
Not a valid date
good
$ echo -n "year :"; read year; echo -n "month :"; read month; echo -n "day :"; read day; date -d "$year/$month/$day" || echo "Not a valid date"
year :2020
month :3
day :23
Mon Mar 23 00:00:00 CDT 2020
Given an input date, I want to write a bash function that will output the previous business day.
By this I mean the preceding weekday (Monday through Friday);
I don't need it to take holidays into account.
So, for example, given "Jan 2, 2018" the result should be "Jan 1, 2018"
(even though that is a holiday),
but given "Jan 1, 2018" the result should be "Dec 29, 2017"
(because Dec 30 and 31 were Saturday and Sunday).
I don't require any particular format;
just something that is human-readable and acceptable to date -d.
I have tried the following but the input date does not seem to be correctly taken into account:
function get_previous_busday()
{
DAY_OF_WEEK=`$1 +%w`
if [ $DAY_OF_WEEK -eq 0 ] ; then
LOOKBACK=-2
elif [ $DAY_OF_WEEK -eq 1 ] ; then
LOOKBACK=-3
else
LOOKBACK=-1
fi
PREVDATE=date -d "$1 $LOOKBACK day"
}
I want to apply it for today:
PREVDATE=$(get_previous_busday $(date))
echo $PREVDATE
and for yesterday:
PREVDATE=$(get_previous_busday (date -d "$(date) -1 day"))
echo $PREVDATE
But it is not working:
main.sh: line 3: Fri: command not found
main.sh: line 4: [: -eq: unary operator expected
main.sh: line 6: [: -eq: unary operator expected
main.sh: line 11: -d: command not found
main.sh: command substitution: line 20: syntax error near unexpected token `date'
main.sh: command substitution: line 20: `get_previous_busday (date -d "$(date) -1 day"))'
A function to do what you want is:
get_previous_busday() {
if [ "$1" = "" ]
then
printf 'Usage: get_previous_busday (base_date)\n' >&2
return 1
fi
base_date="$1"
if ! day_of_week="$(date -d "$base_date" +%u)"
then
printf 'Apparently "%s" was not a valid date.\n' "$base_date" >&2
return 2
fi
case "$day_of_week" in
(0|7) # Sunday should be 7, but apparently some people
# expect it to be 0.
offset=-2 # Subtract 2 from Sunday to get Friday.
;;
(1) offset=-3 # Subtract 3 from Monday to get Friday.
;;
(*) offset=-1 # For all other days, just go back one day.
esac
if ! prev_date="$(date -d "$base_date $offset day")"
then
printf 'Error calculating $(date -d "%s").\n' "$base_date $offset day"
return 3
fi
printf '%s\n' "$prev_date"
}
For example,
$ get_previous_busday
Usage: get_previous_date (base_date)
$ get_previous_busday foo
date: invalid date ‘foo’
Apparently "foo" was not a valid date.
$ get_previous_busday today
Fri, Nov 30, 2018 1:52:15 AM
$ get_previous_busday "$(date)"
Fri, Nov 30, 2018 1:52:51 AM
$ PREVDATE=$(get_previous_busday $(date))
$ echo "$PREVDATE"
Fri, Nov 30, 2018 12:00:00 AM
$ get_previous_busday "$PREVDATE"
Thu, Nov 29, 2018 12:00:00 AM
$ PREVPREVDATE=$(get_previous_busday "$PREVDATE")
$ printf '%s\n' "$PREVPREVDATE"
Thu, Nov 29, 2018 12:00:00 AM
$ get_previous_busday "$PREVPREVDATE"
Wed, Nov 28, 2018 12:00:00 AM
There are a couple of ways to get the offset needed to get back to a business day.
You can write a case statement:
case $dow in
0|7) backday=2;; # For Sunday (either named 0 or 7) go back 2 days
1) backday=3;; # For monday go back three (3) days.
*) backday=1;; # For the rest, just one day.
esac
You can use math:
backday=$(( ((dow%7)>1) ? 1 : (dow%7)+2 ))
Or a lookup array:
a=(0 1 2 3 4 5 6 7)
b=(2 3 1 1 1 1 1 2)
backday=${b[dow]}
For any alternative, you may use this (without error detection) function,
and some tests:
#!/bin/bash
get_previous_busday() { dow=$(date -d "$*" '+%w')
backday=$(( ((dow%7)>1) ? 1 : (dow%7)+2 ))
prev_date="$(date -d "$* -$backday day")"
printf '%s\n' "$prev_date"
}
get_previous_busday "$(date)"
get_previous_busday $(date -d "-1 day")
get_previous_busday $(date -d "-2 day")
get_previous_busday $(date -d "-3 day")
get_previous_busday $(date -d "-4 day")
Will print:
Fri Nov 30 10:03:45 UTC 2018
Fri Nov 30 10:03:45 UTC 2018
Fri Nov 30 10:03:45 UTC 2018
Thu Nov 29 10:03:45 UTC 2018
Wed Nov 28 10:03:45 UTC 2018
This question already has answers here:
Grep error due to expanding variables with spaces
(3 answers)
Closed 5 years ago.
Solution: My date variables were in the wrong format (day number and day of the week were flipped). I changed this, then used the if statement proposed by #PesaThe below instead of my test.
Original Post:
I am writing a bash script to run as part of my servers' daily maintenance tasking. This particular job is to search for entries in input_file matching yesterday's and today's time stamps. Here are my date variables.
today=$(date "+%a, %b %d %Y")
yesterday=$(date --date=yesterday "+%a, %b %d %Y")
Here are the declarations, which are exactly as they should be:
declare -- adminLogLoc="/opt/sc/admin/logs/"
declare -- adminLog="/opt/sc/admin/logs/201801.log"
declare -- today="Tue, Jan 02 2018"
declare -- yesterday="Mon, Jan 01 2018"
declare -- report="/maintenance/daily/2018-01-02_2.2.txt"
Here are some actual log entries like those I need output. These were found with grep $today $adminLog | grep error
Tue, 02 Jan 2018 14:38:50 +0000||error|WARNING|13|Query #2464 used to generate source data is inactive.
Tue, 02 Jan 2018 14:38:50 +0000||error|WARNING|13|Query #2468 used to generate source data is inactive.
Tue, 02 Jan 2018 14:38:50 +0000||error|WARNING|13|Query #2470 used to generate source data is inactive.
Tue, 02 Jan 2018 14:38:50 +0000||error|WARNING|13|Query #2474 used to generate source data is inactive.
Here is the if statement I am trying to run:
# Check for errors yesterday
if [ $(grep $yesterday $adminLog|grep "error") != "" ]; then
echo "No errors were found for $yesterday." >> $report
else
$(grep $yesterday $adminLog|grep "error") >> $report
fi
# Check for errors today (at the time the report is made, there
# probably won't be many, if any at all)
if [ $(grep $today $adminLog|grep "error") != "" ]; then
echo "No errors were found for $today." >> $report
else
$(grep $today $adminLog|grep "error") >> $report
fi
I have tried this several ways, such as putting double quotes around the variables in the test, so on. When I run the grep search in the command line after setting the variables, it works perfectly, but when I run it in the test brackets, grep uses each term (i.e. Tue, Jan... so on) as individual arguments. I have also tried
grep $yesterday $adminLog 2> /dev/null | grep -q error
if [ $? = "0" ] ; then
with no luck.
How can I get this test to work so I can input the specified entry into my log file? Thank you.
Could you please try following script and let me know if this helps you. This snippet will help to simply print the yesterday's and today's logs in case you want to take them into a output file or so, we could adjust it accordingly too then.
#!/bin/bash
today=$(date "+%a, %b %d %Y")
yesterday=$(date --date=yesterday "+%a, %b %d %Y")
todays=$(grep "$today" Input_file)
yesterdays=$(grep "$yesterday" Input_file)
if [[ -n $todays ]]
then
echo "$todays"
else
echo "no logs found for todays date."
fi
if [[ -n $yesterdays ]]
then
echo "$yesterdays"
else
echo "NO logs found for yesterday's date."
fi
This loop should print the dates of each day from 9/28 through 10/14, but it seems to get confused after the end of September. However, it eventually moves onto October. Is there something wrong with the syntax or date formatting or incrementation?
Here is the code in test2.sh:
#!/bin/sh
# this is meant to run on the data-science server, so uses GNU syntax
BASEDIR=$(dirname $0)
cd $BASEDIR
start_date=$(date -d 2015-09-28 +"%y%m%d")
end_date=$(date -d 2015-10-14 +"%y%m%d")
dateTs=$start_date
while [ $dateTs -le $end_date ]
do
date=$(date -d $dateTs +%Y-%m-%d)
printf '%s\n' $date
dateTs=$(($dateTs+1))
done
This is the result of running sh test2.sh:
2015-09-28
2015-09-29
2015-09-30
date: invalid date ‘150931’
date: invalid date ‘150932’
date: invalid date ‘150933’
... [leaving out a bunch more of these] ...
date: invalid date ‘150997’
date: invalid date ‘150998’
date: invalid date ‘150999’
date: invalid date ‘151000’
2015-10-01
2015-10-02
2015-10-03
2015-10-04
2015-10-05
2015-10-06
2015-10-07
2015-10-08
2015-10-09
2015-10-10
2015-10-11
2015-10-12
2015-10-13
2015-10-14
... I'd love to know what I'm doing wrong.
You can't use ordinary arithmetic on date strings, because most numbers are not valid dates. When you go past the end of a month, the day part will be outside the possible range of dates. And when you go past the end of the year, you'll have invalid months.
The date command allows you to specify increments. Use:
start_date=2015-09-28
end_date=2015-10-14
date=$start_date
while [[ $date <= $end_date ]]
do
printf '%s\n' $date
date=$(date -d "$date + 1 day" +%Y-%m-%d)
done
This question already has answers here:
Extract data from log file in specified range of time [duplicate]
(5 answers)
Closed 6 years ago.
Okay, So i have log files and I would like to search within specific ranges. These ranges will be different throughout the day. Below is a piece of a log file and this is the only piece I can show you, sorry work stuff. I am using the cat command if that matters.
Working EXAMPLE : cat /dir/dir/dir/2014-07-30.txt | grep *someword* | cut -d',' -f1,4,3,7
2014-07-30 19:17:34.542 ;; (p=0,siso=0)
The above gets me the info I need along with the time stamp, but shows all time ranges and that is what I would like to correct. Lets say I only want ranges of 18 to 20 in the first column of the time.
Actual --> 2014-07-30 19:17:34.542 ;; (p=0,siso=0)
Only range I am looking for --> [18-20]:00:00.000 ;; (p=0,siso=0)
I am not worried about the 00s as they can be any digit.
Thanks for looking. I have not used much in the way of scripting as you can tell from my example, but any help is greatly appreciated.
I have included a log file, the colons and commas are where they should be.
2014-07-30 14:33:19.259 ;; (p=0,ser=0,siso=0) IN ### Word:Numbers=00000,word=None something goes here and here (something here andhere:here also here:2222),codeword=8,codeword=0,Noideanumbers=00000000,something=something, ;;
Using awk:
logsearch() {
grep "$3" "$4" | awk -v start="$1" -v end="$2" '{split($2, a, /:/)} (a[1] >= start) && (a[1] <= end)'
}
# logsearch <START> <END> <PATTERN> <FILE>
logsearch 18 20 '*someword*' /dir/dir/dir/2014-07-30.txt
Or with only awk (possibly different pattern quoting requirements):
logsearch2 ()
{
awk -v start="$1" -v end="$2" -v pat="$3" '($0 ~ pat) {split($2, a, /:/)} ($0 ~ pat) && (a[1] >= start) && (a[1] <= end)' "$4"
}
Not having seen the original input data I'm guessing from your cut what's going on.
Will this give you something similar to your desired outcome?
awk -F, '/someword/ && $4 ~ /^(18|19|20)/{printf "%s %s %s %s\n", $1,$4,$3,$7}' /dir/dir/dir/2014-07-30.txt
That said: a bit of sample data typically goes a long way!
Edit1:
Given the input line you added to both your comment and the original post the following awk statement does what you're asking:
awk '/something/ && $2 ~ /^(18|19|20)/{printf "%s %s %s %s\n", $1,$2,$3,$4} /path/to/your/input_file
This is a very interesting question. The pure BASH solution offers quite a bit of flexibility in how you deal with or process the entries after you identify those responsive to the range of date/time of interest. The simplest way in BASH is simply to get your start-time and stop-time in seconds since epoch and then test each log entry to determine if it falls within that range and then -- do something with the log entry. The basic logic involved is relatively short. The width of the date_time field within the log can be set by passing the width as argument 4. Set the default dwidth as needed (currently 15 to match syslog and journalctl format. The only required argument is the logfile name. If no start/stop time is specified, it will find all entries:
## set filename, set start time and stop time (in seconds since epoch)
# and time_field width (number of chars that make up date in log entry)
lfname=${1}
test -n "$2" && starttm=`date --date "$2" +%s` || starttm=0
test -n "$3" && stoptm=`date --date "$3" +%s` || stoptm=${3:-`date --date "Jan 01 2037 00:01:00" +%s`}
dwidth=${4:-15}
## read each line from the log file and act on only those with
# date_time between starttm and stoptm (inclusive)
while IFS=$'\n' read line || test -n "$line"; do
test "${line:0:1}" != - || continue # exclude journalctl first line
logtm=`date --date "${line:0:$dwidth}" +%s` # get logtime from entry in seconds since epoch
if test $logtm -ge $starttm && test $logtm -le $stoptm ; then
echo "logtm: ${line:0:$dwidth} => $logtm"
fi
done < "${lfname}"
working example:
#!/bin/bash
## log date format len
# journalctl 15
# syslog 15
# your log example 23
function usage {
test -n "$1" && printf "\n Error: %s\n" "$1"
printf "\n usage : %s logfile ['start datetime' 'stop datetime' tmfield_width]\n\n" "${0//*\//}"
printf " example: ./date-time-diff.sh syslog \"Jul 31 00:15:02\" \"Jul 31 00:18:30\"\n\n"
exit 1
}
## test for required input & respond to help
test -n "$1" || usage "insufficient input."
test "$1" = "-h" || test "$1" = "--help" && usage
## set filename, set start time and stop time (in seconds since epoch)
# and time_field width (number of chars that make up date in log entry)
lfname=${1}
test -n "$2" && starttm=`date --date "$2" +%s` || starttm=0
test -n "$3" && stoptm=`date --date "$3" +%s` || stoptm=${3:-`date --date "Jan 01 2037 00:01:00" +%s`}
dwidth=${4:-15}
## read each line from the log file and act on only those with
# date_time between starttm and stoptm (inclusive)
while IFS=$'\n' read line || test -n "$line"; do
test "${line:0:1}" != - || continue # exclude journalctl first line
logtm=`date --date "${line:0:$dwidth}" +%s` # get logtime from entry in seconds since epoch
if test $logtm -ge $starttm && test $logtm -le $stoptm ; then
echo "logtm: ${line:0:$dwidth} => $logtm"
fi
done < "${lfname}"
exit 0
usage:
$ ./date-time-diff.sh -h
usage : date-time-diff.sh logfile ['start datetime' 'stop datetime' tmfield_width]
example: ./date-time-diff.sh syslog "Jul 31 00:15:02" "Jul 31 00:18:30"
Remember to quote your starttm and stoptm strings. Testing with 20 entries in logfile between Jul 31 00:12:58 and Jul 31 00:21:10.
test output:
$ ./date-time-diff.sh jc.log "Jul 31 00:15:02" "Jul 31 00:18:30"
logtm: Jul 31 00:15:02 => 1406783702
logtm: Jul 31 00:15:10 => 1406783710
logtm: Jul 31 00:15:11 => 1406783711
logtm: Jul 31 00:15:11 => 1406783711
logtm: Jul 31 00:15:11 => 1406783711
logtm: Jul 31 00:15:11 => 1406783711
logtm: Jul 31 00:18:30 => 1406783910
Depending on what you need, another one of the solutions may fit your needs, but if you need to be able to process or manipulate the matching log entries, it is hard to beat a BASH script.
You can pipe the results to grep again.
cat /dir/dir/dir/2014-07-30.txt | grep someword | cut -d',' -f1,4,3,7 \
| grep '^\d\d\d\d-\d\d-\d\d \(1[89]\|20\)'
I don't have enough reputation to comment, but as minopret suggested do one grep at a time.
Here is one of the solutions to get the 18-20 range:
grep ' 20: \| 17:\| 18:' filename.txt
I have found the answer in the form I was looking for:
cat /dir/dir/dir/2014-07-30.txt | grep *someword* | cut -d',' -f1,4,3,7 | egrep '[^ ]+ (2[0-2]):[0-9]'
The following command gets me all the information I need from the cut, and greps for the someword I need and with the egrep I can search the times I need.