This question already has answers here:
Filter log file entries based on date range
(5 answers)
Closed 5 years ago.
I need to find the entries from a huge log file which are 1 month older than the current date using shell scripts. The log entry date is in the 3rd field of each line.
This is my sample log file:
0114374510,OK,10/23/2017 9:22:50 AM,0016692200,OK
0112364510,OK,10/23/2017 9:22:50 AM,0016692200,OK
0112364510,TX,10/21/2017 9:10:00 AM,0016692200,OK
0115364510,TX,10/21/2017 10:52:00 AM,0016692200,OK
0112368979,OK,7/29/2016 10:25:00 AM,0000718374,OK
0113368979,OK,7/29/2016 12:50:00 PM,0000718374,OK
0112368979,TX,9/16/2015 10:57:00 AM,0033545820,OK
Thanks!
Here is a solution, using grep to match certain lines in your file and date to manipulate dates.
Linux version
grep -E "$(date --date='-1 month' '+%m')/[0-9]+/$(date --date='-1 month' +'%Y') <your_file>"
date --date='-1 month' '+%m' - Subtract 1 month to the current date and print the month
[0-9]+ - Regex matching any number (any day in our case)
$(date --date='-1 month' +'%Y') - Subtract 1 month to the current date and print the year
macOS version
grep -E "$(date -v -1m '+%m')/[0-9]+/$(date -v -1m +'%Y') <your_file>"
date -v -1m '+%m' - Subtract 1 month to the current date and print the month
[0-9]+ - Regex matching any number (any day in our case)
date -v -1m +'%Y' - Subtract 1 month to the current date and print the year
Related
I want the previous month & previous year in the format: "December2020".
But while executing the below Linux command I'm getting July2020.
echo `date -d "2021-01-08" '+%B' -d 'last month'``date -d '1 year ago' +%Y
The date hardcoded parameter is in format: YYYY-MM-DD (2021-01-08)
date accepts only one -d option. In your command date -d "2021-01-08" '+%B' -d 'last month' the first -d is ignored. Only the -d "last month" applies. Because of that, and since we have August right now, the output is July.
You probably wanted to use
date -d '2021-01-08 - 1 month' +%B%Y
which prints December2020.
If you really wanted to concat the previous month and previous year together, you could use
echo "$(date -d'2021-01-08 - 1 month' +%B)$(date -d'2021-01-08 - 1 year' +%Y)"
but that would give rather strange results:
2021-01-08 → December2020 # 1 month before input date
2021-04-30 → March2020 # 13 months before input date
I want a command line can display number of Monday(s) which fall(s) on the first of the month in a given year without using sed or awk commands
I have this command that display the first date of the current month
date -d "-0 month -$(($(date +%d)-1)) days"
With GNU date, you can read input from a file (or standard input):
printf '%s\n' 2021-{01..12}-01 | date -f- +%u | grep -c 1
This prints dates for the first of each month in a year, then formats them as "weekday" (where 1 is "Monday"), then counts the number of Mondays.
To parametrize the year, replace 2021 with a variable containing the year; wrapped in a function:
mondays() {
local year=$1
printf '%s\n' "$year"-{01..12}-01 | date -f- +%u | grep -c 1
}
Using a for loop, this can be accomplished as follows.
for mon in {01..12}; do date -d "2021-$mon-01" +%u; done | grep -c 1
Breakdown
We iterate through the numbers 01 to 12 representing the months.
We call date passing in the custom date value with the first date of each month in the year. We use +%u to return the day of week where 1 represents Monday.
Lastly we count the number of 1s using grep -c or grep --count
Note, the desired year has been hard coded as 2021. The current year can be used as:
for mon in {01..12}; do date -d "$(date +%Y)-$mon-01" +%u; done | grep -c 1
This can also all be put into a function and the desired year passed in as an argument:
getMondays() {
for mon in {01..12}; do date -d "$1-$mon-01" +%u; done | grep -c 1
}
I implemented it as:
for ((i=1,year=2021,mondays=0; i< 12; i++)) {
if [ $(date -d "$i/1/$year" +%u) -eq 1 ]
then
let "mondays++"
fi
}
echo "There are $mondays Mondays in $year."
That said, I like Mushfiq's answer. Quite elegant.
I have a log file abc.log in which each line is a date in date +%m%d%y format:
061019:12
062219:34
062319:56
062719:78
I want to see the all the logs between this date range (7 days before date to current date) i.e (from 062019 to 062719 in this case). The result should be:
062219:34
062319:56
062719:78
I have tried few things from my side to achieve:
awk '/062019/,/062719' abc.log
This gives me correct answer, but if i don't want to hard-code the date value and try achieving the same it does not give the correct value.
awk '/date --date "7 days ago" +%m%d%y/,/date +%m%d%y' abc.log
Note:
date --date "7 days ago" +%m%d%y → 062019 (7 days back date)
date +%m%d%y → 062719 (Current date)
Any suggestions how this can be achieved?
Your middle-endian date format is unfortunate for sorting and comparison purposes. Y-m-d would have been much easier.
Your approach using , ranges in awk requires exactly one log entry per day (and that the log entries are sorted chronologically).
I would use perl, e.g. something like:
perl -MPOSIX=strftime -ne '
BEGIN { ($start, $end) = map strftime("%y%m%d", localtime $_), time - 60 * 60 * 24 * 7, time }
print if /^(\d\d)(\d\d)(\d\d):/ && "$3$1$2" ge $start && "$3$1$2" le $end' abc.log
Use strftime "%y%m%d" to get the ends of the date range in big-endian format (which allows for lexicographical comparisons).
Use a regex to extract day/month/year from each line into separate variables.
Compare the rearranged date fields of the current line to the ends of the range to determine whether to print the line.
To get around the issue of looking for dates that may not be there, you could generate a pattern that matches any of the dates (since there are only 8 of them it doesn’t get too big, if you want to look for the last year it might not work as well):
for d in 7 6 5 4 3 2 1 0
do
pattern="${pattern:+${pattern}\\|}$(date --date "${d} days ago" +%m%d%y)"
done
grep "^\\(${pattern}\\)" abc.log
I have 2 script in bash, and i have some files:
transaction-2012-01-01.csv.bz2
transaction-2012-01-02.csv.bz2
transaction-2012-01-03.csv.bz2
transaction-2012-01-04.csv.bz2
.
.
transaction-2012-01-31.csv.bz2
transaction-2012-02-01.csv.bz2
.
.
transaction-2012-02-28.csv.bz2
I have a script called script.sh
cat script.sh
YEAR_MONTH=$1
FILEPATH="transaction-$YEAR_MONTH*.csv.bz2"
bzcat $FILEPATH|strings|grep -v "code" >> output
And if you need call the script you can use other script
cat script2.sh
LAST_MONTH=$(date -d -1month +%Y"-"%m)
if [ $# -eq 1 ]; then
DATE=$1
else
DATE=$LAST_MONTH
fi
script.sh $DATE 1>output$DATE.csv 2>> log.txt
And it do cat the files in a month, but now i need call the script with a specific week in a year:
bash script2.sh 2012-01
where 2012 is the year and 01 is the month
Now i need call the script with:
bash script2.sh 2012 13
where 2012 is the year and 13 is the week in a year
Now i need the cat only to the files in the year and week that the user specified, no per month per week
But the format of the files do not help me!!!! because the name is transaction-year-month-day.csv.bz2, not transaction-year-week.csv.bz2
Take a look at the manpage for strftime. These are date format codes. For example:
$ date +"%A, %B %e, %Y at %I:%m:%S %p"
Will print out a date like:
Thursday, May 30, 2013 at 02:05:31 PM
Try to see why this works.
On some systems, the date command will have a -j switch. This means, don't set the date, but reformat the given date. This allows you to convert one date to another:
$ date -f"$input_format" "$string_date" +"$output_format"
The $input_format is the format of your input date. $string_date is the string representation of the date in your $input_format. And, $output_format is the format you want your date in.
The first two fields are easy. Your date is in YY-MM-DD format:
$ date -f"%Y-%m-%d" "$my_date_string"
The question is what can you do for the final format. Fortunately, there is a format for the week in the year. %V which represents the weeks at 01-53 and %W which represents the weeks as 00-53.
What you need to do is find the date string on your file name, then convert that to the year and week number. If that's the same as the input, you need to concatenate this file.
find $dir -type f | while read transaction_file
do
file_date=${transaction_file#transaction-} #Removes the prefix
file_date=${file_date%.csv.bz2} #Removes the suffix
weekdate=$(date -j -f"%Y-%m-%d" "$file_date" +"%Y %W")
[ "$weekdate" -eq "$desired_date" ] || continue
...
done
For example, someone puts in 2013 05 as the desired date, you will go through all of your files and find ones with dates in the range you want. NOTE: That the week of the year is zero filled. You may need to zero fill the input of the week number to match.
I have a string in the format "yyyymmdd". It is a string in bash and I want to get it converted into a date so that all other date functions can be used on it.
"20121212" string into "20121212" date with format "%Y%m%d".
This worked for me :
date -d '20121212 7 days'
date -d '12-DEC-2012 7 days'
date -d '2012-12-12 7 days'
date -d '2012-12-12 4:10:10PM 7 days'
date -d '2012-12-12 16:10:55 7 days'
then you can format output adding parameter '+%Y%m%d'
We can use date -d option
1) Change format to "%Y-%m-%d" format i.e 20121212 to 2012-12-12
date -d '20121212' +'%Y-%m-%d'
2)Get next or last day from a given date=20121212. Like get a date 7 days in past with specific format
date -d '20121212 -7 days' +'%Y-%m-%d'
3) If we are getting date in some variable say dat
dat2=$(date -d "$dat -1 days" +'%Y%m%d')
date only work with GNU date (usually comes with Linux)
for OS X, two choices:
change command (verified)
#!/bin/sh
#DATE=20090801204150
#date -jf "%Y%m%d%H%M%S" $DATE "+date \"%A,%_d %B %Y %H:%M:%S\""
date "Saturday, 1 August 2009 20:41:50"
http://www.unix.com/shell-programming-and-scripting/116310-date-conversion.html
Download the GNU Utilities from Coreutils - GNU core utilities (not verified yet)
http://www.unix.com/emergency-unix-and-linux-support/199565-convert-string-date-add-1-a.html
just use the -d option of the date command, e.g.
date -d '20121212' +'%Y %m'