How can I list files based on names with date prefix & suffix. Ex: I have file with name as "http_access_2017-04-13.log" then how can I with files with last five days back files ??
Create you time stamps with date -d:
ago ()
{
date +%Y-%m-%d -d "$1 days ago"
}
for n in $(seq 5); do
echo http_access_$(ago $n).log
done
Try this command:
ls -lr http_access*.log | tail -5
Related
I want a command line can display number of Monday(s) which fall(s) on the first of the month in a given year without using sed or awk commands
I have this command that display the first date of the current month
date -d "-0 month -$(($(date +%d)-1)) days"
With GNU date, you can read input from a file (or standard input):
printf '%s\n' 2021-{01..12}-01 | date -f- +%u | grep -c 1
This prints dates for the first of each month in a year, then formats them as "weekday" (where 1 is "Monday"), then counts the number of Mondays.
To parametrize the year, replace 2021 with a variable containing the year; wrapped in a function:
mondays() {
local year=$1
printf '%s\n' "$year"-{01..12}-01 | date -f- +%u | grep -c 1
}
Using a for loop, this can be accomplished as follows.
for mon in {01..12}; do date -d "2021-$mon-01" +%u; done | grep -c 1
Breakdown
We iterate through the numbers 01 to 12 representing the months.
We call date passing in the custom date value with the first date of each month in the year. We use +%u to return the day of week where 1 represents Monday.
Lastly we count the number of 1s using grep -c or grep --count
Note, the desired year has been hard coded as 2021. The current year can be used as:
for mon in {01..12}; do date -d "$(date +%Y)-$mon-01" +%u; done | grep -c 1
This can also all be put into a function and the desired year passed in as an argument:
getMondays() {
for mon in {01..12}; do date -d "$1-$mon-01" +%u; done | grep -c 1
}
I implemented it as:
for ((i=1,year=2021,mondays=0; i< 12; i++)) {
if [ $(date -d "$i/1/$year" +%u) -eq 1 ]
then
let "mondays++"
fi
}
echo "There are $mondays Mondays in $year."
That said, I like Mushfiq's answer. Quite elegant.
Let's say I have some holiday data (holiday_master.csv) in columns, something like
...
20200320 Vernal Equinox Day
20200429 Showa Day
20200503 Constitution Day
20200505 Green Day
20200720 Children's Day
20200811 Sea Day
...
Given this set of data, I want to find the next closest holiday from the given date.
For example if the input is 20200420, 20200429 Showa Day is expected.
If the input is 20200620, 20200720 Children's Day is expected.
I have a feeling that awk has the necessary functionality to do this, but any solution that works in a bash script is welcome.
Would you please try the bash script:
#!/bin/bash
input="20200428" # or assign to whatever
< "holiday_master.csv" sort -nk1,1 | # sort the csv file by date and pass to the while loop
while read -r date desc; do
if (( date >= input )); then # if the date is greater than or equal to the input
echo "$date" "$desc" # then print the line
break # and exit the loop
fi
done
Assuming no two days will ever have the same date...
DATE=<some desired input date>
awk "{print (\$1 - $DATE"' "\t" $0)}' calendar.txt | sed '/^-/d' | sort | head -n 1 | awk '{$1=""; print $0}'
Explanation
awk "{print (\$1 - $DATE"' "\t" $0)}' calendar.txt: Prepend a column to the input.txt file describing the difference between the desired input date and the date column
sed '/^-/d': Remove all lines beginning with -. Dates with negative differences have already passed.
sort: Sort the remaining entries from least to greatest (based upon the difference column)
head -n 1: Select only the first row (The lowest difference)
awk '{$1=""; print $0}': Print all but the first column
Prettier script version
#!/bin/bash
# Usage: script <Date> <Calendar file>
DATE=${1:--1}
CAL=${2:-calendar.txt}
# Arg check and execute
if[ ! -f $CAL ]
then
echo "File not found: $CAL"
echo "Usage: script <Date> <Calendar file>"
elif [ $DATE -le 0 ]
then
echo "Invalid date: $DATE"
echo "Usage: script <Date> <Calendar file>"
elif [ $(echo "$DATE" | grep -Ewo -- '-?[0-9]+' | wc -l) -eq 0 ]
then
echo "Invalid date: $DATE"
echo "Usage: script <Date> <Calendar file>"
else
awk '{print ($1 - '"$DATE"' "\t" $0)}' $CAL | sed '/^-/d' | sort | head -n 1 | awk '{$1=""; print $0}'
fi
As you use YYYYMMDD format we might compare it just like numbers (note: year is greater than month, month is greater than day). So you can use AWK following way, let:
20200320 Vernal Equinox Day
20200429 Showa Day
20200503 Constitution Day
20200505 Green Day
20200720 Children's Day
20200811 Sea Day
be file named holidays.txt then:
awk 'BEGIN{inputdate=20200420}{if($1>inputdate){print $2;exit}}' holidays.txt
output:
Showa
Explanation: in BEGIN I set inputdate to 20200420 then when line with greater number in 1st column is found I print content of 2nd column and exit (otherwise later dates would be printed too). Note that AWK does automatically parse number when asked to do comparison (> in this case) so you do not have to care about conversion yourself - you could even do inputdate="20200420" and it would work too.
This solution assumes that all dates in file are already sorted.
Using awk and assuming the source data is comma separated:
awk -F, -v dayte="20200420" '
BEGIN {
"date -d "dayte" +%s" | getline dat1
{
{
"date -d "$1" +%s" | getline dat2;
dat3=dat2-dat1;
if (dat3 > 0 )
{
hols[dat3]=$2
}
}
END {
asorti(hols,hols1,"#ind_num_asc");
print hols[hols1[1]]
}
' holiday_master.csv
One liner:
awk -F, -v dayte="20200420" 'BEGIN { "date -d "dayte" +%s" | getline dat1 } { "date -d "$1" +%s" | getline dat2;dat3=dat2-dat1;if (dat3 > 0 ) { hols[dat3]=$2 } } END { asorti(hols,hols1,"#ind_num_asc");print hols[hols1[1]] }' holiday_master.csv
Set the field separator to , and set a variable dayte to the date we wish to check. In the BEGIN block, we pass the dayte variable through to date command via an awk pipe/getline and read the epoch result into the variable dat1. We do the same with the first column on the master file ($1) and read this into dat2. We take the difference between the epoch dates and read the result into dat3. Only if the result is positive (in the future) do we then use dat3 for an index in a "hols" array, with the holiday description as the value. In the END block, we sort the indexes of hols into a news hols1 array basing the sort on ascending, numeric indexes. We then take the first index of the new hols1 array to attain the holiday that is closest to the dayte variable.
Assuming the holiday list file is sorted by date as you have given, the below would work
$ awk -v dt="20200420" ' (dt-$1)<0 { print;exit } ' holiday.txt
20200429 Showa Day
$ awk -v dt="20200620" ' (dt-$1)<0 { print;exit } ' holiday.txt
20200720 Children's Day
$
If the holiday file is not sorted, then you can use below
$ shuf holiday.txt | awk -v dt="20200420" ' dt-$1<0 { a[(dt-$1)*-1]=$0 } END { asort(a); print a[1] } '
20200429 Showa Day
$ shuf holiday.txt | awk -v dt="20200620" ' dt-$1<0 { a[(dt-$1)*-1]=$0 } END { asort(a); print a[1] } '
20200720 Children's Day
This question already has answers here:
Increment date with AWK for few days and months
(3 answers)
Closed 4 years ago.
I have a file 1.txt like below:
"15227962157615645"$"2018-12-04 06:55:43"
"15227525816721347"$"2018-12-03 18:48:11"
I can get the date using:
awk -F\" '{print $4}' 1.txt
Additionally I need add one month to the date. For the above input my desired output would be:
2019-01-04 06:55:43
2019-01-03 18:48:11
I tried to use
awk -F\" '{print date -d "$4 +1 month"+%Y-%m-%d}' 1.txt
but it does not work.
Awk has limited support for date calculation, so here is a bash only solution relying on the date command:
IFS='$';
while read n t; do
printf '%s$"%s"\n' "$n" "$(date -d "${t//\"/} +1 month" '+%F %T')"
done <file
The input field separator is set to $ to get the time into $t variable.
The double quote of the date field are removed using bash parameter expansion ${t//\"/}.
This allows to pass the +1 month key word to date.
Then the printf prints back to the original format of the input file.
I have 2 script in bash, and i have some files:
transaction-2012-01-01.csv.bz2
transaction-2012-01-02.csv.bz2
transaction-2012-01-03.csv.bz2
transaction-2012-01-04.csv.bz2
.
.
transaction-2012-01-31.csv.bz2
transaction-2012-02-01.csv.bz2
.
.
transaction-2012-02-28.csv.bz2
I have a script called script.sh
cat script.sh
YEAR_MONTH=$1
FILEPATH="transaction-$YEAR_MONTH*.csv.bz2"
bzcat $FILEPATH|strings|grep -v "code" >> output
And if you need call the script you can use other script
cat script2.sh
LAST_MONTH=$(date -d -1month +%Y"-"%m)
if [ $# -eq 1 ]; then
DATE=$1
else
DATE=$LAST_MONTH
fi
script.sh $DATE 1>output$DATE.csv 2>> log.txt
And it do cat the files in a month, but now i need call the script with a specific week in a year:
bash script2.sh 2012-01
where 2012 is the year and 01 is the month
Now i need call the script with:
bash script2.sh 2012 13
where 2012 is the year and 13 is the week in a year
Now i need the cat only to the files in the year and week that the user specified, no per month per week
But the format of the files do not help me!!!! because the name is transaction-year-month-day.csv.bz2, not transaction-year-week.csv.bz2
Take a look at the manpage for strftime. These are date format codes. For example:
$ date +"%A, %B %e, %Y at %I:%m:%S %p"
Will print out a date like:
Thursday, May 30, 2013 at 02:05:31 PM
Try to see why this works.
On some systems, the date command will have a -j switch. This means, don't set the date, but reformat the given date. This allows you to convert one date to another:
$ date -f"$input_format" "$string_date" +"$output_format"
The $input_format is the format of your input date. $string_date is the string representation of the date in your $input_format. And, $output_format is the format you want your date in.
The first two fields are easy. Your date is in YY-MM-DD format:
$ date -f"%Y-%m-%d" "$my_date_string"
The question is what can you do for the final format. Fortunately, there is a format for the week in the year. %V which represents the weeks at 01-53 and %W which represents the weeks as 00-53.
What you need to do is find the date string on your file name, then convert that to the year and week number. If that's the same as the input, you need to concatenate this file.
find $dir -type f | while read transaction_file
do
file_date=${transaction_file#transaction-} #Removes the prefix
file_date=${file_date%.csv.bz2} #Removes the suffix
weekdate=$(date -j -f"%Y-%m-%d" "$file_date" +"%Y %W")
[ "$weekdate" -eq "$desired_date" ] || continue
...
done
For example, someone puts in 2013 05 as the desired date, you will go through all of your files and find ones with dates in the range you want. NOTE: That the week of the year is zero filled. You may need to zero fill the input of the week number to match.
I'm using an RPM-based distro and I want to dynamically search a log file for today's date and yesterday's date to output a report. The string has to be dynamic ( no egrep "\b2012-10-[20-30]\b" ) meaning that I can take the same one-liner or script and search a file for today's date and yesterday's date and print some output. Basically searching log files for specific entries.
Here's what I got, but I want to replace the egrep with something dynamic:
grep "No Such User Here" /var/log/maillog | egrep "\b2012-10-2[3-4]\b" | cut -d "<" -f 3 | egrep -o '\b[a-zA-Z0-9._%-]+#[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}\b' | cut -d "#" -f 2 | sort -d |uniq -ci | awk -F" " '{ print "Domain: " $2 " has been sent " $1 " messages that got a No Such User Here error." }'
Any help is appreciated. I'm looking for something that very likely uses the date command
date "+%Y-%m-%d"
but I need to take the %d and search for both the current day, and yesterday. Can this be done?
Any insight is much appreciated.
If you have GNU date:
$ x=$(date "+%Y-%m-%d")
$ y=$(date "+%Y-%m-%d" -d "-1 day")
$ egrep "($x|$y)" file
x contains current date and y contains the yesterday's date.
With GNU awks time functions:
gawk 'BEGIN{
today = strftime("%Y-%m-%d")
yesterday = strftime("%Y-%m-%d",systime()-24*60*60)
}
$0 ~ "(" today "|" yesterday ")"
' file