I have this script I am building and need a bit of help. I want to use as minimal lines of code as possible. However what I want the script to do is the following.
Backup a specified file such as authlog into a new file with the keywords I specify.
However. I also want it to back up the file if current backed up file is older than 2+ days ago and if it is newer. Leave it alone. if (file) is older than +2 days old delete it and replace it with an updated one.
I'm getting integer expression expected and I'm not sure how to fix it
#!/bin/bash
authlog=/home/(myhomedir)/logs/backups/authlog-${current_date}
backup="$(cat /var/log/auth.log | grep -e failed -e invalid > /home/(myhomedir)/logs/backup)"
dayold=$(date -d '1 day ago' "+%Y-%m-%d")
current_date=$(date +"%m_%d_%Y")
if [ $authlog -lt $dayold ] ;
then ${backup}
else
echo $authlog is newer than $current_date $authlog has not been updated
if [ $authlog -eq ${current_date} ]
then "$(rm {authlog})"
fi
fi
You can't use -lt to compare non-numeric strings. This is why you're getting "integer expression expected". You're trying to compare a filename to a date string.
The easiest way to compare dates is by converting them to seconds since epoch (Jan 1, 1970 00:00:00 UTC). As an example, suppose I have a file called foo.bar and I want to check if it was last modified 3 or more days ago:
filetime=$(stat -c %Y foo.bar)
three_days_ago=$(date -d "3 days ago" +%s)
if [ $filetime -lt $three_days_ago ]; then
echo "foo.bar was last modified more than 3 days ago"
fi
Also, your authlog variable does not contain a date. I'm assuming there's a missing step that you forgot.
Related
I am trying to create a new file if the last modified date of the previously created file is less then the current date. Below is the code i tried but not sure getting what value as difference between last modified date and current date.
Code
LAST_MODIFIED_DATE=$(stat -c %y ${APPDIR}/data/XYZ.csv)
echo "LAST_MODIFIED_DATE ${LAST_MODIFIED_DATE}"
NOW=$(date +%s)
echo "NOW ${NOW}"
let diff=${NOW}-${LAST_MODIFIED_DATE}
echo "Diff ${diff}"
Result
LAST_MODIFIED_DATE 2021-08-03 10:30:56.627022878 -0500
NOW 1629354883
Diff 1629354883-2021-08-03 10:30:56.627022878 -0500
You should convert LAST_MODIFIED_DATE in seconds since 1970-01-01 first:
NOW="$(date +%s)"
LAST_MODIFIED_DATE="$(stat -c %y ${APPDIR}/data/XYZ.csv)"
LAST_MODIFIED_DATE_EPOCH="$(date +%s --date="$LAST_MODIFIED_DATE")"
diff=$(($NOW - $LAST_MODIFIED_DATE_EPOCH))
echo "Diff ${diff}"
But diff will be all the time positive unless the file ${APPDIR}/data/XYZ.csv was created with a date in the future.
If you want to test against a different date (for instance to check whether the file has been created in the last hour or not) you can modify the definition of NOW:
NOW="$(date +%s --date '1 hour ago')"
I am creating a script to pull out the list of highest cpu consuming jobs on my server. I am taking the dump of processes in a flat file in below format:
202009080230,4.1,218579,padaemon,02:30,00:00:01,SD_RCC_FRU:wf_rcc_ds_CRS_FactSuspendedGdc_d.s_m_rcc_ds_Staging_Dimension_FactSuspendedGdc_ipi_d
The second data point(4.1) is my cpu utilization while the sixth one is the time taken(00:00:01).
I intend to filter out only the entries from this file that are greater than 75% and run for more than 00:05:00 i.e. 5 minutes. I wrote an if logic for the same like below:
var10=`ls -lrt /projects/Informatica/INFA_ADMIN/LogFiles/myfile.txt |awk '{ print $9 }'`
sed -i 's/ /,/g' $var10
for i in `cat $var10`
do
var12=`echo $i |cut -d',' -f2`
var16=`echo $i |cut -d',' -f6`
if [ $var12 -ge 75.0 ] && ("$var16" > "00:05:00");
then
<logic>
fi
done
the script is able to identify processes taking more than 75.0 cpu but failing in the second condition.
[ 136 -ge 75.0 ]
00:20:26
1> 00:05:00 cpu_report_email_prateek_test.sh[63]: 00:20:26: not found [No such file or directory]
Can you please guide how this can be fixed?
First, I understand that on line 1 you're trying to have the latest file. I would rather have the file created always with the same name, then at the end of the script, after processing it, mv to another name (like myfile.txt.20200101). So you'll always know that myfile.txt is the latest one. More efficient than doing ls. Then again, you could improve your line with : (it's the number one, not the letter "l")
var10=$(ls -1rt /projects/Informatica/INFA_ADMIN/LogFiles/myfile.txt)
From your line 2, I assume the original file is space delimited. Actually, you might use that to your advantage instead of adding commas. Also, using a for loop with "cat" is less performant than using a while loop. For the time comparison, I would use a little trick. Transforming your threshold and reading, in second since epoch then compared them.
MyTreshold=$(date -d 00:05:00 +"%s")
while read -r line; do
MyArray=( $line )
MyReading=$(date -d ${MyArray[5]} +"%s")
#the array starts a index 0
if [[ ${MyArray[1]} -ge 75 && $MyReading -gt $MyThreshold ]]; then
<LOGIC>
fi
done <$var10
The RHS of your && expression would expand $var16 and attempt to execute the result, redirecting the output to a file 00:05:00. Replace with [ $x -gt $y ]. For a numeric comparison, you'll need straight numbers (no ':' chars). You could modify from hh:mm:ss to total seconds then compare.
#!/bin/bash
cd /home/pi/cc/uvreadings
while true
do
ATIME=`stat -c %Z /home/pi/cc/uvreadings/uvreadings.log`
if [[ "$ATIME" < "$LTIME" ]]
then
echo "log file not updated for +24 hours"
else
echo "log file WAS updated in last +24 hours"
fi
sleep 10
done
I am trying to check if a file has been modified in the last 24 hours by comparing atime to Ltime
atime will always be less than ltime
so can i modify the statement
if [[ "$ATIME" < "$LTIME" ]]
to
if [[ "$ATIME +1 day" < "$LTIME" ]]`
of is the a better way to achieve this
thanks for any advice
First of all, there are (at least) three different timestamps on a file, and you seem to have them mixed up. You talk about atime, the %Z option instead gives you ctime, and it sounds like what you want is actually mtime: the time at which the file's contents last changed. You can get mtime with stat -c %Y.
To see if it was less than one day ago, you can indeed add one day to it; but the time is in seconds, so you need to add 24*60*60 seconds. The easiest way to do that is probably with arithmetic expansion. It could look something like this:
if [[ $((MTIME + (24 * 60 * 60))) > $LTIME ]] # modified in the last day
I assume you wrote code to get the current time and store it in LTIME, even though you didn't show that code in your question. It has to go inside the loop, of course. One way to do that would be
LTIME=`date +%s`
I have a shell script that runs on Linux and uses this call to get yesterday's date in YYYY-MM-DD format:
date -d "1 day ago" '+%Y-%m-%d'
It works most of the time, but when the script ran yesterday morning at 2013-03-11 0:35 CDT it returned "2013-03-09" instead of "2013-03-10".
Presumably daylight saving time (which started yesterday) is to blame. I'm guessing the way "1 day ago" is implemented it subtracted 24 hours, and 24 hours before 2013-03-11 0:35 CDT was 2013-03-09 23:35 CST, which led to the result of "2013-03-09".
So what's a good DST-safe way to get yesterday's date in bash on Linux?
I think this should work, irrespective of how often and when you run it ...
date -d "yesterday 13:00" '+%Y-%m-%d'
Under Mac OSX date works slightly different:
For yesterday
date -v-1d +%F
For Last week
date -v-1w +%F
This should also work, but perhaps it is too much:
date -d #$(( $(date +"%s") - 86400)) +"%Y-%m-%d"
If you are certain that the script runs in the first hours of the day, you can simply do
date -d "12 hours ago" '+%Y-%m-%d'
BTW, if the script runs daily at 00:35 (via crontab?) you should ask yourself what will happen if a DST change falls in that hour; the script could not run, or run twice in some cases. Modern implementations of cron are quite clever in this regard, though.
Here a solution that will work with Solaris and AIX as well.
Manipulating the Timezone is possible for changing the clock some hours.
Due to the daylight saving time, 24 hours ago can be today or the day before yesterday.
You are sure that yesterday is 20 or 30 hours ago. Which one? Well, the most recent one that is not today.
echo -e "$(TZ=GMT+30 date +%Y-%m-%d)\n$(TZ=GMT+20 date +%Y-%m-%d)" | grep -v $(date +%Y-%m-%d) | tail -1
The -e parameter used in the echo command is needed with bash, but will not work with ksh.
In ksh you can use the same command without the -e flag.
When your script will be used in different environments, you can start the script with #!/bin/ksh or #!/bin/bash. You could also replace the \n by a newline:
echo "$(TZ=GMT+30 date +%Y-%m-%d)
$(TZ=GMT+20 date +%Y-%m-%d)" | grep -v $(date +%Y-%m-%d) | tail -1
date -d "yesterday" '+%Y-%m-%d'
To use this later:
date=$(date -d "yesterday" '+%Y-%m-%d')
you can use
date -d "30 days ago" +"%d/%m/%Y"
to get the date from 30 days ago, similarly you can replace 30 with x amount of days
Just use date and trusty seconds:
As you rightly point out, a lot of the details about the underlying computation are hidden if you rely on English time arithmetic. E.g. -d yesterday, and -d 1 day ago will have different behaviour.
Instead, you can reliably depend on the (precisely documented) seconds since the unix epoch UTC, and bash arithmetic to obtain the moment you want:
date -d #$(( $(date +"%s") - 24*3600)) +"%Y-%m-%d"
This was pointed out in another answer. This form is more portable across platforms with different date command line flags, is language-independent (e.g. "yesterday" vs "hier" in French locale), and frankly (in the long-term) will be easier to remember, because well, you know it already. You might otherwise keep asking yourself: "Was it -d 2 hours ago or -d 2 hour ago again?" or "Is it -d yesterday or -d 1 day ago that I want?"). The only tricky bit here is the #.
Armed with bash and nothing else:
Bash solely on bash, you can also get yesterday's time, via the printf builtin:
%(datefmt)T
causes printf to output the date-time string resulting from using
datefmt as a format string for strftime(3). The corresponding argu‐
ment is an integer representing the number of seconds since the
epoch. Two special argument values may be used: -1 represents the
current time, and -2 represents the time the shell was invoked.
If no argument is specified, conversion behaves as if -1 had
been given.
This is an exception to the usual printf behavior.
So,
# inner printf gets you the current unix time in seconds
# outer printf spits it out according to the format
printf "%(%Y-%m-%d)T\n" $(( $(printf "%(%s)T" -1) - 24*3600 ))
or, equivalently with a temp variable (outer subshell optional, but keeps environment vars clean).
(
now=$(printf "%(%s)T" -1);
printf "%(%Y-%m-%d)T\n" $((now - 24*3600));
)
Note: despite the manpage stating that no argument to the %()T formatter will assume a default -1, i seem to get a 0 instead (thank you, bash manual version 4.3.48)
You can use:
date -d "yesterday 13:55" '+%Y-%m-%d'
Or whatever time you want to retrieve will retrieved by bash.
For month:
date -d "30 days ago" '+%Y-%m-%d'
As this question is tagged bash "DST safe":
And using fork to date command implie delay, there is a simple and more efficient way using pure bash built-in:
printf -v tznow '%(%z %s)T' -1
TZ=${tznow% *} printf -v yesterday '%(%Y-%m-%d)T' $(( ${tznow#* } - 86400 ))
echo $yesterday
This is a lot quicker on more system friendly than having to fork to date.
From bash version 5.0, there is a new variable $EPOCHSECONDS
printf -v tz '%(%z)T' -1
TZ=$tz printf -v yesterday '%(%Y-%m-%d)T' $(( EPOCHSECONDS - 86400 ))
echo $yesterday
I want to check in linux bash whether a file was created more than x time ago.
let's say the file is called text.txt and the time is 2 hours.
if [ what? ]
then
echo "old enough"
fi
Only for modification time
if test `find "text.txt" -mmin +120`
then
echo old enough
fi
You can use -cmin for change or -amin for access time. As others pointed I don’t think you can track creation time.
I always liked using date -r /the/file +%s to find its age.
You can also do touch --date '2015-10-10 9:55' /tmp/file to get extremely fine-grained time on an arbitrary date/time.
Using the stat to figure out the last modification date of the file, date to figure out the current time and a liberal use of bashisms, one can do the test that you want based on the file's last modification time1.
if [ "$(( $(date +"%s") - $(stat -c "%Y" "$somefile") ))" -gt "7200" ]; then
echo "'$somefile' is older then 2 hours"
fi
While the code is a bit less readable then the find approach, I think its a better approach then running find to look at a file you already "found". Also, date manipulation is fun ;-)
As Phil correctly noted creation time is not recorded, but use %Z instead of %Y below to get "change time" which may be what you want.
[Update]
For mac users, use stat -f "%m" "$somefile" instead of the Linux specific syntax above
Creation time isn't stored.
What are stored are three timestamps (generally, they can be turned off on certain filesystems or by certain filesystem options):
Last access time
Last modification time
Last change time
a "Change" to the file is counted as permission changes, rename etc. While the modification is contents only.
Although ctime isn't technically the time of creation, it quite often is.
Since ctime it isn't affected by changes to the contents of the file, it's usually only updated when the file is created. And yes - I can hear you all screaming - it's also updated if you change the access permissions or ownership... but generally that's something that's done once, usually at the same time you put the file there.
Personally I always use mtime for everything, and I imagine that is what you want. But anyway... here's a rehash of Guss's "unattractive" bash, in an easy to use function.
#!/bin/bash
function age() {
local filename=$1
local changed=`stat -c %Y "$filename"`
local now=`date +%s`
local elapsed
let elapsed=now-changed
echo $elapsed
}
file="/"
echo The age of $file is $(age "$file") seconds.
The find one is good but I think you can use anotherway, especially if you need to now how many seconds is the file old
date -d "now - $( stat -c "%Y" $filename ) seconds" +%s
using GNU date
Consider the outcome of the tool 'stat':
File: `infolog.txt'
Size: 694 Blocks: 8 IO Block: 4096 regular file
Device: 801h/2049d Inode: 11635578 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1000/ fdr) Gid: ( 1000/ fdr)
Access: 2009-01-01 22:04:15.000000000 -0800
Modify: 2009-01-01 22:05:05.000000000 -0800
Change: 2009-01-01 22:05:05.000000000 -0800
You can see here the three dates for Access/modify/change. There is no
created date. You can only really be sure when the file contents were
modified (the "modify" field) or its inode changed (the "change"
field).
Examples of when both fields get updated:
"Modify" will be updated if someone concatenated extra information to
the end of the file.
"Change" will be updated if someone changed permissions via chmod.
I use
file_age() {
local filename=$1
echo $(( $(date +%s) - $(date -r $filename +%s) ))
}
is_stale() {
local filename=$1
local max_minutes=20
[ $(file_age $filename) -gt $(( $max_minutes*60 )) ]
}
if is_stale /my/file; then
...
fi