I have a file in which i have following content
2020-07-30 14:28:02|INFO|0||agent 1|CUpload|CUploadService
2020-07-30 14:28:02|INFO|0||agent 1|CUpload|CUpload
2020-07-30 14:28:04|INFO|0||agent 1|CUpload|CUplo
I need to write a script through which i can calculate the time difference. If time is more than 60 minutes since the file updated last time, i need to write the column 3 in another file.
I am able to get the date in right format but i dont know how to take difference so it will tell me if it has been more than 60 mins or not.
date +%Y-%m-%d" "%H:%M:%S
2020-07-30 14:47:24
I have placed two times in a file and took their difference but it came out to be zero
more 3.unl | awk -F '|' '{print$2 - $1}'
One possibility: convert date and time to epoch and subtract, eg:
#!/bin/bash
first='2020-07-30 14:28:04'
later='2020-07-30 15:39:09'
ep1=$(date --date="$first" +%s)
ep2=$(date --date="$later" +%s)
diff=$((ep2 - ep1))
echo diff $diff
if (( $diff > 3600 )); then
echo actions ...
fi
I have a log file which is daily rotate( let say main log file) I need capture the log hourly up-to now then store the hourly log file
Ex;
When I execute the script command at 02/06/2017 03:41:35 I need logs between 02:41:35 to 03:41:35
Main log file
[02/06/2017][00:12:41][58162][3690952448][000000000000000000000000
[02/06/2017][00:12:41][58162][3690952448][000000000000000000000000
----------
[02/06/2017][03:41:35][57732][3674167040][000000000000000000000000
[02/06/2017][03:41:35][57732][3674167040][000000000000000000000000
hourly log file
[02/06/2017][02:41:35][58162][3690952448][000000000000000000000000
[02/06/2017][02:41:35][58162][3690952448][000000000000000000000000
----------
[02/06/2017][03:41:35][57732][3674167040][000000000000000000000000
[02/06/2017][03:41:35][57732][3674167040][000000000000000000000000
I have execute the below commands, but it’s not capturing hourly log instead of capturing all the logs
echo $(date -d'now-1 hours' +"[%d/%m/%Y][%H:%M:%S]") | cat mainlog.log
echo $(date --date='1 hours ago' +"[%d/%m/%Y][%H:%M:%S]") | cat mainlog.log
You are piping the output of echo into cat - that won't filter anything. cat will ignore the input coming through the pipe (stdin) since it is reading from the file mainlog.log.
You could do this instead (note the change in date format - I have removed the %M and %S parts):
grep -F "$(date -d'now-1 hours' +'[%d/%m/%Y][%H:]')" mainlog.log
grep -F "$(date -d'1 hours ago' +'[%d/%m/%Y][%H:]')" mainlog.log
Here, grep would look for the string returned by the date command in your log file.
I'm looking for a way to replace the text between positions 10 and 17 in a file with a Linux command or script. For example, I'd like to replace the date text 20140101 with 01/01/2014.
I'm hoping this is something I can in a single command from the command line with maybe sed or awk?
Using sed, you can capture the first 9 chars in a capture group that would be placed as is. The remaining would be broken in 3 capture groups and re-arranged as you desire.
Something like:
sed -r 's#(.{9})(.{4})(.{2})(.{2})#\1\3/\4/\2#' file
If you are on a system that does not have GNU sed escape ( ) { } with \.
If all your strings are dates then the best is to use date command:
$ date -d 20140101 +%m/%d/%Y
01/01/2014
$ date -d 20140923 +"%m-%d-%Y %a %b"
09-23-2014 Tue Sep
It is especially great tool to translate number of seconds from 1970 (unix epoch) used in many log files:
$ date --date='#1411199063' +"%m-%d-%Y %H:%M:%S"
09-20-2014 07:44:23
I have a shell script that runs on Linux and uses this call to get yesterday's date in YYYY-MM-DD format:
date -d "1 day ago" '+%Y-%m-%d'
It works most of the time, but when the script ran yesterday morning at 2013-03-11 0:35 CDT it returned "2013-03-09" instead of "2013-03-10".
Presumably daylight saving time (which started yesterday) is to blame. I'm guessing the way "1 day ago" is implemented it subtracted 24 hours, and 24 hours before 2013-03-11 0:35 CDT was 2013-03-09 23:35 CST, which led to the result of "2013-03-09".
So what's a good DST-safe way to get yesterday's date in bash on Linux?
I think this should work, irrespective of how often and when you run it ...
date -d "yesterday 13:00" '+%Y-%m-%d'
Under Mac OSX date works slightly different:
For yesterday
date -v-1d +%F
For Last week
date -v-1w +%F
This should also work, but perhaps it is too much:
date -d #$(( $(date +"%s") - 86400)) +"%Y-%m-%d"
If you are certain that the script runs in the first hours of the day, you can simply do
date -d "12 hours ago" '+%Y-%m-%d'
BTW, if the script runs daily at 00:35 (via crontab?) you should ask yourself what will happen if a DST change falls in that hour; the script could not run, or run twice in some cases. Modern implementations of cron are quite clever in this regard, though.
Here a solution that will work with Solaris and AIX as well.
Manipulating the Timezone is possible for changing the clock some hours.
Due to the daylight saving time, 24 hours ago can be today or the day before yesterday.
You are sure that yesterday is 20 or 30 hours ago. Which one? Well, the most recent one that is not today.
echo -e "$(TZ=GMT+30 date +%Y-%m-%d)\n$(TZ=GMT+20 date +%Y-%m-%d)" | grep -v $(date +%Y-%m-%d) | tail -1
The -e parameter used in the echo command is needed with bash, but will not work with ksh.
In ksh you can use the same command without the -e flag.
When your script will be used in different environments, you can start the script with #!/bin/ksh or #!/bin/bash. You could also replace the \n by a newline:
echo "$(TZ=GMT+30 date +%Y-%m-%d)
$(TZ=GMT+20 date +%Y-%m-%d)" | grep -v $(date +%Y-%m-%d) | tail -1
date -d "yesterday" '+%Y-%m-%d'
To use this later:
date=$(date -d "yesterday" '+%Y-%m-%d')
you can use
date -d "30 days ago" +"%d/%m/%Y"
to get the date from 30 days ago, similarly you can replace 30 with x amount of days
Just use date and trusty seconds:
As you rightly point out, a lot of the details about the underlying computation are hidden if you rely on English time arithmetic. E.g. -d yesterday, and -d 1 day ago will have different behaviour.
Instead, you can reliably depend on the (precisely documented) seconds since the unix epoch UTC, and bash arithmetic to obtain the moment you want:
date -d #$(( $(date +"%s") - 24*3600)) +"%Y-%m-%d"
This was pointed out in another answer. This form is more portable across platforms with different date command line flags, is language-independent (e.g. "yesterday" vs "hier" in French locale), and frankly (in the long-term) will be easier to remember, because well, you know it already. You might otherwise keep asking yourself: "Was it -d 2 hours ago or -d 2 hour ago again?" or "Is it -d yesterday or -d 1 day ago that I want?"). The only tricky bit here is the #.
Armed with bash and nothing else:
Bash solely on bash, you can also get yesterday's time, via the printf builtin:
%(datefmt)T
causes printf to output the date-time string resulting from using
datefmt as a format string for strftime(3). The corresponding argu‐
ment is an integer representing the number of seconds since the
epoch. Two special argument values may be used: -1 represents the
current time, and -2 represents the time the shell was invoked.
If no argument is specified, conversion behaves as if -1 had
been given.
This is an exception to the usual printf behavior.
So,
# inner printf gets you the current unix time in seconds
# outer printf spits it out according to the format
printf "%(%Y-%m-%d)T\n" $(( $(printf "%(%s)T" -1) - 24*3600 ))
or, equivalently with a temp variable (outer subshell optional, but keeps environment vars clean).
(
now=$(printf "%(%s)T" -1);
printf "%(%Y-%m-%d)T\n" $((now - 24*3600));
)
Note: despite the manpage stating that no argument to the %()T formatter will assume a default -1, i seem to get a 0 instead (thank you, bash manual version 4.3.48)
You can use:
date -d "yesterday 13:55" '+%Y-%m-%d'
Or whatever time you want to retrieve will retrieved by bash.
For month:
date -d "30 days ago" '+%Y-%m-%d'
As this question is tagged bash "DST safe":
And using fork to date command implie delay, there is a simple and more efficient way using pure bash built-in:
printf -v tznow '%(%z %s)T' -1
TZ=${tznow% *} printf -v yesterday '%(%Y-%m-%d)T' $(( ${tznow#* } - 86400 ))
echo $yesterday
This is a lot quicker on more system friendly than having to fork to date.
From bash version 5.0, there is a new variable $EPOCHSECONDS
printf -v tz '%(%z)T' -1
TZ=$tz printf -v yesterday '%(%Y-%m-%d)T' $(( EPOCHSECONDS - 86400 ))
echo $yesterday
What is the syntax to extract the day of the week from a stored date variable?
The dateinfile format is always [alphanum]_YYYYMMDD.
In this pseudocode example, trying to get dayofweek to store Saturday:
#! /bin/bash
dateinfile="P_20090530"
dayofweek="$dateinfile -u +%A"
[me#home]$ date --date=${dateinfile#?_} "+%A"
Saturday
Or, to put it as you've requested:
[me#home]$ dayofweek=$(date --date=${dateinfile#?_} "+%A")
[me#home]$ echo $dayofweek
Saturday
date -d $(echo $dateinfile | cut -f2 -d_) -u +%A
The inner expression separates the 20090530 from P_20090530, and the outer one extracts the day of week from that date
I needed the same output recently and googled upon this, but I had the same problem stating Bad Substition error.
I then read the date manual and made my version up as follows:
#! /bin/sh
dateinfile="P_20090530"
dayofweek=`date --reference $dateinfile +%A`
echo $dayofweek