How to add text at the end of each line after skipping Last N characters - linux

I'm on Linux command line and I have log file with following content
Subscription Service Shutdown at 9:59PM UTC
Subscription Service Restarted at 11:57PM UTC
I want to add date before Time Stamp for every line
Subscription Service Shutdown at 01 Oct 2016 9:59PM UTC
Subscription Service Restarted at 01 Oct 2016 11:57PM UTC

the vim way:
:%s/ at /&knownString/

Something like this:
sed -r "s/([0-9]+{1}:[0-9]+{1}[A-Z]{2}.*$)/01 Oct 2016 \1/g" test.txt > new.txt
Where text.txt is your log file and "01 Oct 2016 " is your text and the \1 is the time inserted back in from the group match.

you can try something like this;
echo "Subscription Service Shutdown at 9:59PM UTC" | sed "s/at */at $(date '+%Y %b %d') /"
Eg;
$ echo "Subscription Service Shutdown at 9:59PM UTC" | sed "s/at */at $(date '+%Y %b %d') /"
Subscription Service Shutdown at 2016 Nov 08 9:59PM UTC

Different ways to do it in vim
1) add string after at
:%s/at \zs/01 Oct 2016 /
2) Find time string of the format HH:MM followed by AM UTC or PM UTC and add string. Add $ anchor if you need to match only at end of line
:%s/\v(\d{1,2}:\d{1,2}[AP]M UTC)/01 Oct 2016 \1/

Related

Converting date in a different timezone with `date`

I am trying to convert dates from different timezones with UNIX date (I am on Ubuntu 20.04).
With current date, it works well:
$ date
dim. 12 déc. 2021 11:59:16 CET
$ TZ=Pacific/Tahiti date
dim. 12 déc. 2021 00:59:32 -10
But when I am working with a string, it fails:
$ export testdate="2021/10/28 17:47:26"
$ date -d "$test"
jeu. 28 oct. 2021 17:47:26 CEST
$ TZ=Pacific/Tahiti date -d "$test"
jeu. 28 oct. 2021 17:47:26 -10
as I am expecting:
jeu 28 oct. 2021 05:47:26 -10
I don't understand why I don't get the proper shift. And of course if I try with a date and time where the day should also change, it doesn't work either:
$ export test="2021/10/28 7:47:26"
$ date -d "$test"
jeu. 28 oct. 2021 07:47:26 CEST
$ TZ=Pacific/Tahiti date -d "$test" "+%F %T %Z"
2021-10-28 07:47:26 -10
while I am expecting:
mer 27 oct. 2021 19:47:26 -10
why I don't get the proper shift
test="2021/10/28 17:47:26"
Is a date in unknown timezone. No one knows what timezone it is in, what the daylight is. GNU date tries to "guess" what offset you meant, it generally traverses TZ database for current timezone and just picks the first offset that matches. Also, specifying timezone is not enough to know what daylight it is, you have to be specific.
Also, because of the daylight time you can "go back" in time, it's now known what the offset to UTC is even when you know the timezone.
Also, you don't have to export it - date does not care about test environment variable.
Converting date in a different timezone with date
If the input is in UTC, tell date that.
$ LC_ALL=C TZ=Pacific/Tahiti date -d "2021/10/28 17:47:26 UTC"
Thu Oct 28 07:47:26 -10 2021
If the input is with any other offset, tell date that.
$ LC_ALL=C TZ=Pacific/Tahiti date -d "2021/10/28 17:47:26 CEST"
Thu Oct 28 05:47:26 -10 2021
Te parsing of GNU date of input format is generally a mystery. The documentation lists 2004-02-29 16:21:42 format as an example input, so I recommend that format. If you want to be exact, I recommend strptime from dateutils (or a real programming language).
One simple way is to convert time first to epoch time :
test="2021/10/28 17:47:26"
TZ=Pacific/Tahiti date -d #$(date -d "$test" +%s)
date -d "$test" +%s converts local time to epoch time.
TZ=Pacific/Tahiti date -d #$(date -d "$test" +%s) prints Tahiti time from epoch time.
I'm not fully confident, but i get the impression you're looking the following syntax:
LC_TIME="es_ES.UTF8" TZ="America/New_York" date --date='TZ="Europe/Amsterdam" 2021/10/28 17:47:26' "+%A %F %T %B"
That takes a predefined datetime (interpreted as being local to Amsterdam), adjusts the datetime (based on the time difference) to New York-time at that same moment; Then it prints that result with Spanish names for the months/weekdays (provided that language' locale is present on your system).

Convert GMT timestamp to local time using Bash and GNU tools

I am getting results to a log file that contain a line like this:
date: Sat, 12 Dec 2020 22:33:34 GMT
I want to use only Bash and GNU tools on my Ubuntu Linux box if possible to convert this to my local time "Eastern" or Michigan/Detroit. It should work even on Daylight Saving Time or if past/before Midnight. I want the result stored in a variable in a common format such as 2020-12-01 for December 1, 2020. One variable for the military time, a second for the date would probably be best. I can calculate the "Sat/Sun/Mon/etc" and probably don't need that anyway.
I would expect the "cut" command could separate out the different fields, but how to deal with GMT?
#!/bin/bash
datime="date: Sat, 12 Dec 2020 22:33:34 GMT"
#magic happens
echo dalocaltime is $dalocaltime and dalocaldate is $dalocaldate
results:
dalocaltime is 14:20:33 and dalocaldate is 2020-01-30
This works for me, but I can't explain the ${datime#* } part
datime="date: Sat, 12 Dec 2020 22:33:34 GMT"
dalocaltime=$(date -d "${datime#* }" '+%R')
dalocaldate=$(date -d "${datime#* }" '+%Y-%m-%d')
echo dalocaltime is $dalocaltime and dalocaldate is $dalocaldate

Getting specific part of output in Linux

I have an output from a shell script like this:
aaa.sh output
Tue Mar 04 01:00:53 2014
Time drift detected. Please check VKTM trace file for more details.
Tue Mar 04 07:21:52 2014
Time drift detected. Please check VKTM trace file for more details.
Tue Mar 04 13:17:16 2014
Time drift detected. Please check VKTM trace file for more details.
Tue Mar 04 16:56:01 2014
SQL> ALTER DISKGROUP fra ADD DISK '/dev/rhdisk20'
Wed Mar 05 00:03:42 2014
Time drift detected. Please check VKTM trace file for more details.
Wed Mar 05 04:13:39 2014
Time drift detected. Please check VKTM trace file for more details.
Tue Mar 05 05:56:07 2014
GMON querying group 3 at 10 for pid 18, osid 27590856
GMON querying group 3 at 11 for pid 18, osid 27590856
I need to get the part, beginning from today's date:
Wed Mar 05 00:03:42 2014
Time drift detected. Please check VKTM trace file for more details.
Wed Mar 05 04:13:39 2014
Time drift detected. Please check VKTM trace file for more details.
Tue Mar 05 05:56:07 2014
GMON querying group 3 at 10 for pid 18, osid 27590856
GMON querying group 3 at 11 for pid 18, osid 27590856
You can get the date in the correct format like this:
today=$(date +'%a %b %d')
and then search for it like this:
grep "$today" aaa.sh
If there are lines from today without a date, such as your GMON lines, you could add -A to say how many lines after the match you want and use a big number:
grep -A 999999 "$today" aaa.sh
If you are on AIX and there is no -A option, use sed like this:
today=$(date +'%a %b %d')
sed -n "/${today}/,$ p" aaa.sh
Explanation:
That says store today's date in the variable today in the format "Wed Mar 05". Then search, without printing anything (-n) till you find that date, From that point on, till the end of file ($) print all lines (p).
I think I have an easy solution:
Get date to output the date in a format that would match the date in the file (check man date on formatting options). Since we don't want to match the hours/minutes/seconds we have to call date twice: once for the weekday/month/day half and once for the year half on the end of the full date. Between these two halves we match the horus/minutes/seconds with .* regex.
Then do:
aaa.sh | grep -E '`date --only-weekday-month-day`.*`date --only-year`' -A 999999
though I am using answer by NewWorld it can be modified as,
convert output of date similar to your file format
suppose in variable 'D'you get that output
sed '1,/${D}/d' aaa.sh
that will output all lines after match date match.
example: suppose you get D="Wed Mar 05 00:03:42 2014"
output will be as expected.
You can use
tail -n 7 filename
for getting the desired output . It will basically give you the last seven lines of the text file named filename .
For getting solution from today's date you can use :
k=$(date +"%a %b %d")
g=$(grep -nr "$k" in|cut -f1 -d:|head -1)
total=$(wc -l<in)
l=`expr $total - $g + 1
tail -n$l in
Try
sed -n '/Wed Mar 05/,$p' aaa.sh
Here -n means "don't print anything unless specified to".
First appearance of a line that matches the expression /Wed\ Mar\ 05/ till the end of the file, will be printed(p)"

How to remove terminal control escape sequences in the file?

I got a log from remote linux computer. It looks like:
2013-10-23T08:19:05+0300 Last login: Wed Oct 23 08:17:38 EEST 2013 from 10.9.167.55 on pts/0
2013-10-23T08:19:05+0300 Last login: Wed Oct 23 08:19:05 2013 from 10.9.167.55^M
2013-10-23T08:19:07+0300 ^[[?1034h-bash-4.1$ date
2013-10-23T08:19:07+0300 Wed Oct 23 08:19:07 EEST 2013
2013-10-23T08:19:08+0300 -bash-4.1$ ls
2013-10-23T08:19:08+0300 ^[[0m^[[01;34m99^[[0m #avail.info ^[[01;34mgmoTemp^[[0m raml21.dtd SNMP4JTestAgentBC.cfg
2013-10-23T08:19:08+0300 an_mainHost_localhost_20131023081654000136.xml #avail.info~ gsh.txt ^[[01;34mresults^[[0m
2013-10-23T08:19:09+0300 ^[[m-bash-4.1$ exit
2013-10-23T08:19:09+0300 logout
But it should be:
Last login: Wed Oct 23 08:17:38 EEST 2013 from 10.9.167.55 on pts/0
Last login: Wed Oct 23 08:19:05 2013 from 10.9.167.55
-bash-4.1$ date
Wed Oct 23 08:19:07 EEST 2013
-bash-4.1$ ls
99 #avail.info gmoTemp raml21.dtd SNMP4JTestAgentBC.cfg
an_mainHost_localhost_20131023081654000136.xml #avail.info~ gsh.txt results
-bash-4.1$ exit
logout
The messy codes are terminal control escape sequences, you can use command "infocmp xterm" and "man terminfo" to get more details.
My question is how can I remove these terminal control escape sequences in the file?
Thanks a lot!
Simple way to remove most parts of the control character is using the command below in vim:
:%s/<escape-key>\[[0-9;]*m/ /g
Press Ctrl+V followed by esc-key for the <escape-key> character above. Everything else is the same literal key as in your keyboard.
i use a pipe or direct sed like this
sed 's/[^[:print:]]\[[^a-zA-Z]*[a-zA-Z]//g' YourFile
I solved this issue using lots of regular expressions according to http://invisible-island.net/xterm/ctlseqs/ctlseqs.html

one week information of alertlofile

I want to view the ORA errors in alertlogfile of past 7 (monday-sunday)days,
by writting in shell scripts.
Can anybody help me.
Thanks
Something like:
sed -n -e '/start_time/,/end_time/ {/ORA/ p}' logfile
or with awk
$ start="Fri Feb 27 08:00:00 2009"
$ end="Fri Mar 6 08:00:00 2009"
$ awk -v prev="$start" -v last="$end" '$0 ~ prev,$0 ~ last' logfile
A more sophisticated script looking for last date entries in ORA file is available here, but also at dba-oracle.com
This does not answer exactly your request but might give you some clues to start your own script.
I want the scripts which give output as follows (one week errors) and it should be mail to my id.
Sat Mar 14 10:30:51 IST 2009
ORA-01157: cannot identify/lock data file 2 - see DBWR trace file
Sat Mar 12 12:35:06 IST 2009
ORA-01110: data file 2: '/u02/oradata/Globe/undotbs01.dbf'
Sat Mar 10 09:54:05 IST 2009
ORA-27037: unable to obtain file status
Sat Mar 08 :15:02 IST 2009
ORA-1157 signalled during: ALTER DATABASE OPEN...
Sat Mar 07 12:35:51 IST 2009
ORA-01157: cannot identify/lock data file 2 - see DBWR trace file

Resources