I have a script that prints out information regarding running jobs.
I'm interested to capture name and status (Wait/Run)
They are located on different lines (status is below name)
name = any_name
status = Wait
I tried:
myScript -status | grep name
How to display also status together with name?
Sample data file:
test
name = any_name
status = Wait
some data
Search for name and grep line after (grep and awk)
grep -A 1 "name" file
awk '/name/ {p=2} p--' file
name = any_name
status = Wait
Search for status and grep line before (grep and awk)
grep -B 1 "status" file
awk '/status/ {print s"\n"$0} {s=$0}' file
name = any_name
status = Wait
grep -C 1 shoud do the job.
From the manpage of grep:
Print NUM lines of output context.
I looked almost everywhere (there, there, there, there and there) with no luck.
What I have here is a bunch of log files in a directory, where I need to look for a specific ID (myID) and sort the output by date. Here is an example :
in file1.log :
2015-09-26 15:39:50,788 - DEBUG - blabla : {'id' : myID}
in file2.log:
2015-09-26 15:39:51,788 - ERROR - foo : {'id' : myID}
in file3.log:
2015-09-26 15:39:48,788 - ERROR - bar : {'id' : myID}
Exepected output :
2015-09-26 15:39:48,788 - ERROR - bar : {'id' : myID}
2015-09-26 15:39:50,788 - DEBUG - blabla : {'id' : myID}
2015-09-26 15:39:51,788 - ERROR - foo : {'id' : myID}
What I am doing now (and it works pretty well), is :
grep -hri --color=always "myID" | sort -n
The only problem is that with the -h option of grep, the file names are hidden. I'd like to keep the file names AND keep the sorting.
I tried :
grep -ri --color=always "myID" | sort -n -t ":" -k1,1 -k2,2
But it doesn't work. Basically, the grep command outputs the name of the file followed by ":", I'd like to sort the results from this character.
Thanks a lot
Try this:
grep --color=always "myID" file*.log | sort -t : -k2,2 -k3,3n -k4,4n
Output:
file3.log:2015-09-26 15:39:48,788 - ERROR - bar : {'id' : myID}
file1.log:2015-09-26 15:39:50,788 - DEBUG - blabla : {'id' : myID}
file2.log:2015-09-26 15:39:51,788 - ERROR - foo : {'id' : myID}
Another solution, a little bit longer but I think it should work:
grep -l "myID" file* > /tmp/file_names && grep -hri "myID" file* | sort -n > /tmp/grep_result && paste /tmp/file_names /tmp/grep_result | column -s $'\t' -t
What it does basically is, first store files names by:
grep -l "myID" file* > /tmp/file_names
Store grep sorted results:
grep -hri "myID" file* | sort -n > /tmp/grep_result
Paste the results column-wise (using a tab separator):
paste /tmp/file_names /tmp/grep_result | column -s $'\t' -t
The column ordering for sort is 1-based, so k1 will be your filename part. That means that in your attempt, you are sorting by filename, then by date and hour of your log line. Also, the -n means that you are using numeric ordering, which won't be playing nicely with yyyy-mm-dd hh:mm:ss format (it will read yyyy-mm-dd hh as only the first number, i.e. the year).
You can use:
sort -t ":" -k2
Note that I specified column 2 as the start, and left the end blank. The end defaults to the end-of-line.
If you want to sort specific columns, you need to explicitly set the start and end, for example: -k2,2. You can use this to sort out-of-sequence columns, for example -k4,4 -k2,2 will sort by column 4 and use column 2 for tie-breaking.
You could also use -k2,4, which would stop sorting at the colon just before your log details (i.e. it would use 2015-09-26 15:39:48,788 - ERROR - bar)
Finally, perhaps you want to have your log files in a consistent order if the time is the same:
sort -t ":" -k2,4 -k1,1
Try rust-based tool Super Speedy Syslog Searcher
(assuming you have rust installed)
cargo install super_speedy_syslog_searcher
then
s4 file1.log file2.log file3.log | grep "myID"
The only problem is that with the -h option of grep, the file names are hidden. I'd like to keep the file names AND keep the sorting.
You could try
$ s4 --color=never -nw file1.log file2.log file3.log | grep "myID"
file1.log:2015-09-26 15:39:48,788 - ERROR - bar : {'id' : myID}
file2.log:2015-09-26 15:39:50,788 - DEBUG - blabla : {'id' : myID}
file3.log:2015-09-26 15:39:51,788 - ERROR - foo : {'id' : myID}
I have a log.text file with this structure:
user session login_time application database db_connect_time request request_time connection_source connection_ip request_state
+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------
admin 0 9 0 none 0 Not Requested* a00:bf32::
admin 989855740 1335 DRRDEVMH DRRPRODB 1201 none 0 Not Requested a00:8a45::
admin 1768947706 932 test test 916 none 0 Not Requested a00:94b6::
WARNING - 1241024 - Possible string truncation in column 1.
WARNING - 1241028 - Output column defined with warnings.
WARNING - 1241024 - Possible string truncation in column 9.
WARNING - 1241028 - Output column defined with warnings.
WARNING - 1241024 - Possible string truncation in column 10.
WARNING - 1241028 - Output column defined with warnings.
OK/INFO - 1241044 - Records returned: [3].
As we can see in the last line of log.txt there is a string Records returned: [3]. That digit 3 is my target, extracting that digit (as 3 in this case) I want to print following line in a separate file.
The total records returned = 3
I am using:
sed -n 's#^.*Records returned.*[\(.*\)$#\1#p' log.txt > out.txt
but its not giving the result. What mistake I am making here, please ?
you need to escape [, try this one
sed -n 's#^.*Records returned.*\[\(.*\)\].*$#\1#p' log.txt > out.txt
Edit
If you want to print out the string like this
The total records returned = 3
just prepend The total records returned = before \1, so the script will be
sed -n 's#^.*Records returned.*\[\(.*\)\].*$#The total records returned = \1#p' log.txt > out.txt
Using awk
awk -F "[][]" '$0~t {print "The total",t,"=",$2}' t="Records returned" log.txt > out.txt
cat out.txt
The total Records returned = 3
sed -n '$ s/.*\([[:digit:]]\{1,\}\)].$/The total records returned = \1/p'
Assuming, as your sample and explaination state, that info is on the last line with this format.
Suppose Your Data in a Test.txt file then You can simply use below Command
echo "Total Records Count = `cat Test.txt | tail -n 1 | cut -d '[' -f2 | cut -d ']' -f1` "
Total Records Count = 3
Basically I am trying to read information from three text files in which it contains unique information.
The way the text file is setup is this:
textA.txt
----------------
something.awesome.com
something2.awesome.com
something3.awesome.com
...
textB.txt
----------------
123
456
789
...
textC.txt
----------------
12.345.678.909
87.65.432.1
102.254.326.12
....
Now what its suppose to look like when i output it something like this
something.awesome.com : 123 : 12.345.678.909
something2.awesome.com : 456 : 87.65.432.1
something3.awesome.com : 789 : 102.254.326.12
The code I am trying now is this:
for each in `cat site.txt` ; do
site=`echo $each | cut -f1`
for line in `cat port.txt` ; do
port=`echo $line | cut -f1`
for this in `cat ip.txt` ; do
connect=`echo $this | cut -f1`
echo "$site : $port : $connect"
done
done
done
The result I am getting is just crazy wrong and just not what i want. I don't know how to fix this.
I want to be able to call the information through variable form.
paste testA.txt testB.txt testC.txt | sed -e 's/\t/ : /g'
Output is:
something.awesome.com : 123 : 12.345.678.909
something2.awesome.com : 456 : 87.65.432.1
something3.awesome.com : 789 : 102.254.326.12
Edit: Here is a solution using pure bash:
#!/bin/bash
exec 7<testA.txt
exec 8<testB.txt
exec 9<testC.txt
while true
do
read site <&7
read port <&8
read connect <&9
[ -z "$site" ] && break
echo "$site : $port : $connect"
done
exec 7>&-
exec 8>&-
exec 9>&-
Have you looked at using paste ?
e.g.
$ paste testA.txt testB.txt
etc. The -d operator will specify a separator character.
A related utility is the SQL-like join, which you can use in scenarios where you have to join using fields common to your input files.
head -2 /etc/hosts | tail -1 | awk '{print$2}'
where /etc/hosts is the name of a file.
(head -2 ) is used to retrieve top 2 lines from the file.
(tail -1) is used to retrieve only last one line outputed from (head -2).
(awk '{print$2}') is used to print the 2nd column of line outputted from (tail -1).
I've written a script to get the browser version of users but I need to clean up the output. What the script does is looks at the apache logs for # and IE8 then emails me the information. The problem I have is the output as when the grep finds a email address and IE8 it gives me the full output - i.e. /page/code/user#foobar.com/home.php whereas the output i'm looking is the just the email address and to only have this information recorded once a day:
Example:
user#foobar IE8
Thanks
#!/bin/bash
#Setting date and time (x and y and z aren't being used at the moment)
x="$(date +'%d/%b/%Y')"
y="$(date +'%T')"
z="$(date +'%T' | awk 'BEGIN { FS =":"} ; {print $1}')"
#Human readable for email title
emaildate=$(date +"%d%b%Y--Hour--%H")
#Setting date and time for grep and filename
beta="$(date +'%d/%b/%Y:%H')"
sigma="$(date +'%d-%b-%Y-%H')"
#CurrentAccess logs
log='/var/logs/access.log'
#Set saved log location
newlogs=/home/user/Scripts/browser/logs
#Prefrom the grep for the current day
grep # $log | grep $beta | awk 'BEGIN { FS = " " } ; { print $7 }' | sort -u >> $newlogs/broswerusage"$sigma".txt
mail -s "IE8 usage for $emaildate" user#exmaple.com < $newlogs/broswernusage"$sigma".txt