Split the content of a string using as delimiter the "=" - linux

How can I split the following output, in order to store the ttl value (64 and 128) into a loop variable?
64 bytes from client2 (192.168.42.5): icmp_seq=1 ttl=64 time=0.324 ms
64 bytes from server (192.168.42.6): icmp_seq=1 ttl=128 time=0.663 ms
Thanks in advance
Regards

The following pipeline is one way to get the specific items you want. The grep extracts only the ttl=something bit and the cut removes the ttl=:
grep -o 'ttl=[^ ]*' | cut -c5-
You can see this in the following transcript:
pax> printf 'AA ttl=64 BB\nCC ttl=128 DD\n'
AA ttl=64 BB
CC ttl=128 DD
pax> printf 'AA ttl=64 BB\nCC ttl=128 DD\n' | grep -o 'ttl=[^ ]*' | cut -c5-
64
128
Or from a real ping command:
:: ping -c 5 127.0.0.1 | grep -o 'ttl=[^ ]*' | cut -c5-
64
64
64
64
64
There are no doubt other pipelines that can do the same thing (perhaps even simpler) but that's the first one that popped into my head. The grep -o flag is quite handy for displaying only the matched text rather than the entire line.
For example, a more complete solution for handling that output may be as follows:
pax> (
...> echo '64 bytes from client2 (192.168.42.5): icmp_seq=1 ttl=64 time=0.324 ms'
...> echo '64 bytes from server (192.168.42.6): icmp_seq=1 ttl=128 time=0.663 ms'
...> ) | awk '
...> / bytes from / {
...> gsub(/ttl=/, "", $7)
...> gsub(/\(/, "", $5)
...> gsub(/\):/, "", $5)
...> print $4" "$5" "$7
...> }' | while read NAME IP TTL ; do
...> echo "Machine ${NAME} with IP ${IP} has TTL ${TTL}"
...> done
Machine client2 with IP 192.168.42.5 has TTL 64
Machine server with IP 192.168.42.6 has TTL 128
The awk first selects the correct records, then modifies the fields so that you don't get the extra stuff (like the ttl= or the parentheses around the IP address). It then prints out three of the fields and sends that through a while loop to process them as single units (one ping response line per unit).
The body of the loop simply outputs the details but you can adjust the behaviour to do something else if desired.

You can use bash regular expressions:
while IFS= read -r line; do
if [[ $line =~ "ttl="([[:digit:]]+) ]]; then
ttl=${BASH_REMATCH[1]}
echo "do something with ttl value $ttl"
fi
done <<END
64 bytes from client2 (192.168.42.5): icmp_seq=1 ttl=64 time=0.324 ms
64 bytes from server (192.168.42.6): icmp_seq=1 ttl=128 time=0.663 ms
END
outputs
do something with ttl value 64
do something with ttl value 128
The BASH_REMATCH array variable contains the text that matched capturing parentheses. Also the 0th index of that array contains the part of the string that matched the whole regex, for example "tty=64" for the first line.

Something like:
stuff | sed -E 's/.*[[:space:]]ttl=([^[:space:]]+).*/\1/'
The regular expression is:
.* match anything
[[:space:]] a single space
ttl= ttl=
([^[:space:]]+) captures a group of one or more non-space characters
.* the rest of the string
Then the end
\1 replaces the whole line with the captured value

Thank you very much for all your answers!!!
I think that is enough to have a fair idea to start.

Related

Add line feed every 2391 byte

I am using Redhat Linux 6.
I have a file which should comes from mainframe MVS with EBCDIC-ASCII conversion.
(But I suspect some conversion may be wrong)
Anyway, I know that the record length is 2391 byte. There are 10 records and the file size is 23910 byte.
For each 2391 byte record, there are many 0a or 0d char (not CRLF). I want to replace them with, say, # and #.
Also, I want to add a LF (i.e.0a) every 2391 byte so as to make the file become a normal unix text file for further processing.
I have try to use
dd ibs=2391 obs=2391 if=emyfile of=myfile.new
But, this cannot work. Both files are the same.
I also try
dd ibs=2391 obs=2391 if=myfile | awk '{print $0}'
But, this also not work
Can anyone help on this ?
Something like this:
#!/bin/bash
for i in {0..9}; do
dd if=emyfile bs=2391 count=1 skip=$i | LC_CTYPE=C tr '\r\n' '##'
echo
done > newfile
If your files are longer, you will need more than 10 iterations. I would look to handle that by running an infinite looop and exiting the loop on error, like this:
#!/bin/bash
i=0
while :; do
dd if=emyfile bs=2391 count=1 skip=$i | LC_CTYPE=C tr '\r\n' '##'
[ ${PIPESTATUS[0]} -ne 0 ] && break
echo
((i++))
done > newfile
However, on my iMac under OSX, dd doesn't seem to exit with an error when you go past end of file - maybe try your luck on your OS.
You could try
$ dd bs=2391 cbs=2391 conv=ascii,unblock if=emyfile of=myfile.new
conv=ascii converts from EBCDIC to ASCII. conv=unblock inserts a newline at the end of each cbs-sized block (after removing trailing spaces).
If you already have a file in ASCII and just want to replace some characters in it before splitting the blocks, you could use tr(1). For example, the following will replace each carriage return with '#' and each newline (linefeed) with '#':
$ tr '\r\n' '##' < emyfile | dd bs=2391 cbs=2391 conv=unblock of=myfile.new

Read column from file and run commands with the values

I have such textfile:
313 "88.68.245.12"
189 "87.245.108.11"
173 "84.134.230.11"
171 "87.143.88.4"
158 "77.64.132.10"
....
I want to grep only the IP from the first 10 lines, run whois over the IP adress and from that output I want to grep the line where it says netname.
How can I achieve this?
Just loop through the file with while - read:
while IFS='"' read -r a ip c
do
echo "ip: $ip"
whois "$ip" | grep netname
done < <(head -10 file)
This is giving IFS='"' so that the field separator is a double quote ". This way, the values within double quotes will be stored in $ip.
Then, we print the ip and perform the whois | grep thing.
Finally, we feed the loop with head -10 file, so that we just read the first 10 lines.

ping script and log output and cut with grep

I want to ping a bunch of locations but not at the same time, in order so they don't timeout.
The input is for example: ping google.com -n 10 | grep Minimum >> output.txt
This will make the output of: Minimum = 29ms, Maximum = 46ms, Average = 33ms
But there are extra spaces in front of it which I don't know how to cut off, and when it outputs to the txt file it doesn't go to a new line. What I am trying to do is make it so I can copy and paste the input and ping a bunch of places once the previous finishes and log it in a .txt file and number them so it would look like:
Server 1: Minimum = 29ms, Maximum = 46ms, Average = 33ms
Server 2: Minimum = 29ms, Maximum = 46ms, Average = 33ms
Server 3: Minimum = 29ms, Maximum = 46ms, Average = 33ms
Server 4: Minimum = 29ms, Maximum = 46ms, Average = 33ms
Well, first of all, ping on linux limits packet number to send with -c, not -n.
Secondly, output of ping is not Minimum = xx ms, Maximum = yy ms, Avrage = zz ms, but rtt min/avg/max/mdev = 5.953/5.970/5.987/0.017 ms
So basically if you do something in lines of:
for server in google.com yahoo.com
do
rtt=`ping $server -c 2 | grep rtt`
echo "$server: $rtt" >> output.txt
done
You should achieve what you want.
[edit]
If cygwin is your platform, the easiest way to strip the spaces would be either what people are suggesting, sed, or then just | awk '{print $1}', will trim your line as well.
I think you might be able to solve this using sed two times and a while loop at the end:
N=1; ping google.com -n 10 | grep Minimum | sed -r 's/(Average = [[:digit:]]+ms)/\1\n/g' | sed -r s'/[[:space:]]+(Minimum)/\1/g' | while read file; do echo Server "$N": "$file"; N=$((N+1)); done >> output.txt
The steps:
The first sed fixes the newline issue:
Match the final part of the string after which you want a new line, in this case Average = [[:digit:]]+ms and put it into a group using the parenthesis
Then replace it with the same group (\1) and insert a newline character (\n) after it
The second sed removes the whitespaces, by matching the word Minimum and all whitespaces in front of it after which it only returns the word Minimum
The final while statement loops over each line and adds Server "$N": in front of the ping results. The $N was initialized to 1 at the start, and is increased with 1 after each read line
You can use sed to remove first 4 spaces :
ping google.com -n 10 | grep Minimum | sed s/^\ \ \ \ //

How to get the percent of packets received from Ping in bash?

When pinging a host I want my output just to show the percentage of packets (5 sent) received. I assume I need to use grep somehow but I can't figure out how (I'm new to bash programming). Here is where I am: ping -c 5 -q $host | grep ?. What should go in grep? I think I will have to do some arithmetic to get the percent received but I can deal with that. How can I pull out the info I need from the summary that ping will output?
So far we've got an answer using grep, sed, perl, bc, and bash. Here is one in the flavor of AWK, "an interpreted programming language designed for text processing". This approach is designed for watching/capturing real-time packet loss information using ping.
To see only packet loss information:
Command
$ ping google.com | awk '{ sent=NR-1; received+=/^.*(time=.+ ms).*$/; loss=0; } { if (sent>0) loss=100-((received/sent)*100) } { printf "sent:%d received:%d loss:%d%%\n", sent, received, loss }'
Output
sent:0 received:0 loss:0%
sent:1 received:1 loss:0%
sent:2 received:2 loss:0%
sent:3 received:2 loss:33%
sent:4 received:2 loss:50%
sent:5 received:3 loss:40%
^C
However, I find it useful to see the original input as well. For this you just add print $0; to the last block in the script:
Command
$ ping google.com | awk '{ sent=NR-1; received+=/^.*(time=.+ ms).*$/; loss=0; } { if (sent>0) loss=100-((received/sent)*100) } { print $0; printf "sent:%d received:%d loss:%d%%\n", sent, received, loss; }'
Output
PING google.com (173.194.33.104): 56 data bytes
sent:0 received:0 loss:0%
64 bytes from 173.194.33.46: icmp_seq=0 ttl=55 time=18.314 ms
sent:1 received:1 loss:0%
64 bytes from 173.194.33.46: icmp_seq=1 ttl=55 time=31.477 ms
sent:2 received:2 loss:0%
Request timeout for icmp_seq 2
sent:3 received:2 loss:33%
Request timeout for icmp_seq 3
sent:4 received:2 loss:50%
64 bytes from 173.194.33.46: icmp_seq=4 ttl=55 time=20.397 ms
sent:5 received:3 loss:40%
^C
How does this all work?
You read the command, tried it, and it works! So what exactly is happening?
$ ping google.com | awk '...'
We start by pinging google.com and piping the output into awk, the interpreter. Everything in single quotes defines the logic of our script.
Here it is in a whitespace friendly format:
# Gather Data
{
sent=NR-1;
received+=/^.*(time=.+ ms).*$/;
loss=0;
}
# Calculate Loss
{
if (sent>0) loss=100-((received/sent)*100)
}
# Output
{
print $0; # remove this line if you don't want the original input displayed
printf "sent:%d received:%d loss:%d%%\n", sent, received, loss;
}
We can break it down into three components:
{ gather data } { calculate loss } { output }
Each time ping outputs information, the AWK script will consume it and run this logic against it.
Gather Data
{ sent=NR-1; received+=/^.*(time=.+ ms).*$/; loss=0; }
This one has three actions; defining the sent, received, and loss variables.
sent=NR-1;
NR is an AWK variable for the current number of records. In AWK, a record corresponds to a line. In our case, a single line of output from ping. The first line of output from ping is a header and doesn't represent an actual ICMP request. So we create a variable, sent, and assign it the current line number minus one.
received+=/^.*(time=.+ ms).*$/;
Here we use a Regular Expresssion, ^.*(time=.+ ms).*$, to determine if the ICMP request was successful or not. Since every successful ping returns the length of time it took, we use that as our key.
For those that aren't great with regex patterns, this is what ours means:
^ starting at the beginning of the line
.* match anything until the next rule
(time=.+ ms) match "time=N ms", where N can be one or more of any character
.* match anything until the next rule
$ stop at the end of the line
When the pattern is matched, we increment the received variable.
Calculate Loss
{ if (sent>0) loss=100-((received/sent)*100) }
Now that we know how many ICMP requests were sent and received we can start doing the math to determine packet loss. To avoid a divide by zero error, we make sure a request has been sent before doing any calculations. The calculation itself is pretty simple:
received/sent = percentage of success in decimal format
*100 = convert from decimal to integer format
100- = invert the percentage from success to failure
Output
{ print $0; printf "sent:%d received:%d loss:%d%%\n", sent, received, loss; }
Finally we just need to print the relevant info.
I don't want to remember all of this
Instead of typing that out every time, or hunting down this answer, you can save the script to a file (e.g. packet_loss.awk). Then all you need to type is:
$ ping google.com | awk -f packet_loss.awk
As always, there are many different ways to do this., but here's one option:
This expression will capture the percent digits from "X% packet loss"
ping -c 5 -q $host | grep -oP '\d+(?=% packet loss)'
You can then subtract the "loss" percentage from 100 to get the "success" percentage:
packet_loss=$(ping -c 5 -q $host | grep -oP '\d+(?=% packet loss)')
echo $[100 - $packet_loss]
Assuming your ping results look like:
PING host.example (192.168.0.10) 56(84) bytes of data.
--- host.example ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4000ms
rtt min/avg/max/mdev = 0.209/0.217/0.231/0.018 ms
Piping your ping -c 5 -q through:
grep -E -o '[0-9]+ received' | cut -f1 -d' '
Yields:
5
And then you can perform your arithmetic.
echo $((100-$(ping -c 5 -q www.google.hu | sed -rn "/packet loss/ s#.*([0-9]+)%.*#\1#p")))
Try a script:
/bin/bash
rec=ping -c $1 -q $2 | grep -c "$2" | sed -r 's_$_ / \$1_' | xargs expr
Save it, and run it with two command line args. The first is number of packets, the second is the host.
Does this work for you?
bc -l <<<100-$(ping -c 5 -q $host |
grep -o '[0-9]% packet loss' |
cut -f1 -d% )
It takes the percentage reported by ping and subtracts it from 100 to get the percentage of received packets.

Read characters from a text file using bash

Does anyone know how I can read the first two characters from a file, using a bash script. The file in question is actually an I/O driver, it has no new line characters in it, and is in effect infinitely long.
The read builtin supports the -n parameter:
$ echo "Two chars" | while read -n 2 i; do echo $i; done
Tw
o
ch
ar
s
$ cat /proc/your_driver | (read -n 2 i; echo $i;)
I think
dd if=your_file ibs=2 count=1 will do the trick
Looking at it with strace shows it is effectively doing a two bytes read from the file.
Here is an example reading from /dev/zero, and piped into hd to display the zero :
dd if=/dev/zero bs=2 count=1 | hd
1+0 enregistrements lus
1+0 enregistrements écrits
2 octets (2 B) copiés, 2,8497e-05 s, 70,2 kB/s
00000000 00 00 |..|
00000002
echo "Two chars" | sed 's/../&\n/g'
G'day,
Why not use od to get the slice that you need?
od --read-bytes=2 my_driver
Edit: You can't use head for this as the head command prints to stdout. If the first two chars are not printable, you don't see anything.
The od command has several options to format the bytes as you want.

Resources