I am trying to output the size of an ARP table from a FW using an Expect script so it can be graphed. After the below code the output displayed to screen is shown:
/usr/bin/expect -f -<< EOD
spawn ssh test#1.2.3.4
sleep 1
expect {
"*word:" {send "password\r"}
}
sleep 1
expect {
"*>" {send "show arp all | match \"total ARP entries in table\"\r"}
}
sleep 1
expect {
"*>" {send "exit\r"}
}
expect eof
EOD
spawn ssh test#1.2.3.4
FW-Active
Password:
Number of failed attempts since last successful login: 0
test#FW-Active(active)> show arp all | match "total ARP entries in table"
total ARP entries in table : 2861
What I am trying to do is be able to output only the numeric value indicated from the total ARP entries in table. I am assuming I need to some how do a "cut" or "awk" or something to extract only the numbers but I am not having any luck. Any help is greatly appreciated.
You store the output of that whole command in a variable, let's say a.
Something like this will probably work. Since you're using expect, you might want to figure out how to store that output as a variable that way you can manipulate it. I stored the output as $a in my example.
$ echo $a
total ARP entries in table : 2861
$ echo ${a% *}
total ARP entries in table :
$ echo ${a% *}-
total ARP entries in table : -
$ echo ${a##* }
2861
Logic explanation (Parameter/Variable Substituion in BASH):
1) To removing/stripping the left hand side part, use # for reaching the first matching character value (reading / parsing from left side), ## for reaching the last matching character/value. It works by giving *<value> within the { } braces.
2) To removing/stripping the right hand side part, use % for reaching the first matching character value (reading / parsing from right side), %% for reaching the last matching character/value. It works by giving <value>* within the { } braces.
Or if you don't want to store the output or anything, then simply do this:
show arp all | match "total ARP entries in table" | grep -o "[0-9][0-9]*"
Or (the following assumes that you don't change
show arp all | match "total ARP entries in table" | sed "s/ *//g"|cut -d':' -f2
Related
I apologize for my question not being specific enough but I have no choice. So I received an assignment that hasn't been completely covered in the learning material (even the person assigned to help students is having trouble helping me) since this is beyond basic bash scripting. I'm not expecting anybody to do my assignment but if I can get a clue or an idea it'll be very helpful!
My assignment:
Code a script in bash linux that will use user's input of number of rows and number of columns, and print 'hello' strong according to the user's input, like so:
For example:
User's input of number of columns:2
User's input of number of rows: 3
hello hello
hello hello
hello hello
I thought in this direction but I can't figure it out and will appreciate any help :)
echo -e 'Please enter number of rows: \n'
read rows
echo -e 'Please enter number of columns: \n'
read columns
string='hello'
for i in $columns
do
echo $string
string+=$string
done
(this is as far as I got with the first loop as what ive done here doesn't work)
Check this out:
#!/bin/bash
read -p 'Please enter number of rows and columns: ' rows columns # prompt and read both vars at once
string='hello' # set string
printf -v row "%${columns}s" # create var $row consists on N(columns) spaces
row=${row//' '/"$string "} # recreate var $row changing spaces to "$string "
printf -v col "%${rows}s" # create var $col consists on N(rows) spaces
all=${col//' '/"$row\n"} # create full set in var $all by changing spaces to "$row\n"
printf "$all" # print all
Testing:
$ ./ex
Please enter number of rows and columns: 3 5
hello hello hello hello hello
hello hello hello hello hello
hello hello hello hello hello
With two loops:
#!/bin/bash
string='hello'
read -p "x:" x
read -p "y:" y
for ((j=0; j<$y; j++)); do
for ((i=0; i<$x; i++)); do
echo -n "$space$string"
space=" "
done
space=""
echo
done
See: man bash
To read inputs you can use read builtin. For example
read -r row column
Then you can use $row and $column variables.
You'd need a nested for loop to print row x column times.
To not print newlines, use -n option of echo.
Refer help read, help for, and help echo for details. You can obviously Google these terms, too ;-)
Do yo want to golf it? :)
printf "%$((rows*columns))s" | fold -w "$columns" | sed 's/ /hello /g'
To prompt the user for rows and colums, use the read builtin:
read -p 'Enter rows: ' rows
read -p 'Enter columns: ' columns
I prefer to get my arguments on the command line.
Accordingly, one implementation (with no error checking...):
rows=$1 # first arg is rows to output
cols=$2 # second column is columns wanted
str=$3 # third arg is the string to print
while (( rows-- )) # post-decrement rows
do c=$cols # reset a column count for each new row
while (( c-- )) # post-decrement columns done
do printf "%s " "$str" # print the string with a trailing space, NO newline
done
printf "\n" # print a newline at the end of each row
done
Make sure you understand ((...)) arithmetic processing, printf, and command line argument parsing. All these are available in the documentation.
For extra credit, do proper error checking of your inputs.
If you need to read the inputs from stdin instead of the command line, replace
rows=$1 # first arg is rows to output
cols=$2 # second column is columns wanted
str=$3 # third arg is the string to print
with
read rows cols str
Better, read each with an appropriate prompt - again, details available in the manual.
Good luck.
This seems like a simple task, but using duckduckgo I wasn't able to find a way to properly do what I'm trying to.
The main question is: How do I split the output of a command in linux or bash into multiple columns using a delimeter?
I have a file that looks like this: (this is just a simplified example)
-----------------------------------
Some data
that varies in line length
-----------------------------------
-----------------------------------
More data that is seperated
by a new line and dashes
-----------------------------------
And so on. Everytime data gets written to the file, it's enclosed in a line of dashes, seperated by an empty line from the last block. Line-length of the data varies. What I want is basically a tool or way using bash to split the file into multiple columns like this:
----------------------------------- -----------------------------------
Some data More data that is seperated
that varies in line length by a new line and dashes
----------------------------------- -----------------------------------
Each column should take 50% of the screen, no centering (as in alignment) needed. The file has to be split per-block. Splitting the file in the middle or something like that won't work. I basically want block 1 go to the left column, block 2 to the right, 3 to the left again, 4 right, and so on. The file gets updated constantly and updates should be written to the screen right away. (Currently I'm using tail -f)
Since this sounds like a rather common question I would welcome a general approach to this instead of a specific answer that works only for my case so people coming from search engines looking for a way to have a two column layout in bash get some information too. I tried column and pr, both don't work as desired. (I elaborated on this in the comments)
Edit: To be clear, I am looking for a general approach on this. Going through a file, getting data between the delimiter, putting it to column A, getting the next one putting it to column B, and so on.
The question is tagged as Perl so here is a possible Perl answer:
#!/usr/bin/env perl
use strict;
use warnings;
my $is_col1 = 1;
my $in_block = 0;
my #col1;
while (<DATA>) {
chomp;
if (/^\s*-+\s*$/ ... /^\s*-+\s*$/) {
$in_block = 1;
if ($is_col1) {
push #col1, $_;
}
else {
printf "%-40s%-40s\n", shift #col1 // '', $_;
}
}
else {
if ($in_block) {
$in_block = ! $in_block;
$is_col1 = ! $is_col1;
print "\n" if $is_col1; # line separating blocks
}
}
}
print join("\n", #col1), "\n\n" if #col1;
__DATA__
-----------------------------------
Some data
that varies in line length
-----------------------------------
-----------------------------------
More data that is seperated
by a new line and dashes
with a longer column2
-----------------------------------
-----------------------------------
The odd last column
-----------------------------------
Output:
----------------------------------- -----------------------------------
Some data More data that is seperated
that varies in line length by a new line and dashes
----------------------------------- with a longer column2
-----------------------------------
-----------------------------------
The odd last column
-----------------------------------
This script is getting max width of current terminal and splitting it in 2, then printing records split by RS="\n\n" separator, print the first found and placing the cursor at the first line/last column of it to write the next record.
#!/bin/bash
tput clear
# get half current terminal width
twidth=$(($(tput cols)/2))
tail -n 100 -f test.txt | stdbuf -i0 -o0 gawk -v twidth=$twidth 'BEGIN{ RS="\n\n"; FS=OFS="\n"; oldNF=0 } {
sep="-----------------------------------"
pad=" "
printf "%-" twidth "s", $0
getline
for(i = 1; i <= NF; i++){
# move cursor to first line, last column of previous record
print "\033[" oldNF ";" twidth "f" $i
oldNF+=1
}
}'
Here's a simpler version
gawk 'BEGIN{ RS="[-]+\n\n"; FS="\n" } {
sep="-----------------------------------"
le=$2
lo=$3
getline
printf "%-40s %-40s\n", sep,sep
printf "%-40s %-40s\n", le,$2
printf "%-40s %-40s\n", lo,$3
printf "%-40s %-40s\n\n", sep,sep
}' test.txt
Output
----------------------------------- -----------------------------------
Some data More data that is seperated
that varies in line length by a new line and dashes
----------------------------------- -----------------------------------
----------------------------------- -----------------------------------
Some data More data that is seperated
that varies in line length by a new line and dashes
----------------------------------- -----------------------------------
Assuming file contains uniform blocks of five lines each, using paste, sed, and printf:
c=$((COLUMNS/2))
paste -d'#' <(sed -n 'p;n;p;n;p;n;p;n;p;n;n;n;n;n' file) \
<(sed -n 'n;n;n;n;n;p;n;p;n;p;n;p;n;p' file) |
sed 's/.*/"&"/;s/#/" "/' |
xargs -L 1 printf "%-${c}s %-${c}s\n"
Problem with OP spec
The OP reports that the block lengths may vary, and should be separated by a fixed number of lines. Even numbered blocks go in Column A, odd numbered blocks in Column B.
That creates a tail -f problem then. Suppose the block lengths of the source input begin with 1000 lines, then one line, 1000, 1, 1000, 1, etc. So Column A gets all the 1000 line blocks, and Column B gets all the one line blocks. Let's say the blocks in the output are separated by 1 line each. So one block in Column A lines up with 500 blocks in Column B. So for a terminal with scrolling output, that means before we can output the first block in Column A, we have to wait for 1000 blocks of input. To output the third block in Column A, (just below the first block), we have to wait for 2000 blocks of input.
If the blocks are added to the input file relatively slowly, with a one second delay between blocks, then it will take three seconds for the block 3 to appear in the input file, but it will take 33 minutes for block 3 to be displayed in the output file.
Alright, since apprently there is no clean way to do this I came up with my own solution. It's a bit messy and requires GNU screen to be installed, but it works. Any amount of lines within or around the blocks, 50% of the screen automatically resizing and each column prints independantly from each other with a fixed amount of newlines between them. Also automatic updates every x seconds. (120 in my example)
#!/bin/bash
screen -S testscr -X layout save default
screen -S testscr -X split -v
screen -S testscr -X screen tail -f /tmp/testscr1.txt
screen -S testscr -X focus
screen -S testscr -X screen tail -f /tmp/testscr2.txt
while : ; do
echo "" > /tmp/testscr1.txt
echo "" > /tmp/testscr2.txt
cfile=1 # current column
ctype=0 # start or end of block
while read; do
if [[ $REPLY == "------------------------------------------------------------" ]]; then
if [[ $ctype -eq 0 ]]; then
ctype=1
else
if [[ $cfile -eq 1 ]]; then
echo "${REPLY}" >> /tmp/testscr1.txt
echo "" >> /tmp/testscr1.txt
echo "" >> /tmp/testscr1.txt
cfile=2
else
echo "${REPLY}" >> /tmp/testscr2.txt
echo "" >> /tmp/testscr2.txt
echo "" >> /tmp/testscr2.txt
cfile=1
fi
ctype=0
fi
fi
if [[ $ctype -eq 1 ]]; then
if [[ $cfile -eq 1 ]]; then
echo "${REPLY}" >> /tmp/testscr1.txt
else
echo "${REPLY}" >> /tmp/testscr2.txt
fi
fi
done < "$1"
sleep 120
done
First, start a screen session with screen -S testscr then, either within or outside the session, execute the script above. This will split the screen vertically using 50% per column and execute tail -f on both columns, afterwards it will go through the input file and write block by block to each tmp. file in the desired way. Since it's in an infinite while loop it's essentially automatically updating the shown output every x seconds (here 120).
hi I looked the others questions and tried some things but i couldnt manage to solve it.
My string is that
:CONNECTING TO CPL... PROCESS CPL CONNECTED... Enter command: Enter
command:RESP:0,CPrL-E1002:RESPMSG,Invalid session ID.; Enter
command:RESP:0,CPL-E1014:RESPMSG,System internal error; Enter command:
Connection closed by foreign host.
how Can I take RESP:0,CPL-E1014:RESPMSG,System internal error; this line? Also
RESP:0,CPL-E1014:RESPMSG,System internal error; this line wıll change according the request .So I must take only from RESP: to ; for the second line
test=":CONNECTING TO CPL... PROCESS CPL CONNECTED... Enter command: Enter command:RESP:0,CPrL-E1002:RESPMSG,Invalid session ID.; Enter command:RESP:0,CPL-E1014:RESPMSG,System internal error; Enter command:Connection closed by foreign host."
str=echo $test | cut -d";" -f2
echo ${str:15:100}";"
Here I assumed that your response string comes every time after " Enter command:". 100 is for the maximum length of your response string.
Assuming that the string is in a variable named $str, and that there are no other stars * in the string, you could use an awk filter, like this:
awk -F\* 'NF>1{print $2}' <<< "$str"
Here we use the -F option to set the field separator to the star, and if we find a line with more than one elements (one star or more), we print the second field.
output:
$ echo "$str"
:CONNECTING TO CPL... PROCESS CPL CONNECTED... Enter command: Enter
command:*RESP:0,CPrL-E1002:RESPMSG,Invalid session ID.;* Enter
command:AREE:0,CPL-E1014:RESPMSG,System internal error; Enter command:
Connection closed by foreign host.
$
$ awk -F\* 'NF>1{print $2}' <<< "$str"
RESP:0,CPrL-E1002:RESPMSG,Invalid session ID.;
You can use expr and regular expressions, please refer to this answer first. Have a look at this code :
#!/bin/bash
# foo.sh
# Searches the string for a "RESP:#" message, the # sign is a digit.
string='CONNECTING TO CPL... PROCESS CPL CONNECTED... Enter command:RESP:0,CNDB-0,CPL-0,EMA-0:RESPMSG,success; Enter command: Enter command:
RESP:1,CPL-0,EMA-0:RESPMSG,success; Enter command: Connection closed by foreign host'
regex='RESP\:[0-9]\,CPL\-0\,EMA\-0\:RESPMSG\,[a-zA-Z0-9]*\;'
line=`expr "$string" : '.*\(RESP\:[0-9]\,CPL\-0\,EMA\-0\:RESPMSG\,[a-zA-Z0-9]*\;\)'` # Search the string for the desired line.
respCode=${line:5:1} # Extract RESP:# value using substring...
respMessage=${line:27}
echo $line
echo 'Received response code : '$respCode
echo 'Received response message : '$respMessage
Try doing this using only grep :
$ grep -oP 'command:\KRESP:.*?System internal error;' test.txt
RESP:0,CPL-E1014:RESPMSG,System internal error;
I want to ping a bunch of locations but not at the same time, in order so they don't timeout.
The input is for example: ping google.com -n 10 | grep Minimum >> output.txt
This will make the output of: Minimum = 29ms, Maximum = 46ms, Average = 33ms
But there are extra spaces in front of it which I don't know how to cut off, and when it outputs to the txt file it doesn't go to a new line. What I am trying to do is make it so I can copy and paste the input and ping a bunch of places once the previous finishes and log it in a .txt file and number them so it would look like:
Server 1: Minimum = 29ms, Maximum = 46ms, Average = 33ms
Server 2: Minimum = 29ms, Maximum = 46ms, Average = 33ms
Server 3: Minimum = 29ms, Maximum = 46ms, Average = 33ms
Server 4: Minimum = 29ms, Maximum = 46ms, Average = 33ms
Well, first of all, ping on linux limits packet number to send with -c, not -n.
Secondly, output of ping is not Minimum = xx ms, Maximum = yy ms, Avrage = zz ms, but rtt min/avg/max/mdev = 5.953/5.970/5.987/0.017 ms
So basically if you do something in lines of:
for server in google.com yahoo.com
do
rtt=`ping $server -c 2 | grep rtt`
echo "$server: $rtt" >> output.txt
done
You should achieve what you want.
[edit]
If cygwin is your platform, the easiest way to strip the spaces would be either what people are suggesting, sed, or then just | awk '{print $1}', will trim your line as well.
I think you might be able to solve this using sed two times and a while loop at the end:
N=1; ping google.com -n 10 | grep Minimum | sed -r 's/(Average = [[:digit:]]+ms)/\1\n/g' | sed -r s'/[[:space:]]+(Minimum)/\1/g' | while read file; do echo Server "$N": "$file"; N=$((N+1)); done >> output.txt
The steps:
The first sed fixes the newline issue:
Match the final part of the string after which you want a new line, in this case Average = [[:digit:]]+ms and put it into a group using the parenthesis
Then replace it with the same group (\1) and insert a newline character (\n) after it
The second sed removes the whitespaces, by matching the word Minimum and all whitespaces in front of it after which it only returns the word Minimum
The final while statement loops over each line and adds Server "$N": in front of the ping results. The $N was initialized to 1 at the start, and is increased with 1 after each read line
You can use sed to remove first 4 spaces :
ping google.com -n 10 | grep Minimum | sed s/^\ \ \ \ //
When pinging a host I want my output just to show the percentage of packets (5 sent) received. I assume I need to use grep somehow but I can't figure out how (I'm new to bash programming). Here is where I am: ping -c 5 -q $host | grep ?. What should go in grep? I think I will have to do some arithmetic to get the percent received but I can deal with that. How can I pull out the info I need from the summary that ping will output?
So far we've got an answer using grep, sed, perl, bc, and bash. Here is one in the flavor of AWK, "an interpreted programming language designed for text processing". This approach is designed for watching/capturing real-time packet loss information using ping.
To see only packet loss information:
Command
$ ping google.com | awk '{ sent=NR-1; received+=/^.*(time=.+ ms).*$/; loss=0; } { if (sent>0) loss=100-((received/sent)*100) } { printf "sent:%d received:%d loss:%d%%\n", sent, received, loss }'
Output
sent:0 received:0 loss:0%
sent:1 received:1 loss:0%
sent:2 received:2 loss:0%
sent:3 received:2 loss:33%
sent:4 received:2 loss:50%
sent:5 received:3 loss:40%
^C
However, I find it useful to see the original input as well. For this you just add print $0; to the last block in the script:
Command
$ ping google.com | awk '{ sent=NR-1; received+=/^.*(time=.+ ms).*$/; loss=0; } { if (sent>0) loss=100-((received/sent)*100) } { print $0; printf "sent:%d received:%d loss:%d%%\n", sent, received, loss; }'
Output
PING google.com (173.194.33.104): 56 data bytes
sent:0 received:0 loss:0%
64 bytes from 173.194.33.46: icmp_seq=0 ttl=55 time=18.314 ms
sent:1 received:1 loss:0%
64 bytes from 173.194.33.46: icmp_seq=1 ttl=55 time=31.477 ms
sent:2 received:2 loss:0%
Request timeout for icmp_seq 2
sent:3 received:2 loss:33%
Request timeout for icmp_seq 3
sent:4 received:2 loss:50%
64 bytes from 173.194.33.46: icmp_seq=4 ttl=55 time=20.397 ms
sent:5 received:3 loss:40%
^C
How does this all work?
You read the command, tried it, and it works! So what exactly is happening?
$ ping google.com | awk '...'
We start by pinging google.com and piping the output into awk, the interpreter. Everything in single quotes defines the logic of our script.
Here it is in a whitespace friendly format:
# Gather Data
{
sent=NR-1;
received+=/^.*(time=.+ ms).*$/;
loss=0;
}
# Calculate Loss
{
if (sent>0) loss=100-((received/sent)*100)
}
# Output
{
print $0; # remove this line if you don't want the original input displayed
printf "sent:%d received:%d loss:%d%%\n", sent, received, loss;
}
We can break it down into three components:
{ gather data } { calculate loss } { output }
Each time ping outputs information, the AWK script will consume it and run this logic against it.
Gather Data
{ sent=NR-1; received+=/^.*(time=.+ ms).*$/; loss=0; }
This one has three actions; defining the sent, received, and loss variables.
sent=NR-1;
NR is an AWK variable for the current number of records. In AWK, a record corresponds to a line. In our case, a single line of output from ping. The first line of output from ping is a header and doesn't represent an actual ICMP request. So we create a variable, sent, and assign it the current line number minus one.
received+=/^.*(time=.+ ms).*$/;
Here we use a Regular Expresssion, ^.*(time=.+ ms).*$, to determine if the ICMP request was successful or not. Since every successful ping returns the length of time it took, we use that as our key.
For those that aren't great with regex patterns, this is what ours means:
^ starting at the beginning of the line
.* match anything until the next rule
(time=.+ ms) match "time=N ms", where N can be one or more of any character
.* match anything until the next rule
$ stop at the end of the line
When the pattern is matched, we increment the received variable.
Calculate Loss
{ if (sent>0) loss=100-((received/sent)*100) }
Now that we know how many ICMP requests were sent and received we can start doing the math to determine packet loss. To avoid a divide by zero error, we make sure a request has been sent before doing any calculations. The calculation itself is pretty simple:
received/sent = percentage of success in decimal format
*100 = convert from decimal to integer format
100- = invert the percentage from success to failure
Output
{ print $0; printf "sent:%d received:%d loss:%d%%\n", sent, received, loss; }
Finally we just need to print the relevant info.
I don't want to remember all of this
Instead of typing that out every time, or hunting down this answer, you can save the script to a file (e.g. packet_loss.awk). Then all you need to type is:
$ ping google.com | awk -f packet_loss.awk
As always, there are many different ways to do this., but here's one option:
This expression will capture the percent digits from "X% packet loss"
ping -c 5 -q $host | grep -oP '\d+(?=% packet loss)'
You can then subtract the "loss" percentage from 100 to get the "success" percentage:
packet_loss=$(ping -c 5 -q $host | grep -oP '\d+(?=% packet loss)')
echo $[100 - $packet_loss]
Assuming your ping results look like:
PING host.example (192.168.0.10) 56(84) bytes of data.
--- host.example ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4000ms
rtt min/avg/max/mdev = 0.209/0.217/0.231/0.018 ms
Piping your ping -c 5 -q through:
grep -E -o '[0-9]+ received' | cut -f1 -d' '
Yields:
5
And then you can perform your arithmetic.
echo $((100-$(ping -c 5 -q www.google.hu | sed -rn "/packet loss/ s#.*([0-9]+)%.*#\1#p")))
Try a script:
/bin/bash
rec=ping -c $1 -q $2 | grep -c "$2" | sed -r 's_$_ / \$1_' | xargs expr
Save it, and run it with two command line args. The first is number of packets, the second is the host.
Does this work for you?
bc -l <<<100-$(ping -c 5 -q $host |
grep -o '[0-9]% packet loss' |
cut -f1 -d% )
It takes the percentage reported by ping and subtracts it from 100 to get the percentage of received packets.