Getting number of newlines and storing each in a variable - linux

I am making a script that will let you choose between which interface you want to use.
I need a way to get the interfaces and store each of them in a variable.
Here is my code, but it only gets the interfaces:
Interfaces=$(ifconfig | awk '{print $1}' | grep ':' | tr -d ':')

You need to only check the lines that contain the interface name, not the lines with details. In ifconfig, detail lines start with a space; in ip, interface lines start with a number.
In bash, you can use select to create a simple menu:
#! /bin/bash
select interface in $(ip link show | grep '^[0-9]' | cut -f2 -d:) ; do
if [[ $interface ]] ; then
echo You selected $interface
break
fi
done
or
select interface in $(ifconfig -a | grep -v '^ ' | cut -f1 -d' ') ; do
if [[ $interface ]] ; then
echo You selected $interface
break
fi
done

Related

Unix Script loop through individual variables in a list and execute code

I have been busting my head all day long without coming up with a sucessfull solution.
Setup:
We have Linux RHEL 8.3 and a file, script.sh
There is an enviroment variable set by an application with a dynamic string in it.
export PROGARM_VAR="abc10,def20,ghi30"
The delimiter is always "," and the values inside vary from 1 to 20.
Inside the script I have defined 20 variables which take the values
using "cut" command I take each value and assign it to a variable
var1=$(echo $PROGARM_VAR | cut -f1 -d,)
var2=$(echo $PROGARM_VAR | cut -f2 -d,)
var3=$(echo $PROGARM_VAR | cut -f3 -d,)
var4=$(echo $PROGARM_VAR | cut -f4 -d,)
etc
In our case we will have:
var1="abc10" var2="def20" var3="ghi30" and var4="" which is empty
The loop must take each variable, test if its not empty and execute 10 pages of code using the tested variable. When it reaches an empty variable it should break.
Could you give me a hand please?
Thank you
Just split it with a comma. There are endless possibilities. You could:
10_pages_of_code() { echo "$1"; }
IFS=, read -a -r vars <<<"abc10,def20,ghi30"
for i in "${vars[#]}"; do 10_pages_of_code "$i"; done
or:
printf "%s" "abc10,def20,ghi30" | xargs -n1 -d, bash -c 'echo 10_pages_of_code "$1"' _
A safer code could use readarray instead of read to properly handle newlines in values, but I doubt that matters for you:
IFS= readarray -d , -t vars < <(printf "%s" "abc10,def20,ghi30")
You could also read in a stream up:
while IFS= read -r -d, var || [[ -n "$var" ]]; do
10_pages_of_code "$var"
done < <(printf "%s" "abc10,def20,ghi30")
But still you could do it with cut... just actually write a loop and use an iterator.
i=0
while var=$(printf "%s\n" "$PROGARM_VAR" | cut -f"$i" -d,) && [[ -n "$var" ]]; do
10_pages_of_code "$var"
((i++))
done
or
echo "$PROGRAM_VAR" | tr , \\n | while read var; do
: something with $var
done

Bash Command Substitution fail

I need some Specs and Data from Number of Latops in a ini-file.
When I try to get the IP-Adress with ip addr list <interface> nothing happens in the script. Not even a Error.
I try to quote stuff in my code. Nothing change.
for w2 in /sys/class/net/wl*
do
w2i=$(basename $w2)
echo $w2i
addr=$(ip -o -4 addr list $w2i | awk '{print $4}' | cut -d/ -f1)
echo $addr
echo -e "$w2i=$addr"
done
I Think maybe it is because of the Varibales but this fails also (interface is set 'manually'):
for w2 in /sys/class/net/wl*
do
w2i=$(basename $w2)
echo $w2i
addr=$(ip -o -4 addr list wlp3s0 | awk '{print $4}' | cut -d/ -f1)
echo $addr
echo -e "$w2i=$addr"
done
When i run the script i get wlp3s0 and i must close the script with ctl-c. When i run ip -o -4 addr list wlp3s0 | awk '{print $4}' | cut -d/ -f1 it gives me my ip like i expected.
EDIT
I just need to wirte the full-path of the ip-command. Can somebody explain why i have to do this? And especially it is not necessary for cut or awk that also based in /usr/bin/?
for w2 in /sys/class/net/wl*
do
w2i=$(basename $w2)
echo $w2i
addr=$(/usr/bin/ip -o -4 addr list "$w2i" | awk '{print $4}' | cut -d/ -f1)
echo $addr
echo -e "$w2i=$addr"
done
}

Optimizing Bash script, subshell removal

I have a bash script that lists the amount of ip addresses connected on a port. My issue is, is that with large amounts of connections it is slow as poo. I think it is because of the subshells in use, but I am having trouble removing them without borking the rest of the script. Here is the script in its entirety as it is fairly short:
#!/bin/bash
portnumber=80
reversedns_enabled=0
[ ! -z "${1}" ] && portnumber=${1}
[ ! -z "${2}" ] && reversedns_enabled=${2}
#this will hold all of our ip addresses extracted from netstat
ipaddresses=""
#get all of our connected ip addresses
while read line; do
ipaddress=$( echo ${line} | cut -d' ' -f5 | sed s/:[^:]*$// )
ipaddresses="${ipaddresses}${ipaddress}\n"
done < <( netstat -ano | grep -v unix | grep ESTABLISHED | grep \:${portnumber} )
#remove trailing newline
ipaddresses=${ipaddresses%%??}
#output of program
finaloutput=""
#get our ip addresses sorted, uniq counted, and reverse sorted based on amount of uniq
while read line; do
if [[ ${reversedns_enabled} -eq 1 ]]; then
reversednsname=""
#we use justipaddress to do our nslookup(remove the count of uniq)
justipaddress=$( echo ${line} | cut -d' ' -f2 )
reversednsstring=$( host ${justipaddress} )
if echo "${reversednsstring}" | grep -q "domain name pointer"; then
reversednsname=$( echo ${reversednsstring} | grep -o "pointer .*" | cut -d' ' -f2 )
else
reversednsname="reverse-dns-not-found"
fi
finaloutput="${finaloutput}${line} ${reversednsname}\n"
else
finaloutput="${finaloutput}${line}\n"
fi
done < <( echo -e ${ipaddresses} | uniq -c | sort -r )
#tabulate that sheet son
echo -e ${finaloutput} | column -t
The majority of the time spent is doing this operation: echo ${line} | cut -d' ' -f5 | sed s/:[^:]*$// what is the best way to inline this to produce a faster script. It takes well over a second with 1000 concurrent users (which is my base target, although should be able to process more without using up all of my cpu).
You could reduce that with cut -d' ' <<< "$line" | sed .... You could write a more complex sed script and avoid the use of cut.
But the real benefit would be in avoiding the loop so there's only one sed (or awk or perl or …) script involved. I'd probably look to reduce it to ipaddresses=$(netstat -ano | awk '...') so that instead of 3 grep processes, plus one cut and sed per line, there was just a single awk process.
ipaddresses=$(netstat -ano |
awk " /unix/ { next } # grep -v unix
!/ESTABLISHED/ { next } # grep ESTABLISHED
!/:${portnumber}/ { next } # grep :${portnum} "'
{ sub(/:[^:]*$/, "", $5); print $5; }'
)
That's probably rather clumsy, but it is a fairly direct transliteration of the existing code. Watch for the quotes to get ${portnumber} into the regex.
Since you feed the list of IP addresses into uniq -c and sort -r. You probably should use sort -rn, and you could use awk to do the uniq -c, too.
The only bit that you can't readily improve is host; that seems to only take one host or IP address argument at a time, so you have to run it for each name or address.
I'll take a stab at a couple of issues:
The following line from the script which performs incremental string concatenation will not be be efficient without the means to allocate a reasonable buffer:
ipaddresses="${ipaddresses}${ipaddress}\n"
For another, using a while loop with read line when a pipeline will do is significantly worse than the pipeline. Try something like this instead of the first loop:
netstat -ano |
grep -v 'unix' |
grep 'ESTABLISHED' |
grep "\:${portnumber}" |
cut -d' ' -f5 |
sed 's/:[^:]*$//' |
while read line; do ...
Also, try combining at least two of the three sequential grep commands into one invocation of grep.
If nothing else, this will mean you are no longer spawning a pipeline which creates new cut and sed processes for each line of input processed in the first loop.
Here is a whole script optimized & refactored:
#!/bin/bash
portnumber=80
reversedns_enabled=0
[[ $1 ]] && portnumber=$1
[[ $2 ]] && reversedns_enabled=$2
#this will hold all of our ip addresses extracted from netstat
ipaddresses=''
#get all of our connected ip addresses
while IFS=' :' read -r type _ _ _ _ ipaddress port state _; do
if [[ $type != 'unix' && $port == "$portnumber" && $state == 'ESTABLISHED' ]]; then
ipaddresses+="$ipaddress\n"
fi
done < <(netstat -ano)
#remove trailing newline
ipaddresses=${ipaddresses%%??}
#output of program
finalOutput=""
#get our ip addresses sorted, uniq counted, and reverse sorted based on amount of uniq
while read -r line; do
if (( reversedns_enabled == 1 )); then
reverseDnsName=""
#we use justipaddress to do our nslookup(remove the count of uniq)
read -r _ justipaddress _ <<< "$line"
reverseDnsString=$(host "$justipaddress")
if [[ $reverseDnsString == *'domain name pointer'* ]]; then
reverseDnsName=${reverseDnsName##*domain name pointer }
else
reverseDnsName="reverse-dns-not-found"
fi
finalOutput+="$line $reverseDnsName\n"
else
finalOutput+="$line\n"
fi
done < <(echo -e "$ipaddresses" | sort -ur)
#tabulate that sheet son
echo -e "$finalOutput" | column -t
As you can see, there are almost no external tools used (no sed, awk or grep). Awesome!

Cycle through windows of the same application using wmcrtl

I am configuring xbindkeys to change window focus using shortcuts.
For example, I managed to create a shortcut to focus on a an application window, let's say a terminator window:
wmctrl -xa terminator
Unfortunately it focuses always at the same terminator window, preventing me to cycle through the terminator windows.
Could you suggest me a command to focus on a terminator window and, if pressed again, will cycle through all the terminator windows, please?
UPDATE 30 Mar 2013
I modified this script
http://lars.st0ne.at/blog/switch%20between%20windows%20within%20the%20same%20application
to make a script such that
script.sh NAME
focus on application NAME or cycle through all the windows of NAME if a window of it is already focused, but it doesn't work properly.
Here is the script
win_class=$1 # 'terminator' # $1
# get list of all windows matching with the class above
win_list=$(wmctrl -x -l | grep -i $win_class | awk '{print $1}' )
# get id of the focused window
active_win_id=$(xprop -root | grep '^_NET_ACTIVE_W' | awk -F'# 0x' '{print $2}')
# get next window to focus on, removing id active
switch_to=$(echo $win_list | sed s/.*$active_win_id// | awk '{print $1}')
# if the current window is the last in the list ... take the first one
if [ "$switch_to" == '' ];then
switch_to=$(echo $win_list | awk '{print $1}')
fi
# switch to window
wmctrl -i -a $switch_to
The script does focus on a windows of the application, and cycle through them until it reach a window, I guess the last created. At that point, cycling doesn't work anymore.
I've found a problem in the script, if no window has focus.
May you try the following modified script:
#!/bin/bash
win_class=$1 # 'terminator' # $1
# get list of all windows matching with the class above
win_list=$(wmctrl -x -l | grep -i $win_class | awk '{print $1}' )
# get id of the focused window
active_win_id=$(xprop -root | grep '^_NET_ACTIVE_W' | awk -F'# 0x' '{print $2}')
if [ "$active_win_id" == "0" ]; then
active_win_id=""
fi
# get next window to focus on, removing id active
switch_to=$(echo $win_list | sed s/.*$active_win_id// | awk '{print $1}')
# if the current window is the last in the list ... take the first one
if [ "$switch_to" == '' ];then
switch_to=$(echo $win_list | awk '{print $1}')
fi
# switch to window
wmctrl -i -a $switch_to
The script works for me.
Anyway, it seems that the script does not find the active window in you case. Therefore it manages to switch to your application but fails to cycle through. It switches to the fist window in $win_list because, the sed command fails to remove the active window ( and all list entries before ) from $win_list.
Try the the following command:
xprop -root _NET_ACTIVE_WINDOW
The output should be something like this:
_NET_ACTIVE_WINDOW(WINDOW): window id # 0x2400005
The property "_NET_ACTIVE_WINDOW" is part of the EWMH standard. see: http://standards.freedesktop.org/wm-spec/wm-spec-1.3.html
Maybe you are using a non EWMH( Extended Window Manager Hint ) compliant window manager!
Which WM are you using?
... some window manager allow to enable EWMH compatibility via configuration or plugin.
After adapting the script by st0ne, I have a version that works generically (don't need to specify the app_name). Hope that is useful to somebody. :)
#!/bin/bash
active_win_id=`xprop -root | grep '^_NET_ACTIVE_W' | awk -F'# 0x' '{print $2}' | awk -F', ' '{print $1}'`
if [ "$active_win_id" == "0" ]; then
active_win_id=""
fi
app_name=`wmctrl -lx | grep $active_win_id | awk '{print $3}'`
workspace_number=`wmctrl -d | grep '\*' | cut -d' ' -f 1`
win_list=`wmctrl -lx | grep -ri $app_name | grep " $workspace_number " | awk '{print $1}'`
# get next window to focus on, removing id active
switch_to=`echo $win_list | sed s/.*$active_win_id// | awk '{print $1}'`
# if the current window is the last in the list ... take the first one
if [ "$switch_to" == "" ];then
switch_to=`echo $win_list | awk '{print $1}'`
fi
if [[ -n "${switch_to}" ]]
then
(wmctrl -ia "$switch_to") &
else
if [[ -n "$2" ]]
then
($2) &
fi
fi
exit 0
I encountered a small issue1 with tkt028's answer, but I liked what they were doing in terms of handling any generic application. But I also liked how st0ne's answer handles cycling through the windows of a specifically named application. So I combined the approaches.
My script takes an optional first argument to specify an application whose windows should be cycled. If no such windows are found, and if the optional second argument was provided, it falls back to launching the command specified by the second argument.
If no arguments are provided at all, then it just cycles through the windows of the currently active application.
#!/bin/bash
if [[ "$1" == "-h" || "$1" == "--help" ]]; then
echo "Cycle through windows of the active, or specified, application."
echo ""
echo "Usage: $(basename $0) [window_class_name [application_launcher]]"
echo ""
echo " window_class_name: regex string specifying an application's window name,"
echo " as specified by the third column of"
echo " 'wmctrl -l -x'"
echo " application_launcher: application to optionally launch if no windows"
echo " matching window_class_name are found"
echo ""
echo "If no arguments are specified, cycles through the windows of the active application."
exit
fi
# get ID of active window
active_win_id=`xprop -root | grep '^_NET_ACTIVE_W' | awk -F'# 0x' '{print $2}' | awk -F', ' '{print $1}'`
if [ "$active_win_id" == "0" ]; then
active_win_id=""
fi
if [[ -n "$1" ]]; then
# get app name from input argument
app_name="$1"
else
# get corresponding app name
app_name="${app_name:-$(wmctrl -lx | grep $active_win_id | awk '{print $3}')}"
fi
# get active workspace number
workspace_number=`wmctrl -d | grep '\*' | cut -d' ' -f 1`
# get list of windows corresponding to the desired app
win_list=`wmctrl -lx | grep -i $app_name | grep " $workspace_number " | awk '{print $1}'`
# get next window of app to focus on
#
# (Parses $win_list as a single string, removing everything except the token
# after the active ID. If active ID is sole token or last token, string will be
# left unmodified, producing an array from which we'll extract the first element.)
# Note: If active window was not of class app_name, then this will end up
# selecting the first window of app_name, if running. Otherwise, we'll fall
# through to launching a new instance of the app in the else of the next block.
switch_to=($(echo $win_list | sed "s/.*\<\(0x0\+\)\?$active_win_id\>\s*\(\<0x[0-9a-f]\+\>\).*/\2/"))
# if we have a valid window to switch to, do so
if [[ -n "${switch_to}" ]]; then
wmctrl -ia "${switch_to[0]}"
exit $?
else
# if the user specified a fallback application to run if target window
# was not found, try to launch it
if [[ -n "$2" ]]; then
$2 &
# check whether process corresponding to PID of background
# process we just launched is still running
ps -p $! > /dev/null
exit $?
else
exit $?
fi
fi
1 The recursive grep on this line in tkt028's answer didn't work in my environment. Maybe it's dependent on your version of grep.
win_list=`wmctrl -lx | grep -ri $app_name | grep " $workspace_number " | awk '{print $1}'`
I simply removed the r argument from the grep, and then their script worked as advertised.
win_list=`wmctrl -lx | grep -i $app_name | grep " $workspace_number " | awk '{print $1}'`

Writing bash code for performance standards

Is there a better way to rewrite this code to get enhanced performance?
If you were to get a bunch of IPs the system seems to hang.
TMP_PREFIX='/tmp/synd'
TMP_FILE="mktemp $TMP_PREFIX.XXXXXXXX"
BANNED_IP_MAIL=`$TMP_FILE`
BANNED_IP_LIST=`$TMP_FILE`
echo "Banned the following ip addresses on `date`" > $BANNED_IP_MAIL
echo >> $BANNED_IP_MAIL
BAD_IP_LIST=`$TMP_FILE`
netstat -ntu | grep SYN_RECV | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr > $BAD_IP_LIST
cat $BAD_IP_LIST
if [ $KILL -eq 1 ]; then
IP_BAN_NOW=0
while read line; do
CURR_LINE_CONN=$(echo $line | cut -d" " -f1)
CURR_LINE_IP=$(echo $line | cut -d" " -f2)
if [ $CURR_LINE_CONN -lt $NO_OF_CONNECTIONS ]; then
break
fi
IGNORE_BAN=`grep -c $CURR_LINE_IP $IGNORE_IP_LIST`
if [ $IGNORE_BAN -ge 1 ]; then
continue
fi
IP_BAN_NOW=1
echo "$CURR_LINE_IP with $CURR_LINE_CONN SYN_RECV connections" >> $BANNED_IP_MAIL
echo $CURR_LINE_IP >> $BANNED_IP_LIST
echo $CURR_LINE_IP >> $IGNORE_IP_LIST
if [ $CSF_BAN -eq 1 ]; then
$CSF -d $CURR_LINE_IP
else
$IPT -I INPUT -s $CURR_LINE_IP -j DROP
fi
done < $BAD_IP_LIST
if [ $IP_BAN_NOW -eq 1 ]; then
dt=`date`
hn=`hostname`
if [ $EMAIL_TO != "" ]; then
cat $BANNED_IP_MAIL | mail -s "IP addresses banned on $dt $hn" $EMAIL_TO
fi
fi
fi
rm -f $TMP_PREFIX.*
Sure, there are lots of ways that can be improved, but you should try to figure out where the real bottleneck is. (It may well be iptables, in which case you might want to try to do all the table updates in a single invocation instead of one at a time. But I'm just guessing.)
Here are a few suggestions; I didn't read all the way through:
netstat -ntu | grep SYN_RECV | awk '{print $5}' | cut -d: -f1 |
sort | uniq -c | sort -nr > $BAD_IP_LIST
If you're only interested in connections in SYN_RECV state, why list udp? Anyway, you're using three utilities (grep, awk and cut) to do one simple line-oriented action. You might as well just do it all in one, for example awk:
awk '$6 == "SYN_RECV" {print substr($5, 1, index($5, ":") - 1)}'
In fact, you could do the uniquifying and counting in awk as well:
awk '$6 == "SYN_RECV" {++ip[substr($5, 1, index($5, ":") - 1)]} END{for (i in ip) print ip[i], i}'
Edit: you could also filter by required count here:
awk '$6 == "SYN_RECV" {++ip[substr($5, 1, index($5, ":") - 1)]}
END {for (i in ip) if (ip[i] >= '$NO_OF_CONNECTIONS') print ip[i], i}'
Now you only need to output the ip address, since you no longer need to filter in the bash script. I don't know if that's faster than piping through sort and uniq and sort again, but it might very well be.
while read line; do
CURR_LINE_CONN=$(echo $line | cut -d" " -f1)
CURR_LINE_IP=$(echo $line | cut -d" " -f2)
if [ $CURR_LINE_CONN -lt $NO_OF_CONNECTIONS ]; then
break
fi
You want to read two fields from stdin. Why don't you just do that:
while read CURR_LINE_CONN CURR_LINE_IP IGNORED &&
((CURR_LINE_CONN >= NO_OF_CONNECTIONS)); do
That saves two subshells and two cut invocations. (The IGNORED in the read built-in is just paranoia, since there will only be two fields output by awk. It's not good paranoia, though, because it silently ignores errors.)
Edit: as above, you could get rid of the test here, too. So it would just be:
netstat -nt |
awk '$6 == "SYN_RECV" {++ip[substr($5, 1, index($5, ":") - 1)]}
END { for (i in ip)
if (ip[i] >= '$NO_OF_CONNECTIONS')
print ip[i], i}' | tee $BAD_IP_LIST
if ((KILL)); then
IP_BAN_NOW=0
while read IP IGNORED; do
Next:
IGNORE_BAN=`grep -c $CURR_LINE_IP $IGNORE_IP_LIST`
if [ $IGNORE_BAN -ge 1 ]; then
continue
fi
grep -c makes grep read the entire input file to get the count; you only want to know if the ip is present. You want grep -q:
if $(grep -q -F -x $CURR_LINE_IP $IGNORE_IP_LIST); then continue; fi
(-F tells grep to interpret the pattern as a string instead of a regex, which is what you want since otherwise . are wildcards. -x tells grep to match the entire line. It's possible for one ip to be a prefix or a suffix or even an infix of another one, which would lead to false matches. The combination of -F and -x might be a bit faster, too, since grep can then optimize the matching quite a bit.)
There's probably more. That's as far as I got.

Resources