I'm pretty new to Linux and writing scripts etc. I have this task where I need to find an IP-address from a database and then grep a bunch of files with this IP and the next one to see, if they have any presence there. Currently I have to first write:
rwhois -Br 0.0.0.0
and then
grep -wl '0.0.0.0\|0.0.0.1' /path/to/some/files
And I have to manually change the last digit from the rwhois and from the grep.
I got as far as to write a simple function like this
function info () {
rhowis -Br $1
grep -w '$1\|$1'
}
But of course I'd have to somehow increase the value of the latter input by 1. Any good advice? And a small explanation of what you changed is appreciated so I can learn from this. Thanks!
It's simple to just increase the last digit with awk:
info() {
local ip="$1"
local nextip=$(awk -F. '{ print $1 "." $2 "." $3 "." ($4+1) }' <<<"$1")
rhowis -Br "$ip"
grep -w "$ip\|$nextip'
}
Note that this will not handle wrapping (when the last digit is 255), but that shouldn't be a problem if you don't need to handle broadcast addresses.
ip=$(awk -F\. '{ print $1"."$2"."$3"."($4+1) }' <<< $1)
With awk you can set up a variable ip. We set the delimited as "." and then take in the initial IP address then printing out the first,second and third delimited pieces along with the fourth incremented by one.
We would then action:
grep -w '$1\|$ip'
Related
I am writing a simple script to collect 2 IP addresses. I am using the open stack client to gather the allocation pool of a provider network. I used awk to gather the 2 IP addresses (start and end) and put them into 2 variables. When I echo the 2 variables alone they print out how I expect. However, if I try to echo something after the variable, it seems to replace the first few characters of the IP address.
It hard to explain, but if you refer to the output it should make more sense. If you look at my script below, I just put the string "hello" after the variable in each echo statement for demonstrative purposes.
#!/bin/bash
NETWORK=$1
#just gets the allocation pool IP addresses from openstack
ALLOCATION_POOLS=$(openstack subnet show $NETWORK --insecure|grep -w "allocation_pools"|awk -F " " '{print $4}')
POOL_START=$(awk -F "-" '{print $1}' <<< "$ALLOCATION_POOLS")
echo $POOL_START"hello"
POOL_END=$(awk -F "-" '{print $2}' <<< "$ALLOCATION_POOLS")
echo $POOL_END"hello"
Here is the output:
hello.146.87
hello.146.126
If I did not put "hello" in the echo statement, the output looks more like this:
10.28.146.87
10.28.146.126
Another thing I did was tested the length of the strings, and the length was larger then the number of characters in the ip address. I believe that there is some strange character after the IP addresses that is causing this. If that is the case, how can I remove it?
I am currently working with a vendor-provided software that is trying to handle sending attachment files to another script that will text-extract from the listed file. The script fails when we receive files from an outside source that contain spaces, as the vendor-supplied software does not surround the filename in quotes - meaning when the text-extraction script is run, it receives a filename that will split apart on the space and cause an error on the extractor script. The vendor-provided software is not editable by us.
This whole process is designed to be an automated transfer, so having this wrench that could be randomly thrown into the gears is an issue.
What we're trying to do, is handle the spaced name in our text extractor script, since that is the piece we have some control over. After a quick Google, it seems like changing the IFS value for the script would be the quick solution, but unfortunately, that script would take effect after the extensions have already mutilated the incoming data.
The script I'm using takes in a -e value, a -i value, and a -o value. These values are sent from the vendor supplied script, which I have no editing control over.
#!/bin/bash
usage() { echo "Usage: $0 -i input -o output -e encoding" 1>&2; exit 1; }
while getopts ":o:i:e:" o; do
case "${o}" in
i)
inputfile=${OPTARG}
;;
o)
outputfile=${OPTARG}
;;
e)
encoding=${OPTARG}
;;
*)
usage
;;
esac
done
shift $((OPTIND-1))
...
...
<Uses the inputfile, outputfile, and encoding variables>
I admit, there may be pieces to this I don't fully understand, and it could be a simple fix, but my end goal is to be able to extract -o, -i, and -e that all contain 1 value, regardless of the spaces within each section. I can handle quoting the script after I can extract the filename value
The script fragment that you have posted does not have any issues with spaces in the arguments.
The following, for example, does not need quoting (since it's an assignment):
inputfile=${OPTARG}
All other uses of $inputfile in the script should be double quoted.
What matters is how this script is called.
This would fail and would assign only hello to the variable inputfile:
$ ./script.sh -i hello world.txt
The string world.txt would prompt the getopts function to stop processing the command line and the script would continue with the shift (world.txt would be left in $1 afterwards).
The following would correctly assign the string hello world.txt to inputfile:
$ ./script.sh -i "hello world.txt"
as would
$ ./script.sh -i hello\ world.txt
The following script uses awk to split the arguments while including spaces in the file names. The arguments can be in any order. It does not handle multiple consecutive spaces in an argument, it collapses them to one.
#!/bin/bash
IFS=' '
str=$(printf "%s" "$*")
istr=$(echo "${str}" | awk 'BEGIN {FS="-i"} {print $2}' | awk 'BEGIN {FS="-o"} {print $1}' | awk 'BEGIN {FS="-e"} {print $1}')
estr=$(echo "${str}" | awk 'BEGIN {FS="-e"} {print $2}' | awk 'BEGIN {FS="-o"} {print $1}' | awk 'BEGIN {FS="-i"} {print $1}')
ostr=$(echo "${str}" | awk 'BEGIN {FS="-o"} {print $2}' | awk 'BEGIN {FS="-e"} {print $1}' | awk 'BEGIN {FS="-i"} {print $1}')
inputfile=""${istr}""
outputfile=""${ostr}""
encoding=""${estr}""
# call the jar
There was an issue when calling the jar where Java threw a MalformedUrlException on a filename with a space.
So after reading through the commentary, we decided that although it may not be the right answer for every scenario, the right answer for this specific scenario was to extract the pieces manually.
Because we are building this for a pre-built script passing to it, and we aren't updating that script any time soon, we can accept with certainty that this script will always receive a -i, -o, and -e flag, and there will be spaces between them, which causes all the pieces passed in to be stored in different variables in $*.
And we can assume that the text after a flag is the response to the flag, until another flag is referenced. This leaves us 3 scenarios:
The variable contains one of the flags
The variable contains the first piece of a parameter immediately after the flag
The variable contains part 2+ of a parameter, and the space in the name was interpreted as a split, and needs to be reinserted.
One of the other issues I kept running into was trying to get string literals to equate to variables in my IF statements. To resolve that issue, I pre-stored all relevant data in array variables, so I could test $variable == $otherVariable.
Although I don't expect it to change, we also handled what to do if the three flags appear in a different order than we anticipate (Our assumption was that they list as i,o,e... but we can't see excatly what is passed). The parameters are dumped into an array in the order they were read in, and a parallel array tracks whether the items in slots 0,1,2 relate to i,o,e.
The final result still has one flaw: if there is more than one consecutive space in the filename, the whitespace is trimmed before processing, and I can only account for one space. But saying as we processed over 4000 files before encountering one with a space, I find it unlikely with the naming conventions that we would encounter something with more than one space.
At that point, we would have to be stepping in for a rare intervention anyways.
Final code change is as follows:
#!/bin/bash
IFS='|'
position=-1
ioeArray=("" "" "")
previous=""
flagArr=("-i" "-o" "-e" " ")
ioePattern=(0 1 2)
#echo "for loop:"
for i in $*; do
#printf "%s\n" "$i"
if [ "$i" == "${flagArr[0]}" ] || [ "$i" == "${flagArr[1]}" ] || [ "$i" == "${flagArr[2]}" ]; then
((position += 1));
previous=$i;
case "$i" in
"${flagArr[0]}")
ioePattern[$position]=0
;;
"${flagArr[1]}")
ioePattern[$position]=1
;;
"${flagArr[2]}")
ioePattern[$position]=2
;;
esac
continue;
fi
if [[ $previous == "-"* ]]; then
ioeArray[$position]=${ioeArray[$position]}$i;
else
ioeArray[$position]=${ioeArray[$position]}" "$i;
fi
previous=$i;
done
echo "extracting (${ioeArray[${ioePattern[0]}]}) to (${ioeArray[${ioePattern[1]}]}) with (${ioeArray[${ioePattern[2]}]}) encoding."
inputfile=""${ioeArray[${ioePattern[0]}]}"";
outputfile=""${ioeArray[${ioePattern[1]}]}"";
encoding=""${ioeArray[${ioePattern[2]}]}"";
I'm new to shell scripting and here is my problem:
I want to store PID's from output of airmon-ng check to some variables (for ex: $1, $2, $3) so that I can execute kill $1 $2 $3.
here is sample output of airmon-ng check:
Found 3 processes that could cause trouble.
If airodump-ng, aireplay-ng or airtun-ng stops working after
a short period of time, you may want to kill (some of) them!
PID Name
707 NetworkManager
786 wpa_supplicant
820 dhclient
I want to grab numbers 707, 786, 820.
I tried using set 'airmon-ng check' and then using for loop:
set `airmon-ng check`
n=$#
for (( i=0; i<=n; i++ ))
do
echo $i
done
it outputs 1,2,3,...36
not words or numbers so I couldn't figure out how I should do it.
airmon-ng check | egrep -o '\b[0-9]+\b' | xargs kill
egrep is grep with extended regular expressions (like grep -E), -o says to extract only the matching parts, \b matches word boundaries so you don't get any numbers accidentally occuring in process names or something, [0-9]+ matches one or more decimal digit, xargs kill passes all the matches as arguments to the kill command.
Note that parsing output intended to be read by humans might not always be a good idea. Also, just killing all those processes doesn't sound too smart either, but proper usage of airocrack is beyond this question.
You can get list of the PIDs separated by spaces e.g. like this (everything from the 1st column after "PID"):
l=`airmon-ng check | awk 'BEGIN { p=0 } { if (p) { print $1" "; } if ($1=="PID") { p=1 } }' | tr '\n' ' '`
Why not use grep?
myvar=$(airmon-ng check | grep '[0-9]\{3,6\}')
This assumes a PID of 3 to 6 digits, and will grab anything from the airmon-ng output of a similar length. So this may not work as well if the output includes other strings with digits of a similar length.
I would use awk for this and store the output in an array
pids=( $(airmon-ng check | awk '/^[[:blank:]]+[[:digit:]]+[[:blank:]]+/{print $1}') )
#'pids' is an array
kill "${pids[#]}" #killing all the processes thus found.
I want to get a mac address from arp table by using ip address. Currently I am using this command
arp -a $ipAddress | awk '{print $4}'
This command prints what I want. But I am not comfortable with it and I wonder if there is any built-in way or more stable way to do this.
You can parse the /proc/net/arp file using awk:
awk "/^${ipAddress//./\.}\>/"' { print $4 }' /proc/net/arp
but I'm not sure it's simpler (it saves one fork and a subshell, though).
If you want a 100% bash solution:
while read ip _ _ mac _; do
[[ "$ip" == "$ipAddress" ]] && break
done < /proc/net/arp
echo "$mac"
Well, you could write a program (such as in C) to actually use the ARP protocol (yes, I know that's redundant, like ATM machine or PIN number) itself to get you the information but that's likely to be a lot harder than a simple pipeline.
Perhaps you should examine your comfort level a little more critically, since it's likely to cause you some unnecessary effort :-)
The manpage for the Linux ARP kernel module lists several methods for manipulating or reading the ARP tabes, ioctl probably being the easiest.
The output of arp -a is locale dependent (i.e. it changes with your system language). So it might be a good idea to at least force it to the default locale:
LC_ALL=C arp -a $ipAddress | awk '{print $4}'
However, I share your fear that the output of arp -a is not meant to be parsed. If your program is restricted to linux system, another option would be to parse the file /proc/net/arp. This file is exported by the kernel and is what arp itself parses to get its information. The format of this file is described in the manpage proc(5), see man 5 proc.
This can be easily done with awk:
awk '$1==IPADDRESS {print $4}' /proc/net/arp
Here's an awk + sed solution which doesn't assume the column number is always 4.
#!/bin/bash
cat /proc/net/arp |\
# remove space from column headers
sed 's/\([^ ]\)[ ]\([^ ]\)/\1_\2/g' |\
# find HW_address column number and/or print that column
awk '{
if ( !column ) {
for (i = 1; i <= NF; i++ ) {
if ( $i ~ /HW_address/ ) { column=i }
};
print $column
}
else {
print $column
}
}'
There are still fragile assumptions here, such as the column name being "HW address".
Update, removed PIPE
sed -nr 's/^'${ipAddress//./\.}'.*(([0-9A-Za-z]{2}:){5}[0-9A-Za-z]{2}).*$/\1/p' /proc/net/arp
Solution for non-fixed column;
arp -a $ipAddress | sed -n 's/^.*\(\([0-9A-Z]\{2\}:\)\{5\}[0-9A-Z]\{2\}\).*$/\1/p'
Explanation
^.* - Match start of string ^ followed by any character .*.
[0-9A-Z]\{2\}: - Match any character of numeric alpha-numeric twice followed by colon.
\([0-9A-Z]\{2\}:\)\{5\} - Match the pattern between the ( ) five times.
[0-9A-Z]\{2\} - Match any character of numeric alpha-numeric twice.
.*$ - Match any characters zero or more times .* until end of string $.
\1/p - Return capture pattern 1 / p print the match.
You can use this one for scripting:
awk ' $1~/[[:digit:]]/ {print $4}' /proc/net/arp
what it do:
read /proc/net/arp (standard arp output)
searchig for strings with [0-9]
get the 4rd "column" with mac adresses
Enjoy!
I prefer to use the arping command to explicitly query the MAC of some IP address (this also updates the local ARP cache):
arping -c 1 192.168.2.24 | grep -Eo "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]"
It's very useful to find if there exist two or more hosts using the same IP address (add -D option), or to check the current IP addresses used in the local VLAN with a simple script like:
for i in $(seq 1 254); do
IP="192.168.5.$i"
MAC=$(arping -c 1 $IP | grep -Eo "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]")
if [ "$MAC" ] ; then
echo "$IP $MAC"
fi
done
Note that arping can't detect the IP address of the local host in this way (but we can add checks in the script to show it if exists in the range).
There exist several versions of arping with slightly different options and output. In Linux Ubuntu there are one in the package iputils-arping and other in the package arping.
Note: To answer the question and not the problem, when filtering /proc/net/arp you must use a regex that ensures the full match, like ending the expression with a space (otherwise, in this example, it will show also 2.240-2.249 addresses if present):
ipaddress="192.168.2.24"
grep "^${ipaddress} " /proc/net/arp | grep -Eo "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]")
Okay so what I am trying to figure out is how do I count the number of periods in a string and then cut everything up to that point but minus 2. Meaning like this:
string="aaa.bbb.ccc.ddd.google.com"
number_of_periods="5"
number_of_periods=`expr $number_of_periods-2`
string=`echo $string | cut -d"." -f$number_of_periods`
echo $string
result: "aaa.bbb.ccc.ddd"
The way that I was thinking of doing it was sending the string to a text file and then just greping for the number of times like this:
grep -c "." infile
The reason I don't want to do that is because I want to avoid creating another text file for I do not have permission to do so. It would also be simpler for the code I am trying to build right now.
EDIT
I don't think I made it clear but I want to make finding the number of periods more dynamic because the address I will be looking at will change as the script moves forward.
If you don't need to count the dots, but just remove the penultimate dot and everything afterwards, you can use Bash's built-in string manuipulation.
${string%substring}
Deletes shortest match of $substring from back of $string.
Example:
$ string="aaa.bbb.ccc.ddd.google.com"
$ echo ${string%.*.*}
aaa.bbb.ccc.ddd
Nice and simple and no need for sed, awk or cut!
What about this:
echo "aaa.bbb.ccc.ddd.google.com"|awk 'BEGIN{FS=OFS="."}{NF=NF-2}1'
(further shortened by helpful comment from #steve)
gives:
aaa.bbb.ccc.ddd
The awk command:
awk 'BEGIN{FS=OFS="."}{NF=NF-2}1'
works by separating the input line into fields (FS) by ., then joining them as output (OFS) with ., but the number of fields (NF) has been reduced by 2. The final 1 in the command is responsible for the print.
This will reduce a given input line by eliminating the last two period separated items.
This approach is "shell-agnostic" :)
Perhaps this will help:
#!/bin/sh
input="aaa.bbb.ccc.ddd.google.com"
number_of_fields=$(echo $input | tr "." "\n" | wc -l)
interesting_fields=$(($number_of_fields-2))
echo $input | cut -d. -f-${interesting_fields}
grep -o "\." <<<"aaa.bbb.ccc.ddd.google.com" | wc -l
5