I am writing a simple script to collect 2 IP addresses. I am using the open stack client to gather the allocation pool of a provider network. I used awk to gather the 2 IP addresses (start and end) and put them into 2 variables. When I echo the 2 variables alone they print out how I expect. However, if I try to echo something after the variable, it seems to replace the first few characters of the IP address.
It hard to explain, but if you refer to the output it should make more sense. If you look at my script below, I just put the string "hello" after the variable in each echo statement for demonstrative purposes.
#!/bin/bash
NETWORK=$1
#just gets the allocation pool IP addresses from openstack
ALLOCATION_POOLS=$(openstack subnet show $NETWORK --insecure|grep -w "allocation_pools"|awk -F " " '{print $4}')
POOL_START=$(awk -F "-" '{print $1}' <<< "$ALLOCATION_POOLS")
echo $POOL_START"hello"
POOL_END=$(awk -F "-" '{print $2}' <<< "$ALLOCATION_POOLS")
echo $POOL_END"hello"
Here is the output:
hello.146.87
hello.146.126
If I did not put "hello" in the echo statement, the output looks more like this:
10.28.146.87
10.28.146.126
Another thing I did was tested the length of the strings, and the length was larger then the number of characters in the ip address. I believe that there is some strange character after the IP addresses that is causing this. If that is the case, how can I remove it?
Related
I'm pretty new to Linux and writing scripts etc. I have this task where I need to find an IP-address from a database and then grep a bunch of files with this IP and the next one to see, if they have any presence there. Currently I have to first write:
rwhois -Br 0.0.0.0
and then
grep -wl '0.0.0.0\|0.0.0.1' /path/to/some/files
And I have to manually change the last digit from the rwhois and from the grep.
I got as far as to write a simple function like this
function info () {
rhowis -Br $1
grep -w '$1\|$1'
}
But of course I'd have to somehow increase the value of the latter input by 1. Any good advice? And a small explanation of what you changed is appreciated so I can learn from this. Thanks!
It's simple to just increase the last digit with awk:
info() {
local ip="$1"
local nextip=$(awk -F. '{ print $1 "." $2 "." $3 "." ($4+1) }' <<<"$1")
rhowis -Br "$ip"
grep -w "$ip\|$nextip'
}
Note that this will not handle wrapping (when the last digit is 255), but that shouldn't be a problem if you don't need to handle broadcast addresses.
ip=$(awk -F\. '{ print $1"."$2"."$3"."($4+1) }' <<< $1)
With awk you can set up a variable ip. We set the delimited as "." and then take in the initial IP address then printing out the first,second and third delimited pieces along with the fourth incremented by one.
We would then action:
grep -w '$1\|$ip'
Is there any linux command or script that I can invoke to get ip range with subnet, if I pass number of ip's as an argument, like if I say 256 then it should return me 10.0.0.0/24, Lets say I am talking about 10.0.0.0 range as of now.
If you have Perl installed, you can calculate that value with the following script:
perl -le 'print 32-int(log(<>)/log(2));'
This takes input from standard in, so you can pipe your desired number of IPs as follows:
echo 256 | perl -le 'print 32-int(log(<>)/log(2));'
# prints 24 followed by a newline to stdout
If you want it to display an IP before, you can run this Perl program:
echo 256 | perl -le 'print "10.0.0.0/" . (32-int(log(<>)/log(2)));'
# prints 10.0.0.0/24 followed by a newline to stdout
The subnet mask can be calculated by subtracting the log base 2 of the desired number of IPs from 32 (IPv4 netmask length), and that's what the above Perl script does.
I've created a script that seperates the IP that it finds, based on NIC card name input.
#!/bin/bash
echo what is your NIC?
read NIC
IP=`ifconfig $NIC 2>/dev/null|awk '/inet addr:/ {print $2}'|sed 's/addr://'`
NEWSTRING=${IP:0:6}
ALPHARETTA="12.101"
EUFAULA="12.102"
if [ "${NEWSTRING}" = "${ALPHARETTA}" ] ; then
echo I'm in Alpharetta
else
echo I'm in Eufaula
fi
If eth0 were to be 12.101.1.1 it would only take (12.101)
I'm comparing 12.101 and 12.101 for my tests... and i'm getting this echo back....
what is your NIC?
eth0
Im in Alpharetta
else
echo Im in Eufaula
I'm obviously doing something silly, and not seeing it.. could somebody point me in the correct direction?
The bash parser is seeing the apostrophe characters you are trying to echo in the word "I'm" and it thinks you are trying to print one long string that spans from line 10 to line 12 of your script. You can even see how the syntax highlighting on this site is also indicating a problem. You should wrap the message you are echoing in quotes. For example:
echo "I'm in Alpharetta"
I have a list of urls like this:
http://noto.zrobimystrone.pl/pucenter/images/NGdocs/
http://visionwebmkt.com/unsubscribe.php?M=879552&C=b744d324e38f5f3b0bcf549f1d57a3ab&L=20&N=497
http://www.meguiatramandai.com.br/unsubscribe.php?M=722&C=8410431be55bf12faac13d18982d71cd&L=1&N=3
http://www.contatoruy.in/link.php?M=86457&N=4&L=1&F=H
http://www.maxxivrimoveis.com.br/
http://www.meguiatramandai.com.br/unsubscribe.php?M=722&C=8410431be55bf12faac13d18982d71cd&L=1&N=2
http://arm.smilecire.com/ch+urch38146263923bpa.stor/imp-roved258021029his+health212149011
http://hurl.zonalrems.com/ge.tyo-ur584372780599hea+lth247408058un/der+control21211901
http://harp.doomyjupe.com/see.this-better/life+58291551346csexdrive663295668+better/how.981692016
http://beefy.toneyvaws.com/no+tice/how/35306640b+see/app=5429204last/attempt=457943182
http://kirk.yournjuju.com/shop/sam.sclub-win=ter/58387369768esame+673844946.bett.er-loo.k981686408
http://idly.theirpoem.com/veri-fy/notice-7853508818b2glob/al=who.43639603inc.lusion-610549278
http://wva188.suleacatan.com/credit-score/review/-551694841511001sfdghsfdgsdfg63887839
http://cop.forterins.com/app.lyto=face962540097dtolo+oko.ung268570307yo.un-ger8752507
http://vni116.gaelsyaray.com/qertqetert//-dghjghjghd5531864856415612229498430
http://ticket.prategama.com/shop/sam.sclub-win=ter/752490935same+226373195.bett.er-loo.k212801
http://cbu125.quetxviii.com/cvbnvbn7551116db537203--swrtytry664896546
http://c5a.dicadodia.com.br/pass4sp09/NetAffProTeste-1.html
http://snub.woadsbevy.com/ama/zing-753773417oppe-tun/ity+217801.is-here/now=236922473
http://mkt.livrariacultura.com.br/pub/cc?_ri_=X0Gzc2X%3DWQpglLjHJlYQGgzfB7tPi0PuyyJ71ES
I wanna extract only the parents domain names, for example:
http://noto.zrobimystrone.pl/pucenter/images/NGdocs/
http://visionwebmkt.com/unsubscribe.php?M=879552&C=b744d324e38f5f3b0bcf549f1d57a3ab&L=20&N=497
http://www.meguiatramandai.com.br/unsubscribe.php?M=722&C=8410431be55bf12faac13d18
Into
zrobimystrone.pl
visionwebmkt.com
meguiatramandai.com.br
I have tried
awk '{gsub("http://|/.*","")}1' list.txt
and got the following results:
noto.zrobimystrone.pl
visionwebmkt.com
www.meguiatramandai.com.br
www.contatoruy.in
www.maxxivrimoveis.com.br
www.meguiatramandai.com.br
arm.smilecire.com
hurl.zonalrems.com
harp.doomyjupe.com
beefy.toneyvaws.com
but dont know how to get only the parent name from noto.zrobimystrone.pl for instance.
Using awk
awk -F \/ '{l=split($3,a,"."); print (a[l-1]=="com"?a[l-2] OFS:X) a[l-1] OFS a[l]}' OFS="." file|sort -u
contatoruy.in
dicadodia.com.br
doomyjupe.com
forterins.com
gaelsyaray.com
livrariacultura.com.br
maxxivrimoveis.com.br
meguiatramandai.com.br
prategama.com
quetxviii.com
smilecire.com
suleacatan.com
theirpoem.com
toneyvaws.com
visionwebmkt.com
woadsbevy.com
yournjuju.com
zonalrems.com
zrobimystrone.pl
You can use this awk:
awk -F'.' '{gsub("http://|/.*","")} NF>2{$1="";$0=substr($0, 2)}1' OFS='.' list.txt
zrobimystrone.pl
visionwebmkt.com
meguiatramandai.com.br
contatoruy.in
maxxivrimoveis.com.br
meguiatramandai.com.br
smilecire.com
zonalrems.com
doomyjupe.com
toneyvaws.com
yournjuju.com
theirpoem.com
suleacatan.com
forterins.com
gaelsyaray.com
prategama.com
quetxviii.com
dicadodia.com.br
woadsbevy.com
livrariacultura.com.br
A "simple" bash solution. Tested in bash shell on Solaris 11.2 x86.
#!/bin/bash
while IFS=/ read HTTP NULL FQDN PAGE
do
PARENT=${FQDN#*.}
if [[ $PARENT != *"."* ]]
then echo $FQDN
else echo $PARENT
fi
done < fileOfURLs.txt
Without the string contains pattern test, too much of the domain could be stripped away. The if paragraph can be reduced,so the whole script now looks like this:
#!/bin/bash
while IFS=/ read HTTP NULL FQDN PAGE
do
PARENT=${FQDN#*.}
[[ $PARENT != *"."* ]] && echo $FQDN || echo $PARENT
done < fileOfURLs.txt
The bash variable substitution is taking the contents of the variable FQDN and stripping from the left any character up to and including the first dot.
The test condition is asking if the contents of the PARENT variable does not contain a dot. If it does not hold a dot somewhere in the value, the test evaluates to true and will display the original FQDN contents. If the test evaluates to false, (there is still a dot in the value) the contents of PARENT are displayed.
I guess it depends on what you mean by parent. If by "parent", you mean the top of the zone apex in DNS (e.g., zrobimystrone.pl ), then the right way to do this is to look that up in DNS. There's a trick with DNS where you get back the parent zone SOA record if you ask for the SOA for any name.. So, try this:
for i in $(awk '{gsub("http://|/.*","")}1' list.txt); do dig soa $i | grep -v ^\; | grep SOA | awk '{print $1}'; done
This will give you a much more accurate list, but it runs way slower and is sub-optimal. The other answers don't take into account all the possible variations of TLD names used within TLDs, e.g., www.somecompany.org.uk, so it all depends on how accurate you need this to be.
An easy solution to get parent domain name
echo http://www.humkinar.pk | awk -F '/' '{print $3}'
www.humkinar.pk
I want to get a mac address from arp table by using ip address. Currently I am using this command
arp -a $ipAddress | awk '{print $4}'
This command prints what I want. But I am not comfortable with it and I wonder if there is any built-in way or more stable way to do this.
You can parse the /proc/net/arp file using awk:
awk "/^${ipAddress//./\.}\>/"' { print $4 }' /proc/net/arp
but I'm not sure it's simpler (it saves one fork and a subshell, though).
If you want a 100% bash solution:
while read ip _ _ mac _; do
[[ "$ip" == "$ipAddress" ]] && break
done < /proc/net/arp
echo "$mac"
Well, you could write a program (such as in C) to actually use the ARP protocol (yes, I know that's redundant, like ATM machine or PIN number) itself to get you the information but that's likely to be a lot harder than a simple pipeline.
Perhaps you should examine your comfort level a little more critically, since it's likely to cause you some unnecessary effort :-)
The manpage for the Linux ARP kernel module lists several methods for manipulating or reading the ARP tabes, ioctl probably being the easiest.
The output of arp -a is locale dependent (i.e. it changes with your system language). So it might be a good idea to at least force it to the default locale:
LC_ALL=C arp -a $ipAddress | awk '{print $4}'
However, I share your fear that the output of arp -a is not meant to be parsed. If your program is restricted to linux system, another option would be to parse the file /proc/net/arp. This file is exported by the kernel and is what arp itself parses to get its information. The format of this file is described in the manpage proc(5), see man 5 proc.
This can be easily done with awk:
awk '$1==IPADDRESS {print $4}' /proc/net/arp
Here's an awk + sed solution which doesn't assume the column number is always 4.
#!/bin/bash
cat /proc/net/arp |\
# remove space from column headers
sed 's/\([^ ]\)[ ]\([^ ]\)/\1_\2/g' |\
# find HW_address column number and/or print that column
awk '{
if ( !column ) {
for (i = 1; i <= NF; i++ ) {
if ( $i ~ /HW_address/ ) { column=i }
};
print $column
}
else {
print $column
}
}'
There are still fragile assumptions here, such as the column name being "HW address".
Update, removed PIPE
sed -nr 's/^'${ipAddress//./\.}'.*(([0-9A-Za-z]{2}:){5}[0-9A-Za-z]{2}).*$/\1/p' /proc/net/arp
Solution for non-fixed column;
arp -a $ipAddress | sed -n 's/^.*\(\([0-9A-Z]\{2\}:\)\{5\}[0-9A-Z]\{2\}\).*$/\1/p'
Explanation
^.* - Match start of string ^ followed by any character .*.
[0-9A-Z]\{2\}: - Match any character of numeric alpha-numeric twice followed by colon.
\([0-9A-Z]\{2\}:\)\{5\} - Match the pattern between the ( ) five times.
[0-9A-Z]\{2\} - Match any character of numeric alpha-numeric twice.
.*$ - Match any characters zero or more times .* until end of string $.
\1/p - Return capture pattern 1 / p print the match.
You can use this one for scripting:
awk ' $1~/[[:digit:]]/ {print $4}' /proc/net/arp
what it do:
read /proc/net/arp (standard arp output)
searchig for strings with [0-9]
get the 4rd "column" with mac adresses
Enjoy!
I prefer to use the arping command to explicitly query the MAC of some IP address (this also updates the local ARP cache):
arping -c 1 192.168.2.24 | grep -Eo "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]"
It's very useful to find if there exist two or more hosts using the same IP address (add -D option), or to check the current IP addresses used in the local VLAN with a simple script like:
for i in $(seq 1 254); do
IP="192.168.5.$i"
MAC=$(arping -c 1 $IP | grep -Eo "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]")
if [ "$MAC" ] ; then
echo "$IP $MAC"
fi
done
Note that arping can't detect the IP address of the local host in this way (but we can add checks in the script to show it if exists in the range).
There exist several versions of arping with slightly different options and output. In Linux Ubuntu there are one in the package iputils-arping and other in the package arping.
Note: To answer the question and not the problem, when filtering /proc/net/arp you must use a regex that ensures the full match, like ending the expression with a space (otherwise, in this example, it will show also 2.240-2.249 addresses if present):
ipaddress="192.168.2.24"
grep "^${ipaddress} " /proc/net/arp | grep -Eo "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]")