I want to get a mac address from arp table by using ip address. Currently I am using this command
arp -a $ipAddress | awk '{print $4}'
This command prints what I want. But I am not comfortable with it and I wonder if there is any built-in way or more stable way to do this.
You can parse the /proc/net/arp file using awk:
awk "/^${ipAddress//./\.}\>/"' { print $4 }' /proc/net/arp
but I'm not sure it's simpler (it saves one fork and a subshell, though).
If you want a 100% bash solution:
while read ip _ _ mac _; do
[[ "$ip" == "$ipAddress" ]] && break
done < /proc/net/arp
echo "$mac"
Well, you could write a program (such as in C) to actually use the ARP protocol (yes, I know that's redundant, like ATM machine or PIN number) itself to get you the information but that's likely to be a lot harder than a simple pipeline.
Perhaps you should examine your comfort level a little more critically, since it's likely to cause you some unnecessary effort :-)
The manpage for the Linux ARP kernel module lists several methods for manipulating or reading the ARP tabes, ioctl probably being the easiest.
The output of arp -a is locale dependent (i.e. it changes with your system language). So it might be a good idea to at least force it to the default locale:
LC_ALL=C arp -a $ipAddress | awk '{print $4}'
However, I share your fear that the output of arp -a is not meant to be parsed. If your program is restricted to linux system, another option would be to parse the file /proc/net/arp. This file is exported by the kernel and is what arp itself parses to get its information. The format of this file is described in the manpage proc(5), see man 5 proc.
This can be easily done with awk:
awk '$1==IPADDRESS {print $4}' /proc/net/arp
Here's an awk + sed solution which doesn't assume the column number is always 4.
#!/bin/bash
cat /proc/net/arp |\
# remove space from column headers
sed 's/\([^ ]\)[ ]\([^ ]\)/\1_\2/g' |\
# find HW_address column number and/or print that column
awk '{
if ( !column ) {
for (i = 1; i <= NF; i++ ) {
if ( $i ~ /HW_address/ ) { column=i }
};
print $column
}
else {
print $column
}
}'
There are still fragile assumptions here, such as the column name being "HW address".
Update, removed PIPE
sed -nr 's/^'${ipAddress//./\.}'.*(([0-9A-Za-z]{2}:){5}[0-9A-Za-z]{2}).*$/\1/p' /proc/net/arp
Solution for non-fixed column;
arp -a $ipAddress | sed -n 's/^.*\(\([0-9A-Z]\{2\}:\)\{5\}[0-9A-Z]\{2\}\).*$/\1/p'
Explanation
^.* - Match start of string ^ followed by any character .*.
[0-9A-Z]\{2\}: - Match any character of numeric alpha-numeric twice followed by colon.
\([0-9A-Z]\{2\}:\)\{5\} - Match the pattern between the ( ) five times.
[0-9A-Z]\{2\} - Match any character of numeric alpha-numeric twice.
.*$ - Match any characters zero or more times .* until end of string $.
\1/p - Return capture pattern 1 / p print the match.
You can use this one for scripting:
awk ' $1~/[[:digit:]]/ {print $4}' /proc/net/arp
what it do:
read /proc/net/arp (standard arp output)
searchig for strings with [0-9]
get the 4rd "column" with mac adresses
Enjoy!
I prefer to use the arping command to explicitly query the MAC of some IP address (this also updates the local ARP cache):
arping -c 1 192.168.2.24 | grep -Eo "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]"
It's very useful to find if there exist two or more hosts using the same IP address (add -D option), or to check the current IP addresses used in the local VLAN with a simple script like:
for i in $(seq 1 254); do
IP="192.168.5.$i"
MAC=$(arping -c 1 $IP | grep -Eo "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]")
if [ "$MAC" ] ; then
echo "$IP $MAC"
fi
done
Note that arping can't detect the IP address of the local host in this way (but we can add checks in the script to show it if exists in the range).
There exist several versions of arping with slightly different options and output. In Linux Ubuntu there are one in the package iputils-arping and other in the package arping.
Note: To answer the question and not the problem, when filtering /proc/net/arp you must use a regex that ensures the full match, like ending the expression with a space (otherwise, in this example, it will show also 2.240-2.249 addresses if present):
ipaddress="192.168.2.24"
grep "^${ipaddress} " /proc/net/arp | grep -Eo "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]")
Related
I am working on a shell script to parse the advertised mode and supported mode from the ethtool command and match them against an expected value. The idea is both expected and supported modes should have a common value to pass the condition.
Data:
# ethtool eth5
Settings for eth1:
Supported ports: [ Backplane ]
Supported link modes: 1000baseKX/Full
10000baseKR/Full
15000baseKR/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Supported FEC modes: None BaseR RS
Advertised link modes: 1000baseKX/Full
10000baseKR/Full
15000baseKR/Full
I have currently used the below for the same.
link_mode_expected=15000baseKR
# get the mode using the shell command and parse the value needed
mode_supported=`ethtool ${eth_device} | sed -ne '/Supported link modes:/,/:/p' |\
sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' | grep "$link_mode_expected"`
mode_advertised=`ethtool ${eth_device} | sed -ne '/Advertised link modes:/,/:/p' |\
sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' | grep "$link_mode_expected"`
if [ "$mode_supported" != "$link_mode_expected" ] ||\
( [ "$mode_supported" != "$mode_advertised" ] ) ; then
#some action
fi
The above code is giving the correct result that is 15000baseKR/Full for both mode_advertised and mode_supported and I am doing the logic as above but I am looking for help with below
how to avoid two ethtool commands, can we add the output to a buffer and grep from that buffer?
what is the awk equivalent of the same and if awk is faster in execution?
Is there any better approach than the above?
Awk will indeed be the answer to all your questions here.
link_mode_expected=15000
if ethtool "$eth_device" |
awk -v lme="$link_mode_expected" '
s && /:/ { s=0 }
/Supported link modes:/ { s=1 }
s && ($NF + 0 == lme) { sgood++ }
a && /:/ { a=0 }
/Advertised link modes:/ { a=1 }
a && ($NF + 0 == lme) { agood++ }
END { exit 1-(agood && sgood) }'
then
# some action
fi
Demo, with some debug prints: https://ideone.com/v5du8g
Having Awk perform the comparison and set its exit status for if to examine is slightly clunky on the Awk side, but makes the script very easy and natural to use from the surrounding shell script.
Like sed, Awk processes one line (or, more specifically, one record) per iteration. In this script, the simple variables s and a reflect whether the current input line is inside a region which enumerates the supported or advertised modes, respectively. If we are in such a region, we check if the last field on the line ($NF) evaluates numerically (+ 0) to the number in lme. Awk conveniently ignores any nonnumeric tail on the value in this scenario. At the end, we set the exit status to 0 for success (both the values were found) or 1 otherwise, in accordance with the shell's conventions (zero exit code means success, anything else is a failure).
As an alternative approach, you could try to figure out a record separator (RS) to split on, instead of newline. Splitting on a colon might be good, as then you can examine the entire region in one go (but then you lose the simple and elegant feature that NF contains the index of the field you want to examine, so it might not be simpler at all in the end).
You could also opt to simply have Awk extract and print the values, and then do the comparison in the shell; but having the shell parse the output from Awk which you just spent so much time parsing is stilted and unattractive.
This script isn't entirely trivial, but you should find that learning enough Awk to write simpler scripts is quite pleasant and quick, and very well worth your time. You can get pretty far already on a budget of 30 minutes to an hour.
Ad-hoc output formats which require custom parsers are a constant source of frustration and friction. Increasingly, modern tools have options to produce output in properly machine-readable standard formats like JSON, YAML, or XML. Alas, ethtool unfortunately does not seem to have any such facility.
I'm pretty new to Linux and writing scripts etc. I have this task where I need to find an IP-address from a database and then grep a bunch of files with this IP and the next one to see, if they have any presence there. Currently I have to first write:
rwhois -Br 0.0.0.0
and then
grep -wl '0.0.0.0\|0.0.0.1' /path/to/some/files
And I have to manually change the last digit from the rwhois and from the grep.
I got as far as to write a simple function like this
function info () {
rhowis -Br $1
grep -w '$1\|$1'
}
But of course I'd have to somehow increase the value of the latter input by 1. Any good advice? And a small explanation of what you changed is appreciated so I can learn from this. Thanks!
It's simple to just increase the last digit with awk:
info() {
local ip="$1"
local nextip=$(awk -F. '{ print $1 "." $2 "." $3 "." ($4+1) }' <<<"$1")
rhowis -Br "$ip"
grep -w "$ip\|$nextip'
}
Note that this will not handle wrapping (when the last digit is 255), but that shouldn't be a problem if you don't need to handle broadcast addresses.
ip=$(awk -F\. '{ print $1"."$2"."$3"."($4+1) }' <<< $1)
With awk you can set up a variable ip. We set the delimited as "." and then take in the initial IP address then printing out the first,second and third delimited pieces along with the fourth incremented by one.
We would then action:
grep -w '$1\|$ip'
i have a file with a lot of IPs and each IP have an ID, like this:
"id":340,"ip":"10.38.6.25"
"id":341,"ip":"10.38.6.26"
"id":345,"ip":"10.38.6.27"
"id":346,"ip":"110.38.6.27"
Below this Ips and after these Ips the file have more information, its a output to an API call..
I need, grep a IP and then the command shows the id, just the number. Like this:
345
EDIT: More information, the ip will be different every time, i need to pass the IP by argument. I cant parse the IP to the syntax X/X/X/X...
any ideas?
Since your current requirement is get the IDs from your broke json file, re-formatting my earlier answer.
Though I do NOT recommend this solution to get the ID, a hacky way to do this would be to use grep in PCRE mode. The way I have done the logic is to get the IP string and get the characters before it. I am not sure how to extract the digit from id alone which returns me
317,"ip":"10.38.6.2"
So using process-substitution to get the value before the first , as below.
IFS="," read -r id _< <(grep -Po ".{0,4}\"ip\":\"10.38.6.2\"" file); printf "%s\n" "$id"
317
IFS="," read -r id _< <(grep -Po ".{0,4}\"ip\":\"10.38.6.3\"" file); printf "%s\n" "$id"
318
Just add the IP you need as part of the grep string.
The below logic applies only to the your initial inputs.
Using multi-character de-limiters ; and , in awk, we can do something like:-
awk -F'[:,]' '/10\.38\.6\.27/{print $2}' file
345
A better way would be to use the match syntax equivalent to the awk // regex feature to use the variables of your choice. Provide the input IP you want in the following format.
input='"10\\.38\\.6\\.25"'
awk -F'[:,]' -v var="$input" '{ if ( match( $0, var )) {print $2};}' file
340
A more robust way to avoid matching incorrect lines would be to use " also as delimiter and do a direct match with the IP as suggested by hek2mgl.
awk -F'[:,"]' -v var="$input" '$9==var{print $4}' file
340
If you want to look up a single IP, use this:
jq ".collection|.[]|select(.ip==\"10.38.6.3\").id" data.json
If you must set IP in an argument, then write a one-liner bash script like this:
jq ".collection|.[]|select(.ip==\"$2\").id" "$1"
And call it like this:
./script data.json 10.38.6.3
grep
grep -Po ':\K\d+(?=,"ip":"xx\.xx\.xx\.xx")' file
awk -F, '/10\.38\.6\.25/ {gsub("\"","");split($1,a,":") ;print a[2]}' ip
340
or
awk -F, -v ipin="10.38.6.25" '$0 ~ ipin {gsub("\"","");split($1,a,":") ;print a[2]}' ip
$ awk -F, -v grep="10.38.6.26" '$2 ~ "\"" grep "\"" && sub(/^.*:/,"",$1) {print $1}' foo
341
Grep, SED, and AWK are inappropriate tools for JSON parsing. You whether need a tool specially designed for working with JSON data (e.g. jq), or write a script in a language that supports JSON parsing in one way, or another (examples: PHP, Perl, JavaScript).
JQ
One of the easiest ways is to use the jq tool (as mentioned in the comments to the question), e.g.:
jq '.collection[] | if .ip == "10.38.6.3" then .id else empty end' < file.json
PHP
Alternatively, you can write a simple tool in PHP, for example. PHP has a built-in JSON support.
ip-ids.php
<?php
$ip = trim($argv[1]);
$json = file_get_contents('file.json');
$json = json_decode($json, true);
foreach ($json['collection'] as $e) {
if ($e['ip'] == $ip)
echo $e['id'], PHP_EOL;
}
(sanity checks are skipped for the sake of simplicity)
Usage
php ip-ids.php '10.38.6.3'
Node.js
If you have Node installed, the following script can be used as a universal solution. You can pass any IP as the first argument, and the script will output a list of corresponding IDs.
ip-ids.js
#!/usr/bin/node
var fs = require('fs');
var ip = process.argv[2];
var json = fs.readFileSync('file.json', 'utf-8');
json = JSON.parse(json);
for (var i = 0; i < json.collection.length; i++) {
if (json.collection[i]['ip'] === ip)
console.log(json.collection[i]['id']);
}
Usage
node ip-ids.js '10.38.6.3'
or, if the executable permissions are set (chmod +x ip-ids.js):
./ip-ids.js '10.38.6.3'
Note, I have skipped sanity checks in the script for the sake of simplicity.
Conclusion
Now you can see that it is pretty easy to use jq. Scripting solutions are slightly more verbose, but not too difficult as well. Both approaches are flexible. You don't have to rely on positions of sub-strings in the JSON string, or to resort to hacks that you will most likely forget after a couple of weeks. The script solutions are reliable and readable (and thus easily maintainable), as opposed to tricky AWK/GREP/SED expressions.
Original answer
This is the original answer for the case of a file in the following format (I didn't know that the input is in JSON format). Still, this solution seems to work even with the partial JSON you currently pasted into the question.
"id":340,"ip":"10.38.6.25"
"id":341,"ip":"10.38.6.26"
"id":345,"ip":"10.38.6.27"
Perl version:
perl -ne '/"id":(\d+).*"ip":"10\.38\.6\.27"/ and print "$1\n"' file
You example is not valid JSON. In order to get valid JSON you have to add curly braces. This is done by the sed in the following example.
$ sed 's/^/{/;s/$/}/' <<EOF | jq -s 'map(select(.ip == "10.38.6.27")) | map(.id) | .[]'
> "id":340,"ip":"10.38.6.25"
> "id":341,"ip":"10.38.6.26"
> "id":345,"ip":"10.38.6.27"
> "id":346,"ip":"110.38.6.27"
> EOF
345
Normally jq reads just one object. With the option -s jq reads all objects, because you have a list input. The first map iterates over the list and selects only those objects with the matching attribute ip. This is the same as a grep. The second map takes just the id attribute from the result and the final .[] the the opposite to the -s option.
If you can make your json pretty and then do cat file, below command might help
cat /tmp/file|grep -B 1 "ipaddress"|grep -w id|tr ' ' '\0'|cut -d: -f2|cut -d, -f1
I'm new to shell scripting and here is my problem:
I want to store PID's from output of airmon-ng check to some variables (for ex: $1, $2, $3) so that I can execute kill $1 $2 $3.
here is sample output of airmon-ng check:
Found 3 processes that could cause trouble.
If airodump-ng, aireplay-ng or airtun-ng stops working after
a short period of time, you may want to kill (some of) them!
PID Name
707 NetworkManager
786 wpa_supplicant
820 dhclient
I want to grab numbers 707, 786, 820.
I tried using set 'airmon-ng check' and then using for loop:
set `airmon-ng check`
n=$#
for (( i=0; i<=n; i++ ))
do
echo $i
done
it outputs 1,2,3,...36
not words or numbers so I couldn't figure out how I should do it.
airmon-ng check | egrep -o '\b[0-9]+\b' | xargs kill
egrep is grep with extended regular expressions (like grep -E), -o says to extract only the matching parts, \b matches word boundaries so you don't get any numbers accidentally occuring in process names or something, [0-9]+ matches one or more decimal digit, xargs kill passes all the matches as arguments to the kill command.
Note that parsing output intended to be read by humans might not always be a good idea. Also, just killing all those processes doesn't sound too smart either, but proper usage of airocrack is beyond this question.
You can get list of the PIDs separated by spaces e.g. like this (everything from the 1st column after "PID"):
l=`airmon-ng check | awk 'BEGIN { p=0 } { if (p) { print $1" "; } if ($1=="PID") { p=1 } }' | tr '\n' ' '`
Why not use grep?
myvar=$(airmon-ng check | grep '[0-9]\{3,6\}')
This assumes a PID of 3 to 6 digits, and will grab anything from the airmon-ng output of a similar length. So this may not work as well if the output includes other strings with digits of a similar length.
I would use awk for this and store the output in an array
pids=( $(airmon-ng check | awk '/^[[:blank:]]+[[:digit:]]+[[:blank:]]+/{print $1}') )
#'pids' is an array
kill "${pids[#]}" #killing all the processes thus found.
I have a list of urls like this:
http://noto.zrobimystrone.pl/pucenter/images/NGdocs/
http://visionwebmkt.com/unsubscribe.php?M=879552&C=b744d324e38f5f3b0bcf549f1d57a3ab&L=20&N=497
http://www.meguiatramandai.com.br/unsubscribe.php?M=722&C=8410431be55bf12faac13d18982d71cd&L=1&N=3
http://www.contatoruy.in/link.php?M=86457&N=4&L=1&F=H
http://www.maxxivrimoveis.com.br/
http://www.meguiatramandai.com.br/unsubscribe.php?M=722&C=8410431be55bf12faac13d18982d71cd&L=1&N=2
http://arm.smilecire.com/ch+urch38146263923bpa.stor/imp-roved258021029his+health212149011
http://hurl.zonalrems.com/ge.tyo-ur584372780599hea+lth247408058un/der+control21211901
http://harp.doomyjupe.com/see.this-better/life+58291551346csexdrive663295668+better/how.981692016
http://beefy.toneyvaws.com/no+tice/how/35306640b+see/app=5429204last/attempt=457943182
http://kirk.yournjuju.com/shop/sam.sclub-win=ter/58387369768esame+673844946.bett.er-loo.k981686408
http://idly.theirpoem.com/veri-fy/notice-7853508818b2glob/al=who.43639603inc.lusion-610549278
http://wva188.suleacatan.com/credit-score/review/-551694841511001sfdghsfdgsdfg63887839
http://cop.forterins.com/app.lyto=face962540097dtolo+oko.ung268570307yo.un-ger8752507
http://vni116.gaelsyaray.com/qertqetert//-dghjghjghd5531864856415612229498430
http://ticket.prategama.com/shop/sam.sclub-win=ter/752490935same+226373195.bett.er-loo.k212801
http://cbu125.quetxviii.com/cvbnvbn7551116db537203--swrtytry664896546
http://c5a.dicadodia.com.br/pass4sp09/NetAffProTeste-1.html
http://snub.woadsbevy.com/ama/zing-753773417oppe-tun/ity+217801.is-here/now=236922473
http://mkt.livrariacultura.com.br/pub/cc?_ri_=X0Gzc2X%3DWQpglLjHJlYQGgzfB7tPi0PuyyJ71ES
I wanna extract only the parents domain names, for example:
http://noto.zrobimystrone.pl/pucenter/images/NGdocs/
http://visionwebmkt.com/unsubscribe.php?M=879552&C=b744d324e38f5f3b0bcf549f1d57a3ab&L=20&N=497
http://www.meguiatramandai.com.br/unsubscribe.php?M=722&C=8410431be55bf12faac13d18
Into
zrobimystrone.pl
visionwebmkt.com
meguiatramandai.com.br
I have tried
awk '{gsub("http://|/.*","")}1' list.txt
and got the following results:
noto.zrobimystrone.pl
visionwebmkt.com
www.meguiatramandai.com.br
www.contatoruy.in
www.maxxivrimoveis.com.br
www.meguiatramandai.com.br
arm.smilecire.com
hurl.zonalrems.com
harp.doomyjupe.com
beefy.toneyvaws.com
but dont know how to get only the parent name from noto.zrobimystrone.pl for instance.
Using awk
awk -F \/ '{l=split($3,a,"."); print (a[l-1]=="com"?a[l-2] OFS:X) a[l-1] OFS a[l]}' OFS="." file|sort -u
contatoruy.in
dicadodia.com.br
doomyjupe.com
forterins.com
gaelsyaray.com
livrariacultura.com.br
maxxivrimoveis.com.br
meguiatramandai.com.br
prategama.com
quetxviii.com
smilecire.com
suleacatan.com
theirpoem.com
toneyvaws.com
visionwebmkt.com
woadsbevy.com
yournjuju.com
zonalrems.com
zrobimystrone.pl
You can use this awk:
awk -F'.' '{gsub("http://|/.*","")} NF>2{$1="";$0=substr($0, 2)}1' OFS='.' list.txt
zrobimystrone.pl
visionwebmkt.com
meguiatramandai.com.br
contatoruy.in
maxxivrimoveis.com.br
meguiatramandai.com.br
smilecire.com
zonalrems.com
doomyjupe.com
toneyvaws.com
yournjuju.com
theirpoem.com
suleacatan.com
forterins.com
gaelsyaray.com
prategama.com
quetxviii.com
dicadodia.com.br
woadsbevy.com
livrariacultura.com.br
A "simple" bash solution. Tested in bash shell on Solaris 11.2 x86.
#!/bin/bash
while IFS=/ read HTTP NULL FQDN PAGE
do
PARENT=${FQDN#*.}
if [[ $PARENT != *"."* ]]
then echo $FQDN
else echo $PARENT
fi
done < fileOfURLs.txt
Without the string contains pattern test, too much of the domain could be stripped away. The if paragraph can be reduced,so the whole script now looks like this:
#!/bin/bash
while IFS=/ read HTTP NULL FQDN PAGE
do
PARENT=${FQDN#*.}
[[ $PARENT != *"."* ]] && echo $FQDN || echo $PARENT
done < fileOfURLs.txt
The bash variable substitution is taking the contents of the variable FQDN and stripping from the left any character up to and including the first dot.
The test condition is asking if the contents of the PARENT variable does not contain a dot. If it does not hold a dot somewhere in the value, the test evaluates to true and will display the original FQDN contents. If the test evaluates to false, (there is still a dot in the value) the contents of PARENT are displayed.
I guess it depends on what you mean by parent. If by "parent", you mean the top of the zone apex in DNS (e.g., zrobimystrone.pl ), then the right way to do this is to look that up in DNS. There's a trick with DNS where you get back the parent zone SOA record if you ask for the SOA for any name.. So, try this:
for i in $(awk '{gsub("http://|/.*","")}1' list.txt); do dig soa $i | grep -v ^\; | grep SOA | awk '{print $1}'; done
This will give you a much more accurate list, but it runs way slower and is sub-optimal. The other answers don't take into account all the possible variations of TLD names used within TLDs, e.g., www.somecompany.org.uk, so it all depends on how accurate you need this to be.
An easy solution to get parent domain name
echo http://www.humkinar.pk | awk -F '/' '{print $3}'
www.humkinar.pk