please advice how to match the following hostname
machine hostname Should be according to the following rule
<a-z word ><number><a-z character/s>
real example
star1a
linux25as
machine2b
linux5a
solaris300C
unix9c
please advice how to machine these hostname with grep
I have for now this syntax
hostname | grep -c '[a-z][1-2][a-z]'
but these syntax not work on all my examples
on solaris the option egrep -E not works
hostname | grep -E '\b[a-z]+[0-9][a-z]+'
grep: illegal option -- E
Usage: grep -hblcnsviw pattern file . . .
Broken Pipe
try the second option ( on solaris machine ):
hostname
swu2a
hostname | grep "^[a-z]\+[0-9][a-z]\+$"
not matched!!!
I also try this:
hostname
swu2a
hostname | grep '[a-z]\+[0-9]\+[a-zA-Z]\+'
NOT MATCHED!!!
Here is an awk using same regex as the grep posted here uses.
awk '/[a-z]+[0-9]+[a-zA-Z]+/'
star1a
linux25as
machine2b
linux5a
solaris300C
unix9c
If you need to make sure there is nothing else in the line, only the words above, use:
awk '/^[a-z]+[0-9]+[a-zA-Z]+$/'
^ marks start of line.
$ marks end of line.
You can use the following pattern:
grep '^[a-z]\+[0-9]\+[a-zA-Z]\+$'
Note that you can use the the return value of grep to decide whether the pattern matches or not, you don't need to use the -c option. Like this:
if [ hostname | grep '^[a-z]\+[0-9]\+[a-zA-Z]\+$' >/dev/null 2>&1 ] ; then
echo "host name OK"
fi
Related
I'm developping a script to search for patterns within scripts executed from CRON on a bunch of remote servers through SSH.
Script on client machine -- SSH --> Remote Servers CRON/Scripts
For now I can't get the correct output.
Script on client machine
#!/bin/bash
server_list=( '172.x.x.x' '172.x.x.y' '172.x.x.z' )
for s in ${server_list[#]}; do
ssh -i /home/user/.ssh/my_key.rsa user#${s} crontab -l | grep -v '^#\|^[[:space:]]*$' | cut -d ' ' -f 6- | awk '{print $1}' | grep -v '^$\|^echo\|^find\|^PATH\|^/usr/bin\|^/bin/' | xargs -0 grep -in 'server.tld\|10.x.x.x'
done
This only gives me the paths of scripts from crontab, not the matched lines and line number plus the first line is prefixed with "grep:" keyword (example below):
grep: /opt/directory/script1.sh
/opt/directory/script2.sh
/opt/directory/script3.sh
/opt/directory/script4.sh
How to get proper output, meaning the script path plus line number plus line of matching pattern?
Remote CRON examples
OO 6 * * * /opt/directory/script1.sh foo
30 6 * * * /opt/directory/script2.sh bar
Remote script content examples
1 ) This will match grep pattern
#!/bin/bash
ping -c 4 server.tld && echo "server.tld ($1)"
2 ) This won't match grep pattern
#!/bin/bash
ping -c 4 8.x.x.x && echo "8.x.x.x ($1)"
Without example input, it's really hard to see what your script is attempting to do. But the cron parsing could almost certainly be simplified tremendously by refactoring all of it into a single Awk script. Here is a quick stab, with obviously no way to test.
#!/bin/sh
# No longer using an array for no good reason, so /bin/sh will work
for s in 172.x.x.x 172.x.x.y 172.x.x.z; do
ssh -i /home/user/.ssh/my_key.rsa "user#${s}" crontab -l |
awk '! /^#|^[[:space:]]*$/ && $6 !~ /^$|^(echo|find|PATH|\/usr\/bin|\/bin\/)/ { print $6 }' |
# no -0; use grep -E and properly quote literal dot
xargs grep -Ein 'server\.tld|10.x.x.x'
done
Your command would not output null-delimited data to xargs so probably the immediate problem was that xargs -0 would receive all the file names as a single file name which obviously does not exist, and you forgot to include the ": file not found" from the end of the error message.
The use of grep -E is a minor hack to enable a more modern regex syntax which is more similar to that in Awk, where you don't have to backslash the "or" pipe etc.
This script, like your original, runs grep on the local system where you run the SSH script. If you want to run the commands on the remote server, you will need to refactor to put the entire pipeline in single quotes or a here document:
for s in 172.x.x.x 172.x.x.y 172.x.x.z; do
ssh -i /home/user/.ssh/my_key.rsa "user#${s}" <<\________HERE
crontab -l |
awk '! /^#|^[[:space:]]*$/ && $6 !~ /^$|^(echo|find|PATH|\/usr\/bin|\/bin\/)/ { print $6 }' |
xargs grep -Ein 'server\.tld|10.x.x.x'
________HERE
done
The refactored script contains enough complexities in the quoting that you probably don't want to pass it as an argument to ssh, which requires you to figure out how to quote strings both locally and remotely. It's easier then to pass it as standard input, which obviously just gets transmitted verbatim.
If you get "Pseudo-terminal will not be allocated because stdin is not a terminal.", try using ssh -t. Sometimes you need to add multiple -t options to completely get rid of this message.
I have a logfile
...
Host: 111.222.121.123 (111.222.121.123.deploy.static.akamaitechnologies.com) Ports: 80/open/tcp//http//AkamaiGHost (Akamai's HTTP Acceleration|Mirror service)/, 443/open/tcp//ssl|http//AkamaiGHost (Akamai's HTTP Acceleration|Mirror service)/
Host: 1.2.3.4 () Ports: 80/open/tcp//http//cloudflare/, 443/open/tcp//ssl|https//cloudflare/, 2052/open/tcp//clearvisn?///, 2053/open/tcp//ssl|http//nginx/, 2082/open/tcp//infowave?///, 2083/open/tcp//ssl|http//nginx/, 2086/open/tcp//gnunet?///, 2087/open/tcp//ssl|http//nginx/, 2095/open/tcp//nbx-ser?///, 2096/open/tcp//ssl|http//nginx/, 8080/open/tcp//http-proxy//cloudflare/, 8443/open/tcp//ssl|https-alt//cloudflare/, 8880/open/tcp//cddbp-alt?///
Host: 2.3.4.5 (a104-96-1-61.deploy.static.akamaitechnologies.com) Ports: 53/open/tcp//domain//(unknown banner: 29571.61)/
...
I need to extract and convert IPs and http ports to the following format
1.2.3.4:80,443,2083
There are just two types of port fields in the logfile
80/open/tcp//http
2083/open/tcp//ssl|http
Tried to use sed but without success. I ended up with this dysfunctional command
cat ../host_ports.txt | sed -rn 's/Host: ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*?([0-9]{1,5}\/open\/tcp\/\/http|[0-9]{1,5}\/open\/tcp\/\/ssl\|http).*/\1 \2/p'
First handle the repeating ports, and next replace Host/Port to the desired format.
sed -r 's/(Ports:|,) ([0-9]*)[^,]*/\1\2/g;s/Host: ([^ ]*).*Ports:/\1:/' ../host_ports.txt
EDIT:
First I gave all ports of a line with http somewhere, now limit the result to ports with http in its description.
sed -nr 's/Ports: /, /;
s/, ([0-9]*)[^,]*http[^,]*/,\1/g;
s/,[^,]*\/[^,]*//g;
s/Host: ([^ ]*)[^,]*,/\1:/p' ../host_ports.txt
This script will do it for you, and you don't need sed :
#!/bin/bash
while read -r line; do
if echo $line | grep -q "http"; then
host=$(echo "$line" | grep -Po '(?<=^Host: )[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+')
ports=$(echo "$line" | grep -Po '[0-9]*((?=\/open\/tcp\/\/http)|(?=\/open\/tcp\/\/ssl\|http))' | tr '\n' ',')
echo "$host:${ports:0:-1}"
fi
done < ../log
The first grep will catch the IP address, with the help of Look behind. the -P is to use perl like regex, and the -o is to output only the matching string
The second regex is much like the first, but uses look after instead of look behind. It will only capture ports which are followed by /open/tcp//http or /open/tcp//ssl|http. The tr right after will replace newlines with commas.
the ${ports:0:-1} is just to eliminate the trailing comma.
Hope this helps!
So I have this script that does something with IPs allocated to my OS (GNU/Linux) that I get from running ifconfig. It works fine, however, I was wondering if I could filter out loopback/localhost IP (127.0.0.1) in the same regex expression [I assume every server within my cluster has said IP and I don't need to do anything with it in my script.]
What my script uses is:
ifconfig | awk '/(([0-9]{1,3}\.){3})/ {print}' |sed -e "s/.*addr\://g" -e "s/\s.*//g"
I get results like:
> ifconfig | awk '/(([0-9]{1,3}\.){3})/ {print}' |sed -e "s/.*addr\://g" -e "s/\s.*//g"
172.16.0.1
127.0.0.1
I know it might be a stupid question, but could I filter any IP that starts with 127 in my first regex?
I could try changing awk for grep, somethin like:
> ifconfig |egrep -o "addr\:(([0-9]{1,3}\.){3}[0-9]{1,3})" |sed -e "s/.*addr\://g"
but if I try to negate (?!127) at the beginning, bash will interpret it as !127 which would just throw me something from the history.
I mean, I could just run another grep at the end of the oneliner like grep -v "127.0.0.1", but I just wanted to avoid greping something already greped. Not that anything is wrong with that, just trying to know little more and be more efficient, I guess.
With only one grep without sed or awk:
# ip a|grep -oP "inet \K[0-9.]*(?=.*[^ ][^l][^o]$)"
192.168.1.31
172.16.5.31
You can just add a clause to match the 127.0.0.1 and exclude it by adding the next as below. This way Awk ignores doing any action on the lines containing this pattern.
.. | awk '/127.0.0.1/{next}/(([0-9]{1,3}\.){3})/{print}' | ..
I have many configuration files ,
the line that start with LINE word have IP address
My target to read the line that start with LINE word from the file and print only the IP address
The problem is that IP address can be in any field in the line so I can’t capture the IP according to field number
example
grep LINE file1.txt
LINE /home/Ariate/run.pl "Voda STS 4 Test - " "102841" && ssh 17.77.170.130 -p 2022
grep LINE file2.txt
LINE /home/Ariate/run.pl 137.77.170.30 "Voda STS 4 Test - " "102841" && ssh ACTIVE
please advice how to capture the IP address from the line ( solution can be also with perl one liner )
expected results
echo $IP_FROM_LINE
17.77.170.130
echo $IP_FROM_LINE
137.77.170.30
perl -MRegexp::Common=net -lne 'print $1 if /^LINE.*\b($RE{net}{IPv4})/'
Using this grep -oE:
grep -oE '\d+\.\d+\.\d+\.\d+' file
17.77.170.130
137.77.170.30
OR else:
grep -oP '\d+\.\d+\.\d+\.\d+' file
The following will get you the desired IP addresses:
grep -oP '^LINE.*\b\K\d+\.\d+\.\d+\.\d+' file
To place the result in a variable as request, you'll need to iterate of the results as follows:
grep -oP '^LINE.*\b\K\d+\.\d+\.\d+\.\d+' file |
while read IP_FROM_LINE ; do
echo $IP_FROM_LINE
done
grep -oE '[0-9]{2,3}(\.[0-9]{2,3}){3}'
matches
17.77.170.130
137.77.170.30
or
grep -oP '\d{2}(\.\d{2}){3}'
if your grep supports -P option.
both of them works with the data you have given.
But if you want really worried of what to be matched, use
grep -Eo '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)'
which would match excat ip addresses.
I'm trying to write a script with a file as an argument that greps the text file to find any word that starts with a capital and has 8 letters following it. I'm bad with syntax so I'll show you my code, I'm sure it's an easy fix.
grep -o '[A-Z][^ ]*' $1
I'm not sure how to specify that:
a) it starts with a capital letter, and
b)that it's a 9 letter word.
Cheers
EDIT:
As an edit I'd like to add my new code:
while read p
do
echo $p | grep -Eo '^[A-Z][[:alpha:]]{8}'
done < $1
I still can't get it to work, any help on my new code?
'[A-Z][^ ]*' will match one character between A and Z, followed by zero or more non-space characters. So it would match any A-Z character on its own.
Use \b to indicate a word boundary, and a quantifier inside braces, for example:
grep '\b[A-Z][a-z]\{8\}\b'
If you just did grep '[A-Z][a-z]\{8\}' that would match (for example) "aaaaHellosailor".
I use \{8\}, the braces need to be escaped unless you use grep -E, also known as egrep, which uses Extended Regular Expressions. Vanilla grep, that you are using, uses Basic Regular Expressions. Also note that \b is not part of the standard, but commonly supported.
If you use ^ at the beginning and $ at the end then it will not find "Wiltshire" in "A Wiltshire pig makes great sausages", it will only find lines which just consist of a 9 character pronoun and nothing else.
This works for me:
$ echo "one-Abcdefgh.foo" | grep -o -E '[A-Z][[:alpha:]]{8}'
$ echo "one-Abcdefghi.foo" | grep -o -E '[A-Z][[:alpha:]]{8}'
Abcdefghi
$
Note that this doesn't handle extensions or prefixes. If you want to FORCE the input to be a 9-letter capitalized word, we need to be more explicit:
$ echo "one-Abcdefghij.foo" | grep -o -E '\b[A-Z][[:alpha:]]{8}\b'
$ echo "Abcdefghij" | grep -o -E '\b[A-Z][[:alpha:]]{8}\b'
$ echo "Abcdefghi" | grep -o -E '\b[A-Z][[:alpha:]]{8}\b'
Abcdefghi
$
I have a test file named 'testfile' with the following content:
Aabcdefgh
Babcdefgh
cabcdefgh
eabcd
Now you can use the following command to grep in this file:
grep -Eo '^[A-Z][[:alpha:]]{8}' testfile
The code above is equal to:
cat testfile | grep -Eo '^[A-Z][[:alpha:]]{8}'
This matches
Aabcdefgh
Babcdefgh