Disclaimer: I'm a noob to linux and scripting, please don't heckle me.
GOAL: Feed my linux system a .txt file full of IP Addresses and Perform a WHOIS look on each IP and show me specific fields (grep) such as the Organization Field. Bonus points if someone can help me figure out how to use my API key to check my IP list against abuseipdb.com
I've created an ip.txt file with my IP Addresses and I've tried using the following syntax. I'm using Kali Linux to perform this, but it worked on my friend's Fedora system.
sudo for ip in $(cat ip.txt); do whois $ip | echo "$ip $(grep -e 'Organization' | grep -v 'Verizon')"; done > whois.txt
My output is: -bash: syntax error near unexpected token 'do'. Remove "do" from my command and then I receive -bash: syntax error near unexpected token 'done'
Remove the "done" and the pipe out and then it's just pissed..
Please help a linux n00b :)
You are attempting to invoke sudo on the command for ip in $(cat ip.txt), and then running the syntactically invalid commands do whois .... (Note that for ip in $(cat ip.txt) is also syntactically invalid, but sudo is never invoked so no shell ever tries to parse that string as a command). You need to narrow the scope of sudo. eg
while read ip; do sudo whois "$ip" | ... ; done > whois.txt < ip.txt
Related
Hello People of the world,
I am trying to write a script that will allow user to failover apps between sites in bash.
Our applications are controlled by Pacemaker and I thought I would be able to write a function that would take in the necessary variables and act. Stop on one site, start on another. Once I have ssh'd to the remote machine, I am unable to get the value of the grep/awk command back for the status of the application in PCS.
I am encountering a few issues, and have tried answers from stackoverflow and other sites.
I send the ssh command to /dev/null 2>&1 as banners pop up on screen that unix admin have on the local user and -q does not deal with it - Does this stop anything being returned?
when using awk '{print \\\\\\$4}' in the code, I get a "backslash not last character on line" error
To get round this, I tried result=$(sudo pcs status | grep nds_$resource), however this resulted in a password error on sudo
I have tried >/dev/tty and >$(tty)
I tried to not suppress the ssh (remove /dev/null 2>&1) and put the output in variable at function call, removing the awk from the sudo pcs status entry.
result=$(pcs_call "$site1" "1" "2" "disable" "pmr")
echo $result | grep systemd
This was OK, but when I added | awk '{print \\\$4}' I then got the fourth word in the banner.
Any help would be appreciated as I have been going at this for a few days now.
I have been looking at this answer from Bruno, but unsure how to implement as I have multiple sudo commands.
Below is my strip down of the function code for testing on one machine;
site1=lon
site2=ire
function pcs_call()
{
site=$1
serverA=$2
serverB=$3
activity=$4
resource=$5
ssh -tt ${site}servername0${serverA} <<SSH > /dev/null 2>&1
sudo pcs resource ${activity} proc_${resource}
sleep 10
sudo pcs status | grep proc_$resource | awk '{print \\\$4}' | tee $output
exit
SSH
echo $output
}
echo ====================================================================================
echo Shutting Down PMR in $site1
pcs_call "$site1" "1" "2" "disable" "pmr"
I'd say start by pasting the whole thing into ShellCheck.net and fixing errors until there are no suggestions, but there are some serious issues here shellcheck is not going to be able to handle alone.
> /dev/null says "throw away into the bitbucket any data that is returned. 2>&1 says "Send any useful error reporting on stderr wherever stdout is going". Your initial statement, intended to retrieve information from a remote system, is immediately discarding it. Unless you just want something to occur on the remote system that you don't want to know more about locally, you're wasting your time with anything after that, because you've dumped whatever it had to say.
You only need one backslash in that awk statement to quote the dollar sign on $4.
Unless you have passwordless sudo on the remote system, this is not going to work out for you. I think we need more info on that before we discuss it any deeper.
As long as the ssh call is throwing everything to /dev/null, nothing inside the block of code being passed is going to give you any results on the calling system.
In your code you are using $output, but it looks as if you intend for tee to be setting it? That's not how that works. tee's argument is a filename into which it expects to write a copy of the data, which it also streams to stdout (tee as in a "T"-joint, in plumbing) but it does NOT assign variables.
(As an aside, you aren't even using serverB yet, but you can add that back in when you get past the current issues.)
At the end you echo $output, which is probably empty, so it's basically just echo which won't send anything but a newline, which would just be sent back to the origin server and dumped in /dev/null, so it's all kind of pointless....
Let's clean up
sudo pcs status | grep proc_$resource | awk '{print \\\$4}' | tee $output
and try it a little differently, yes?
First, I'm going to assume you have passwordless sudo, otherwise there's a whole other conversation to work that out.
Second, it's generally an antipattern to use both grep AND awk in a pipeline, as they are both basically regex engines at heart. Choose one. If you can make grep do what you want, it's pretty efficient. If not, awk is super flexible. Please read the documentation pages on the tools you are using when something isn't working. A quick search for "bash man grep" or "awk manual" will quickly give you great resources, and you're going to want them if you're trying to do things this complex.
So, let's look at a rework, making some assumptions...
function pcs_call() {
local site="$1" serverA="$2" activity="$3" resource="$4" # make local and quotes habits you only break on purpose
ssh -qt ${site}servername0${serverA} "
sudo pcs resource ${activity} proc_${resource}; sleep 10; sudo pcs status;
" 2>&1 | awk -v resource="$resource" '$0~"proc_"resource { print $4 }'
}
pcs_call "$site1" 1 disable pmr # should print the desired field
If you want to cath the data in a variable to use later -
var1="$( pcs_call "$site1" 1 disable pmr )"
addendum
Addressing your question - use $(seq 1 10) or just {1..10}.
ssh -qt chis03 '
for i in {1..10}; do sudo pcs resource disable ipa $i; done;
sleep 10; sudo pcs status;
' 2>&1 | awk -v resource=ipa '$0~"proc_"resource { print $2" "$4 }'
It's reporting the awk first, because order of elements in a pipeline is "undefined", but the stdout of the ssh is plugged into the stdin of the awk (and since it was duped to stdout, so is the stderr), so they are running asynchronously/simultaneously.
Yes, since these are using literals, single quotes is simpler and effectively "better". If abstracting with vars, it doesn't change much, but switch back to double quotes.
# assuming my vars (svr, verb, target) preset in the context
ssh -qt $svr "
for i in {1..10}; do sudo pcs resource $verb $target \$i; done;
sleep 10; sudo pcs status;
" 2>&1 | awk -v resource="$target" '$0~"proc_"resource { print $2" "$4 }'
Does that help?
Can someone help me with the syntax for a simple bash script that trying to write:
echo ping -c 1
echo nslookup
Basically I want to receive output of one line from the ping and the nslookup information for a domain that I'm checking. Unfortunately I'm unable to get this correctly.
P.s. this is basically the first thing that I'm trying to accomplish in bash.
Thank you in advance!
Thank you for the provided information on the matter. I felt little ashamed from the nature of my question so I spent little more time to read. The solution that I found is the following:
#!/bin/bash
for i in $*;do
ping -c 1 $i &
nslookup $i &
done
#
'
Once I added the scrit to the /bin folder I used the commands:
chmod +x "script name"
dos2unix ""scrit name"
so not I'm able to use it only by typing the name of the script.
hi after writing and saving script and giving permission to the script
just go the folder and
./
of else you just can use
sh
Hi i would be better if you elaborate your problem, if you want one line output for ping command filter using head
eg
ping www.google.com | head -n 1
or if you are thinking of sending one packet of data to the server
ping -n 1 www.google.com
nslookup www.google.com
and if your are writing Bash Script
#!/bin/bash
ping -n 1 www.google.com
nslookup www.google.com
save the file
give execute permission and run
Please let me know i answered your question
I have an executable bash script that picks up my external ip address from my modem and uploads it in a dynamic DNS service. The script is accompanied by a configuration file which I source from the script. The configuration file holds values for modem credentials, FQDN for my hostname, modem ip address and "status" website address.
In order to pick up my external IP address, I need to wget the "status" webpage from my modem and do some grep and sed operations to grab the external IP.
So the actual grep and sed code is also custom for every user.
The main question is: Canonically speaking, should such code be present in the conf file or in the script itself.
I am confused because I read this in 2 ways:
it is code so it should be in the script
it is variable so it should be in the config
I know this may sound like storm in a tea cup, but I would like to learn the right way.
Also, if you believe the right way is (2), I don't know how to declare the code in the configuration file. I tried the following and doesn't work. I've also not been able to find similar examples in the internet.
config:
grab_modem_ip='grep "[0-9]*\.[0-9]*\.[0-9]*\.[[0-9]*" |
grep tabdata | sed 's/\(.*\)<\/td>.*/\1/''
or
alias grab_modem_ip='grep "[0-9]*\.[0-9]*\.[0-9]*\.[[0-9]*" |
grep tabdata | sed 's/\(.*\)<\/td>.*/\1/''
(Notice that I have wrapped the whole command in '...' — single quotes.)
script
ipaddr=$( wget --user ${modemuser} --password ${modempass} "${modemsite}" -O - 2>/dev/null | "<grab_modem_ip>" )
I think that maybe 2 the correct solution. And I think that your command will solve using this:
ipaddr=$( wget --user ${modemuser} --password ${modempass} "${modemsite}" -O - 2>/dev/null | eval "grab_modem_ip" )
The answer I was looking for was "Variables hold data. Functions hold code. Don't put code inside variables!" as quoted from Greg's Wiki
so I rewrote grab_modem_ip as a function, that is defined in the conf file.
grab_modem_ip()
{
echo "${1}" | grep "[0-9]*\.[0-9]*\.[0-9]*\.[[0-9]*" | grep tabdata | sed 's/\(.*\)<\/td>.*/\1/'
}
Then invokation in the script is done as
modemstatuspage=$( wget --user ${modemuser} --password ${modempass} "${modemsite}" -O - 2>/dev/null )
ipaddr=$( grab_modem_ip "${modemstatuspage}")
I want command to get Linux machine(amazon) external/public IP Address.
I tried hostname -I and other commands from blogs and stackoverflow like
ifconfig | sed -En 's/127.0.0.1//;s/.*inet (addr:)?(([0-9]*\.){3}[0-9]*).*/\2/p'
and many more. But they all are giving me internal IP Address.
Then I found some sites which provides API for this.
Example : curl http://ipecho.net/plain; echo
But I don't want to rely on third party website service. So, is there any command line tool available to get external IP Address?
simplest of all would be to do :
curl ifconfig.me
A cleaner output
ifconfig eth0 | awk '/inet / { print $2 }' | sed 's/addr://'
You could use this script
# !/bin/bash
#
echo 'Your external IP is: '
curl -4 icanhazip.com
But that is relying on a third party albeit a reliable one.
I don't know if you can get your external IP without asking someone/somesite i.e. some third party for it, but what do I know.
you can also just run:
curl -4 icanhazip.com
This is doing the same thing as a command the -4 is to get the output in Ipv4
You can use this command to get public ip and private ip(second line is private ip; third line is public ip.)
ip addr | awk '/inet / {sub(/\/.*/, "", $2); print $2}'
I would suggest you to use the command external-ip (sudo apt-get install miniupnpc) as it (I'm almost sure) uses upnp protocol to ask the router instead of asking an external website so it should be faster, but of course the router has to have upnp enabled.
You can simply do this :
curl https://ipinfo.io/ip
It might not work on amazon because you might be using NAT or something for the server to access the rest of the world (and for you to ssh into it also). If you are unable to ssh into the ip that is listed in ifconfig then you are either in a different network or dont have ssh enabled.
This is the best I can do (only relies on my ISP):
ISP=`traceroute -M 2 -m 2 -n -q 1 8.8.8.8 | grep -m 1 -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'`
extIP=`ping -R -c 1 -t 1 -s 1 -n $ISP | grep RR | grep -m 1 -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'`
echo $extIP
Or, the functionally same thing on one line:
ISP=`traceroute -M 2 -m 2 -n -q 1 8.8.8.8 | grep -m 1 -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'` | ping -R -c 1 -t 1 -s 1 -n $ISP | grep RR | grep -m 1 -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'
to save it to a temporary & hidden file add > .extIP to the end of the last line, then cat .extIP to see it.
If your ISP's address never changes (honestly i'm not sure if it would or not), then you could fetch it once, and then replace $ISP in line two with it
This has been tested on a mac with wonderful success.
the only adjustment on linux that I've found so far is the traceroute "-M" flag might need to be "-f" instead
and it relies heavily on the ping's "-R" flag, which tells it to send back the "Record Route" information, which isn't always supported by the host. But it's worth a try!
the only other way to do this without relying on any external servers is to get it from curl'ing your modem's status page... I've done this successfully with our frontier DSL modem, but it's dirty, slow, unreliable, and requires hard-coding your modem's password.
Here's the "process" for that:
curl http://[user]:[password]#[modem's LAN address]/[status.html] | grep 'WanIPAddress =' | grep -m 1 -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'
That fetches the raw html, searches for any lines containing "WanIpAddress =" (change that so it's appropriate for your modem's results), and then narrows down those results to an IPv4 style address.
Hope that helps!
As others suggested, we have to rely on third party service which I don't feel safe using it. So, I have found Amazon API on this answer :
$ curl http://169.254.169.254/latest/meta-data/public-ipv4
54.232.200.77
For more details, https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-data-retrieval
The super-easy way is using the glances tool. you can install it on Ubuntu using:
$ sudo apt install glances
then using it with:
$ glances
and at the top of the terminal, it highlights your public IP address, and so many other information about your system (like what htop does) and network status.
For a formatted output use :-
dig TXT +short o-o.myaddr.l.google.com #ns1.google.com
it'll give you formatted output like this
"30.60.10.11"
also FYI,
dig is more faster than curl and wget
The following works as long as you have ifconfig and curl.
curl ifconfig.me
As per project requirement, i need to check the content of zip file generated which been generated on remote machine.This entire activity is done using automation framework suites. which has been written in shell scripts. I am performing above activity using ssh command abd execute unzip command with -l and -q switches. But this command is getting failed. and shows below error messages.
[SOMEUSER#MACHINE IP Function]$ ./TESTS.sh
ssh SOMEUSER#MACHINE IP unzip -l -q SOME_PATH/20130409060734*.zip | grep -i XML |wc -l
unzip: cannot find or open SOME_PATH/20130409060734*.zip, SOME_PATH/20130409060734*.zip.zip or SOME_PATH/20130409060734*.zip.ZIP.
No zipfiles found.
0
the same command i had written manually but that works properly. I really have no idea.Why this is getting failed whenever i executed via shell scripts.
[SOMEUSER#MACHINE IP Function]$ ssh SOMEUSER#MACHINE IP unzip -l -q SOME_PATH/20130409060734*.zip | grep -i XML |wc -l
2
Kindly help me to resolve that issue.
Thanks in Advance,
Priyank Shah
when you run the command from your local machine, the asterisk character is being expanded on your local machine before it is passed on to your remote ssh command. So your command is expecting to find SOME_PATH/20130409060734*.zip files on your machine and insert them into your ssh command to be passed to the other machine, whereas you (I'm assuming) mean, SOME_PATH/20130409060734*.zip files on the remote machine.
for that, precede the * character by a backslash ( \ ) and see if it helps you. In some shells escape character might be defined differently and if yours is one of them you need to find the escape character and use that one instead. Also, use quotes around the commands being passed to other server. Your command line should look something like this in my opinion:
ssh SOMEUSER#MACHINE_IP "/usr/bin/unzip -l -q SOME_PATH/20130409060734\*.zip | grep -i XML |wc -l"
Hope this helps