BASH with sshpass send same command to multiple servers based on list of servers? [duplicate] - linux

This question already has answers here:
Loop through an array of strings in Bash?
(21 answers)
Closed 2 years ago.
Hi I have the following "script" below which works fine.
#!/bin/bash
TARGET="vm-windows-server01.local"
TARGET_USER="administrator"
TARGET_PASSWORD="Qwerty"
TARGET2="vm-linux-server01.local"
TARGET_USER2="root"
TARGET_PASSWORD2="Qwerty123"
#Windows server shutdown
sshpass -p $TARGET_PASSWORD ssh -o stricthostkeychecking=no $TARGET_USER#$TARGET 'shutdown /s /c "Shutting down"'
#Linux server shutdown
sshpass -p $TARGET_PASSWORD2 ssh -o stricthostkeychecking=no $TARGET_USER2#$TARGET2 'shutdown 3 "Shutting down"'
Now instead of copying TARGET variables for over 50 servers along with 50 more lines of the sshpass
Can something like this below be done? Please be advised its bogus code just to give the idea what i want. I'm new to bash so be gently :-)
#!/bin/bash
WINDOWS_SERVERS="
vm-windows-server01.local,
vm-windows-server02.local,
vm-windows-server03.local"
WINDOWS_USER="administrator"
WINDOWS_PASSWORD="Qwerty"
LINUX_SERVERS="
vm-linux-server01.local,
vm-linux-server02.local,
vm-linux-server03.local"
LINUX_USER="root"
LINUX_PASSWORD="Qwerty123"
#Windows server shutdown
sshpass -p $WINDOWS_PASSWORD ssh -o stricthostkeychecking=no $WINDOWS_USER#$WINDOWS_SERVERS 'shutdown /s /c "Shutting down"'
#Linux server shutdown
sshpass -p $LINUX_PASSWORD ssh -o stricthostkeychecking=no $LINUX_USER#$LINUX_SERVERS 'shutdown 3 "Shutting down"'
Basically runs the sshpass for all the hosts in the list.

If you are unsure about the availability of Bash, or another shell which knows about arrays; you can adapt your code to iterate over your server lists with the POSIX available grammar (no array):
#!/usr/bin/env sh
WINDOWS_SERVERS="
vm-windows-server01.local,
vm-windows-server02.local,
vm-windows-server03.local"
WINDOWS_USER="administrator"
WINDOWS_PASSWORD="Qwerty"
LINUX_SERVERS="
vm-linux-server01.local,
vm-linux-server02.local,
vm-linux-server03.local"
LINUX_USER="root"
LINUX_PASSWORD="Qwerty123"
echo "$WINDOWS_SERVERS" | {
IFS=$(printf '\t\n, ')
while read -r windows_host || [ "$windows_host" ]
do
sshpass -p "$WINDOWS_PASSWORD" ssh -o stricthostkeychecking=no "$WINDOWS_USER#$windows_host" 'shutdown /s /c "Shutting down"'
done
}
echo "$LINUX_SERVERS" | {
IFS=$(printf '\t\n, ')
while read -r linux_host || [ "$linux_host" ]
do
sshpass -p "$LINUX_PASSWORD" ssh -o stricthostkeychecking=no "$LINUX_USER#$linux_host" 'shutdown 3 "Shutting down"'
done
}
But if you have bash, zsh or ksh 93+ who supports arrays, your script syntax is simpler and easier to maintain:
#!/usr/bin/env bash
WINDOWS_SERVERS=(
'vm-windows-server01.local'
'vm-windows-server02.local'
'vm-windows-server03.local'
)
WINDOWS_USER='administrator'
WINDOWS_PASSWORD='Qwerty'
LINUX_SERVERS=(
'vm-linux-server01.local'
'vm-linux-server02.local'
'vm-linux-server03.local'
)
LINUX_USER='root'
LINUX_PASSWORD='Qwerty123'
for windows_host in "${WINDOWS_SERVERS[#]}"
do
sshpass -p "$WINDOWS_PASSWORD" ssh -o stricthostkeychecking=no "$WINDOWS_USER#$windows_host" 'shutdown /s /c "Shutting down"'
done
for linux_host in "${LINUX_SERVERS[#]}"
do
sshpass -p "$LINUX_PASSWORD" ssh -o stricthostkeychecking=no "$LINUX_USER#$linux_host" 'shutdown 3 "Shutting down"'
done

Related

Shell script to login to remote VM and run commands from remote VM

I need to login to a remote VM and run some commands. I am able to login successfully but echo command does not return list of files from the remote VM instead it returns output from local machine. Can anyone suggest how can I achieve this?
ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no root#$1<<EOF
cd /var;
echo "$(ls)";
EOF
exit
It worked after removing echo. PFB solution:
ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no root#$1 << EOT
cd /var;
ls -ltr;
EOT
exit
You have to escape $ in EOF sequence like this:
ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no root#$1<<EOF
cd /var;
echo "\$(ls)";
EOF
Or escape whole EOF sequence like this:
ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no root#$1<<'EOF'
cd /var;
echo "$(ls)";
EOF
alternatively, if you quote the marker, then substitution doesn't take place in the local shell:
ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no root#$1<<'EOF'
cd /var;
echo "$(ls)";
EOF
exit

How to run the bash script in parallel rather than sequentially?

#!/bin/bash
some_array=($1)
echo "-- Setting-Up VM --"
for i in ${some_array[#]}; do
echo "VM #: $i"
case "$i" in
"1")
echo "Setting-Up VM $i"
sshpass -p "root12" ssh -tt -o StrictHostKeyChecking=no root#10.xx.x.xx <<EOF
pwd
nohup yes | /etc/rc.d/init.d/lifconfig
su tarts
nohup yes | vncserver
sleep 10
exit
exit
EOF
;;
"2")
echo "Setting-Up VM $i"
sshpass -p "root12" ssh -tt -o StrictHostKeyChecking=no root#10.xx.x.xx <<EOF
pwd
nohup yes | /etc/rc.d/init.d/lifconfig
su tarts
nohup yes | vncserver
sleep 10
exit
exit
EOF
;;
"3")
echo "Setting-Up VM $i"
sshpass -p "root12" ssh -tt -o StrictHostKeyChecking=no root#10.xx.x.xx <<EOF
pwd
nohup yes | /etc/rc.d/init.d/lifconfig
su tarts
nohup yes | vncserver
sleep 10
exit
exit
EOF
;;
*)
echo "unknown VM!"
;;
esac
done
Can someone please guide me I have the above script which is executed when we run the script for instance ./vmSetup.sh "1 2 3" but this is executed sequentially. I had created this script but now I want to run the cases in the script i.e. 1, 2 and 3 in parallel. Can someone also tell me how to run for instance 8 cases in parallel?
Put each case ... esac statement in the background by ending it with &. Then use wait after the loop to wait for all the background processes to finish.
#!/bin/bash
some_array=($1)
echo "-- Setting-Up VM --"
for i in ${some_array[#]}; do
echo "VM #: $i"
case "$i" in
"1")
echo "Setting-Up VM $i"
sshpass -p "root12" ssh -tt -o StrictHostKeyChecking=no root#10.xx.x.xx <<EOF
pwd
nohup yes | /etc/rc.d/init.d/lifconfig
su tarts
nohup yes | vncserver
sleep 10
exit
exit
EOF
;;
"2")
echo "Setting-Up VM $i"
sshpass -p "root12" ssh -tt -o StrictHostKeyChecking=no root#10.xx.x.xx <<EOF
pwd
nohup yes | /etc/rc.d/init.d/lifconfig
su tarts
nohup yes | vncserver
sleep 10
exit
exit
EOF
;;
"3")
echo "Setting-Up VM $i"
sshpass -p "root12" ssh -tt -o StrictHostKeyChecking=no root#10.xx.x.xx <<EOF
pwd
nohup yes | /etc/rc.d/init.d/lifconfig
su tarts
nohup yes | vncserver
sleep 10
exit
exit
EOF
;;
*)
echo "unknown VM!"
;;
esac &
done
wait
Why do you have a case statement at all when all these are identical?
#!/bin/bash
echo "-- Setting-Up VM(s) --"
for i in "$#"; do
case "$i" in
1) IP=1.2.3.4;;
2) IP=2.2.3.4;;
3) IP=3.2.3.4;;
*) echo "Invalid option '$i'" >&2; exit 1;;
esac
echo "Setting-Up VM $i"
sshpass -p root12 ssh -tt -o StrictHostKeyChecking=no root#$IP <<EOF &
pwd
nohup yes | /etc/rc.d/init.d/lifconfig
su tarts
nohup yes | vncserver
sleep 10
exit
exit
EOF
done
To run a job in background, just use &
sshpass -p "root12" ssh -tt -o StrictHostKeyChecking=no root#$IP <<EOF &
It's just confusing because of the here-doc, but the & metacharacter still parses correctly on my system. Try it.
Also, rather than quoting all your options in one string and then parsing them back out to an array, why not just simplify the whole thing and let them come in as separate arguments? Call it as
./vmSetup.sh 1 2 3 # NOT "1 2 3" with quotes
and the loop becomes just
for i in "$#" # properly quoted, though wouldn't matter for 1 2 3
This whole thing seems a lot simpler and easier to maintain.
Another option, which is what I would do: create a file that is a list of the DNS/IP addresses you need, then pass that file as the lone argument.
#!/bin/bash
while read -r addr || [[ -n "$addr" ]]
do echo "Setting-Up VM $addr"
sshpass -p "root12" ssh -tt -o StrictHostKeyChecking=no root#$addr <<EOF &
pwd
nohup yes | /etc/rc.d/init.d/lifconfig
su tarts
nohup yes | vncserver
sleep 10
exit
exit
EOF
done < "$1"
Better, add valid error checking first to make sure the file exists and is readable, etc, but as a simple case, this should work.
For even better, cleaner, safer code, read https://mywiki.wooledge.org/BashFAQ/001 and follow those suggestions. :)
If ./vmSetup.sh 1 works as expected, these should work:
parallel -j8 ./vmSetup.sh {} ::: 1 2 3 ... 100
seq 100 | parallel -j8 ./vmSetup.sh

Error when stacking SSH command arguments within a bash script using other scripts as variables

I have a csv file called addresses.csv which looks like this,
node-1,xx.xxx.xx.xx,us-central-a
....
node-9,xxx.xx.xxx.xx,us-east1-a
I have a script below called 0run.sh,
#!/bin/bash
username='user'
persist="bash /home/${username}/Documents/scripts/disk/persistentDisk.sh"
first="bash /home/${username}/Documents/scripts/disk/firstAttach.sh"
while IFS=, read -r int ip <&3; do
if [ "$int" == "node-1" ]; then
--->ssh -i ~/.ssh/key -o StrictHostKeyChecking=no -l ${username} ${ip} "${persist}; ${first}"<---
else
ssh -i ~/.ssh/key -o StrictHostKeyChecking=no -l ${username} ${ip} "${first}"
fi
done 3<addresses.csv
The error occurs in the part of the code where I drew the arrows.
When it runs on node-1, instead of running ..persistentDisk.sh followed by ..firstAttach.sh, it only runs ..persistentDisk.sh and gives me the following error before it runs ..persistentDisk.
bash: /home/user/Documents/scripts/disk/firstAttach.sh: No such file or directory
The rest of the script runs completely fine. The only error occurs at this one part where it misses the 2nd script.
When I run the command like this it runs fine.
ssh -i ~/.ssh/key -o StrictHostKeyChecking=no -l ${username} ${ext} "${first}"
When I run it like this, it runs fine as well.
ssh -i ~/.ssh/key -o StrictHostKeyChecking=no -l user xxx.xx.xxx.xx "bash /home/${username}/Documents/scripts/disk/persistentDisk.sh; bash /home/${username}/Documents/scripts/disk/firstAttach.sh"
When I run the command like with a \ before the ; to escape it like this,
ssh -i ~/.ssh/key -o StrictHostKeyChecking=no -l ${username} ${ext} "${persist}\; ${first}"
I get the following error, and neither scripts run within the node-1 part of the code, but the rest of the code's else loops run fine.
bash: /home/user/Documents/scripts/disk/persistentDisk.sh;: No such file or directory
Why can't I stack the 2 commands within the if statement in the ssh using variables?
If I clearly understand: your real problem consist to leave STDIN free for interaction in target host!
About read and redirection
Try using:
#!/bin/bash
username='user'
persist="bash /home/${username}/Documents/scripts/disk/persistentDisk.sh"
first="bash /home/${username}/Documents/scripts/disk/firstAttach.sh"
while IFS=, read -r -u $list int ip foo; do
if [ "$int" == "node-1" ]; then
echo CMD... $ip, $persist
else
[ "$ip" ] && echo CMD... $ip, $first
fi
done {list}<addresses.csv
Tested, this èroduce:
CMD... xx.xxx.xx.xx, bash /home/user/Documents/scripts/disk/persistentDisk.sh
CMD... xxx.xx.xxx.xx, bash /home/user/Documents/scripts/disk/firstAttach.sh
-u flag to read, tell to use file descriptor ${list} instead of STDIN
foo is some useless variable used to prevent rest of line to be stored in $ip (xx.xxx.xx.xx,us-central-a in this case)
{list}</path/to/filename create a new variable by finding any free file descriptor.
About ssh (and redirection)
You could use:
#!/bin/bash
username='user'
persist="/home/${username}/Documents/scripts/disk/persistentDisk.sh"
first="/home/${username}/Documents/scripts/disk/firstAttach.sh"
while IFS=, read -r -u $list int ip foo; do
[ "$int" = "node-1" ] && cmd=persist || cmd=first
[ "$ip" ] && ssh -i ~/.ssh/key -t -o StrictHostKeyChecking=no \
-l ${username} ${ext} /bin/bash "${!cmd}"
done {list}<addresses.csv
By using this syntax, you will keep STDIN free for script running on target host.

Automate SSH Configuration for new Hadoop cluster

Guessing this has been done many times, but I could not find a clean answer, so I am appealing to your expertise for a better solution:
Objective: I am setting up a moderate sized RHEL Hadoop cluster and want to automate the configuration of SSH connectivity between all nodes. On the first node I have a list of all of the IPs in the file 'remote_ips', and I have the hduser password in the file 'hduser_pw' (chmod 600).
Step 1) Create "hduser_pw" password file on each host
for x in $(cat remote_ips); do sshpass -p$(cat hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$x "echo $(cat hduser_pw) > hduser_pw; chmod 600 hduser_pw"; done
Step 2) Generate RSA keys for each node in cluster:
for x in $(cat remote_ips); do sshpass -p$(cat hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$x "echo -e 'y'| ssh-keygen -t rsa -N \"\"
"; done
Step 3) Copy the file 'remote_ips' to each node in the cluster:
for x in $(cat remote_ips); do sshpass -p$(cat hduser_pw) scp -o StrictHostKeyChecking=no remote_ips hduser#$x:~; done
Step 4) For each node, copy the RSA public key to "authorized_keys" in every other node:
for x in $(cat remote_ips); do sshpass -p$(cat hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$x 'for y in $(cat remote_ips); do cat /home/hduser/.ssh/id_rsa.pub | sshpass -p$(cat hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$y '\''cat >> .ssh/authorized_keys'\'' ; done '; done
Is there a better way to do this? Really appreciate your help.
EDIT: Here are my revisions:
I incorporated the feedback from #janos and #likewhoa --> I handled the UUOC with redirection, looped across each IP, removed any pwds from the shell history with variables, etc. Thanks so much!
hduser_pw=$(< hduser_pw)
remote_ips=$(< remote_ips)
for x in $remote_ips; do
echo "Create hduser_pw password file on node: ${x}"
sshpass -p$hduser_pw scp -o StrictHostKeyChecking=no hduser_pw hduser#$x:~
echo "chmod 600 hduser_pw on node: ${x}"
sshpass -p$hduser_pw ssh -o StrictHostKeyChecking=no hduser#$x "chmod 600 hduser_pw"
echo "Generate RSA keys for: ${x}"
sshpass -p$hduser_pw ssh -o StrictHostKeyChecking=no hduser#$x "echo y | ssh-keygen -f ~/.ssh/id_rsa -t rsa -N \"\""
echo "SCP the file remote_ips to node: ${x}"
sshpass -p$hduser_pw scp -o StrictHostKeyChecking=no remote_ips hduser#$x:~
done
for x in $remote_ips; do
for y in $remote_ips; do
echo "ssh-copy-id from node ${x} to node ${y}:"
sshpass -p$hduser_pw ssh -o StrictHostKeyChecking=no hduser#$x "sshpass -p${hduser_pw} ssh-copy-id -i ~/.ssh/id_rsa.pub hduser#${y} -o StrictHostKeyChecking=no";
done
done
Instead of running $(cat hduser_pw) and $(cat remote_ips) multiple times,
it would be better to run them only once and, save in variables and reuse.
For example:
hduser_pw=$(cat hduser_pw)
remote_ips=$(cat remote_ips)
# Step 1) Create "hduser_pw" password file on each host
for x in $remote_ips; do
sshpass -p$hduser_pw ssh -o StrictHostKeyChecking=no hduser#$x "echo $hduser_pw > hduser_pw; chmod 600 hduser_pw"
done
Can you spot the security problem? The echo $hduser_pw will be saved in the shell history, and may also be visible in ps printings. It's better to avoid it. Instead of this:
ssh server "echo $hduser_pw > hduser_pw; chmod 600 hduser_pw"
You can do like this:
ssh server "cat > hduser_pw; chmod 600 hduser_pw" < hduser_pw
That's a bit hacky and may be confusing.
A simpler option is to scp and then ssh (for the chmod):
scp hduser_pw server:
ssh server "chmod 600 hduser_pw"
Other simplifications:
Instead of echo -e 'y' | ... you can simplify to echo y | ...
Instead of the messy Step 4, take a look into ssh-copy-id if it's available (usually it is in Linux systems)
First off UUOC, you really should use a while loop and re-directions instead of CAT.
Use this instead for the following steps which merges all steps into one big while loop.
while read ips; do sshpass -p$(<hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$ips "echo $(<hduser_pw) > hduser_pw; chmod 600 hduser_pw"; sshpass -p$(<hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$ips ssh-keygen -f ~/.ssh/id_rsa -t rsa -N \"\"";sshpass -p$(<hduser_pw) scp -o StrictHostKeyChecking=no remote_ips hduser#$ips:;done <remote_ips
I'll let you fix Step 4 on your own. GL!
Few pointers.
Use while loops when reading through lines
ssh-keygen -f ~/.ssh/somekey to avoid 'echo y | ssh-keygen ...'
Use redirect '
Good luck! Perhaps this is best if you use some kind of Configuration Management tools to automate this and more.

shell script ssh command not working

I have a small list of servers, and I am trying to add a user on each of these servers. I can ssh individually to each server and run the command.
sudo /usr/sbin/useradd -c "Arun" -d /home/amurug -e 2014-12-12 -g users -u 1470 amurug
I wrote a script to loop through the list and run this command but I get some errors.
#!/bin/bash
read -p "Enter server list: " file
if [[ $file == *linux* ]]; then
for i in `cat $file`
do
echo "creating amurug on" $i
ssh $i sudo /usr/sbin/useradd -c "Arun" -d /home/amurug -e 2014-12-12 -g users -u 1470 amurug
echo "==============================================="
sleep 5
done
fi
When I run the script it does not execute the command.
creating amurug on svr102
Usage: useradd [options] LOGIN
Options:
What is wrong with my ssh crommand in my script?
Try this script:
#!/bin/bash
read -p "Enter server list: " file
if [[ "$file" == *linux* ]]; then
while read -r server
do
echo "creating amurug on" "$server"
ssh -t -t "$server" "sudo /usr/sbin/useradd -c Arun -d /home/amurug \
-e 2014-12-12 -g users -u 1470 amurug"
echo "==============================================="
sleep 5
done < "$file"
fi
As per man bash:
-t
Force pseudo-tty allocation. This can be used to execute arbitrary screen-based programs on a remote which can be very useful, e.g. when implementing menu services. Multiple -t options force tty allocation, even if ssh has no local tty.

Resources