I am using two linux machines to simulate some firewall tests... I execute the tests by running nc through ssh on a remote machine... if I spawn the ssh like this, it works...
ssh -i id_dsa -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no \
-p 2224 root#a2-idf-lab nc -s 10.26.216.82 10.195.18.132 \
21 < /var/log/messages
However, if I try to control how much of /var/log/messages with head -c 20 /var/log/messages, the command hangs but I don't understand why...
ssh -i id_dsa -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no \
-p 2224 root#a2-idf-lab nc -s 10.26.216.82 10.195.18.132 \
21 < head -c 20 /var/log/messages
I also tried this with no better success...
ssh -i id_dsa -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no \
-p 2224 root#a2-idf-lab nc -s 10.26.216.82 10.195.18.132 \
21 < (head -c 20 /var/log/messages)
Question: Why does the second command hang, and how can I accomplish what I need?
FYI, these experiments were really in preparation for sending cat /dev/urandom | base64 | head -c 20 - into nc... bonus points if you can give me cli that would work with nc through an ssh session...
< is shell redirection, it redirects the input stream to read from a file, not to execute a command. try:
head -c 20 /var/log/messages | ssh -i id_dsa -o UserKnownHostsFile=/dev/null \
-o StrictHostKeyChecking=no \
-p 2224 root#a2-idf-lab nc -s 10.26.216.82 10.195.18.132 21
this pipes /var/log/messages from the local machine into nc on the remote machine.
if you want to use the /var/log/messages file on the remote machine, use quotes around the command:
ssh -i id_dsa -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no \
-p 2224 root#a2-idf-lab "head -c 20 /var/log/messages |\
nc -s 10.26.216.82 10.195.18.132 21"
Try to use
head -n 20
My guess is the problem is the lack of carriage return at the end.
Related
I need to login to a remote VM and run some commands. I am able to login successfully but echo command does not return list of files from the remote VM instead it returns output from local machine. Can anyone suggest how can I achieve this?
ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no root#$1<<EOF
cd /var;
echo "$(ls)";
EOF
exit
It worked after removing echo. PFB solution:
ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no root#$1 << EOT
cd /var;
ls -ltr;
EOT
exit
You have to escape $ in EOF sequence like this:
ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no root#$1<<EOF
cd /var;
echo "\$(ls)";
EOF
Or escape whole EOF sequence like this:
ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no root#$1<<'EOF'
cd /var;
echo "$(ls)";
EOF
alternatively, if you quote the marker, then substitution doesn't take place in the local shell:
ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no root#$1<<'EOF'
cd /var;
echo "$(ls)";
EOF
exit
I am attempting to run the following script on a virtual instance I have created on the google cloud:
#!/bin/bash
set -eu
DS=$(date "+%Y-%m-%d" -d "7 days ago")
DE=$(date "+%Y-%m-%d" -d "1 day ago")
account=123
## above specifies last weeks delivery
rm -f cookiejar
curl /dev/null -s -S -L -f -c cookiejar 'https://url.io/auth/authenticate' -d name=usr -d passwd='pwd'
curl -o /dev/null -s -S -L -f -b cookiejar -c cookiejar 'https://adloox.io/auth/adminaccounts' -d account=$account
curl -s -S -L -f -o "report1.xlsx" -J -b cookiejar -c cookiejar "https://url.io/adquality/ajax-adblocking?categoryFw=&platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&keywords=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=2628&zfTableItemPerPage=10000&zfTableExport=xlsx"
curl -s -S -L -f -o "report2.xlsx" -J -b cookiejar -c cookiejar "https://url.io/report/ajax-by-tag2?platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=10000&zfDetails=true&by_viewability=imps_sivt&device_id[]=all&tag_type_id[]=all&support_id[]=all&by_website=1&zfTableItemPerPage=10000&zfTableExport=xlsx"
Set Up
I have curl installed on my local desktop (running windows 10 with `Ubuntu 18.04.3 LTS)
I have curl installed on my remote (google cloud virtual instance set up with Ubuntu 18.04.3 LTS`)
Both have version curl 7.58.0
Issue
When run on my local desktop there is no issue and the files download. When run on my remote I am able to log in however I receive the following error for the next curl line:
+ curl -o /dev/null -s -S -L -f -b cookiejar -c cookiejar https://url.io/auth/adminaccounts -d account=123
curl: (22) The requested URL returned error: 500 Internal Server Error
Can someone confirm what else I should be looking at here? I would have thought if my linux and curl versions are the same, there wouldn't be an issue. Sorry if there are some other straight forward checks to do. This is my first time setting up a server.
If your virtual instance does not have a public IP address, this might help you out.
#!/bin/bash
set -eu
DS=$(date "+%Y-%m-%d" -d "7 days ago")
DE=$(date "+%Y-%m-%d" -d "1 day ago")
account=123
## above specifies last weeks delivery
rm -f cookiejar
curl /dev/null -s -S -L -f -c cookiejar 'https://url.io/auth/authenticate' -d name=usr -d passwd='pwd'
curl -o /dev/null -s -S -f -b cookiejar -c cookiejar 'https://adloox.io/auth/adminaccounts' -d account=$account
curl -s -S -L -f -o "report1.xlsx" -J -b cookiejar -c cookiejar "https://url.io/adquality/ajax-adblocking?categoryFw=&platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&keywords=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=2628&zfTableItemPerPage=10000&zfTableExport=xlsx"
curl -s -S -L -f -o "report2.xlsx" -J -b cookiejar -c cookiejar "https://url.io/report/ajax-by-tag2?platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=10000&zfDetails=true&by_viewability=imps_sivt&device_id[]=all&tag_type_id[]=all&support_id[]=all&by_website=1&zfTableItemPerPage=10000&zfTableExport=xlsx"
I want to execute specific script at remote server by ssh in background.
I found some solution about nohup.
But, nohup is not running without "2>&1"
I want to know what's the difference between existing "2>&1" and not.
nohup needs "2>&1" expression?
(Please understand my bad English)
This is my 'iperf_server.sh' script.
iperf -s -p 1 -w 128K
And, It is my host machine command.
$ ssh [id]#[host] "nohup echo [password] | sudo -S [Home_dir]/iperf_server.sh > /dev/null &"
$ ssh [id]#[host] "nohup echo [password] | sudo -S [Home_dir]/iperf_server.sh > /dev/null 2>&1 &"
$ ssh -t [id]#[host] "nohup echo [password] | sudo -S [Home_dir]/iperf_server.sh > /dev/null &"
Connection to iperf-server closed.
$ ssh -t [id]#[host] "nohup echo [password] | sudo -S [Home_DIR]/iperf_server.sh > /dev/null 2>&1 &"
Connection to iperf-server closed.
This is ps command result in iperf server
# ps -eLf | grep iperf | grep -v grep
# ps -eLf | grep iperf | grep -v grep
00:00:00 sudo -S [HOME_DIR]/iperf_server.sh
00:00:00 sh [HOME_DIR]/iperf_server.sh
00:00:00 iperf -s -p 1 -w 128K
00:00:00 iperf -s -p 1 -w 128K
00:00:00 iperf -s -p 1 -w 128K
# killall iperf
# ps -eLf | grep iperf | grep -v grep
# ps -eLf | grep iperf | grep -v grep
Take the & off the end.
This should do it:
ssh -t [id]#[host] "nohup echo [password] | sudo -S [Home_dir]/iperf_server.sh > /dev/null 2>&1"
By the way this is a huge security risk. Don't echo the password on the command line! If you really want to use a password like this at least do something like cat pwd.txt | sudo -S instead.
Guessing this has been done many times, but I could not find a clean answer, so I am appealing to your expertise for a better solution:
Objective: I am setting up a moderate sized RHEL Hadoop cluster and want to automate the configuration of SSH connectivity between all nodes. On the first node I have a list of all of the IPs in the file 'remote_ips', and I have the hduser password in the file 'hduser_pw' (chmod 600).
Step 1) Create "hduser_pw" password file on each host
for x in $(cat remote_ips); do sshpass -p$(cat hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$x "echo $(cat hduser_pw) > hduser_pw; chmod 600 hduser_pw"; done
Step 2) Generate RSA keys for each node in cluster:
for x in $(cat remote_ips); do sshpass -p$(cat hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$x "echo -e 'y'| ssh-keygen -t rsa -N \"\"
"; done
Step 3) Copy the file 'remote_ips' to each node in the cluster:
for x in $(cat remote_ips); do sshpass -p$(cat hduser_pw) scp -o StrictHostKeyChecking=no remote_ips hduser#$x:~; done
Step 4) For each node, copy the RSA public key to "authorized_keys" in every other node:
for x in $(cat remote_ips); do sshpass -p$(cat hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$x 'for y in $(cat remote_ips); do cat /home/hduser/.ssh/id_rsa.pub | sshpass -p$(cat hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$y '\''cat >> .ssh/authorized_keys'\'' ; done '; done
Is there a better way to do this? Really appreciate your help.
EDIT: Here are my revisions:
I incorporated the feedback from #janos and #likewhoa --> I handled the UUOC with redirection, looped across each IP, removed any pwds from the shell history with variables, etc. Thanks so much!
hduser_pw=$(< hduser_pw)
remote_ips=$(< remote_ips)
for x in $remote_ips; do
echo "Create hduser_pw password file on node: ${x}"
sshpass -p$hduser_pw scp -o StrictHostKeyChecking=no hduser_pw hduser#$x:~
echo "chmod 600 hduser_pw on node: ${x}"
sshpass -p$hduser_pw ssh -o StrictHostKeyChecking=no hduser#$x "chmod 600 hduser_pw"
echo "Generate RSA keys for: ${x}"
sshpass -p$hduser_pw ssh -o StrictHostKeyChecking=no hduser#$x "echo y | ssh-keygen -f ~/.ssh/id_rsa -t rsa -N \"\""
echo "SCP the file remote_ips to node: ${x}"
sshpass -p$hduser_pw scp -o StrictHostKeyChecking=no remote_ips hduser#$x:~
done
for x in $remote_ips; do
for y in $remote_ips; do
echo "ssh-copy-id from node ${x} to node ${y}:"
sshpass -p$hduser_pw ssh -o StrictHostKeyChecking=no hduser#$x "sshpass -p${hduser_pw} ssh-copy-id -i ~/.ssh/id_rsa.pub hduser#${y} -o StrictHostKeyChecking=no";
done
done
Instead of running $(cat hduser_pw) and $(cat remote_ips) multiple times,
it would be better to run them only once and, save in variables and reuse.
For example:
hduser_pw=$(cat hduser_pw)
remote_ips=$(cat remote_ips)
# Step 1) Create "hduser_pw" password file on each host
for x in $remote_ips; do
sshpass -p$hduser_pw ssh -o StrictHostKeyChecking=no hduser#$x "echo $hduser_pw > hduser_pw; chmod 600 hduser_pw"
done
Can you spot the security problem? The echo $hduser_pw will be saved in the shell history, and may also be visible in ps printings. It's better to avoid it. Instead of this:
ssh server "echo $hduser_pw > hduser_pw; chmod 600 hduser_pw"
You can do like this:
ssh server "cat > hduser_pw; chmod 600 hduser_pw" < hduser_pw
That's a bit hacky and may be confusing.
A simpler option is to scp and then ssh (for the chmod):
scp hduser_pw server:
ssh server "chmod 600 hduser_pw"
Other simplifications:
Instead of echo -e 'y' | ... you can simplify to echo y | ...
Instead of the messy Step 4, take a look into ssh-copy-id if it's available (usually it is in Linux systems)
First off UUOC, you really should use a while loop and re-directions instead of CAT.
Use this instead for the following steps which merges all steps into one big while loop.
while read ips; do sshpass -p$(<hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$ips "echo $(<hduser_pw) > hduser_pw; chmod 600 hduser_pw"; sshpass -p$(<hduser_pw) ssh -o StrictHostKeyChecking=no hduser#$ips ssh-keygen -f ~/.ssh/id_rsa -t rsa -N \"\"";sshpass -p$(<hduser_pw) scp -o StrictHostKeyChecking=no remote_ips hduser#$ips:;done <remote_ips
I'll let you fix Step 4 on your own. GL!
Few pointers.
Use while loops when reading through lines
ssh-keygen -f ~/.ssh/somekey to avoid 'echo y | ssh-keygen ...'
Use redirect '
Good luck! Perhaps this is best if you use some kind of Configuration Management tools to automate this and more.
I have a requirement to start nc on remote machine in screen and start transfer of file from another remote machine in screen, I am trying to run this through deploy machine(jenkins) with bash script
on remote machine 1 i.e tester1 :
ssh -tt mysql#tester1 'screen -d -m nc -l -w 60 5555 | tar xvif -'
on remote machine 2 i.e tester2 :
ssh -tt tester2 'screen -d -m sudo -u mysql innobackupex --stream=tar --databases="sampledb" /mysql-backup/prodfullbkp | nc -w 30 tester 5555'
While the two above commands are not working when running from deploy machine.Could someone please help me give any better way of doing this.
Thanks in advance =)
You can have a better solution like
ssh user#host << EOF
#command to excecute
EOF
ie tester1 would be
ssh -tt mysql#tester1 << EOF
screen -d -m nc -l -w 60 5555 | tar xvif -
EOF
tester2 would be
ssh -tt tester2 << EOF
screen -d -m sudo -u mysql innobackupex --stream=tar --databases="sampledb" /mysql-backup/prodfullbkp | nc -w 30 tester 5555
EOF