I wonder how to create a pipe
program 1 | ... | program N
where multiple of the programs ask for user input. The problem is that | starts the programs in parallel and thus they start reading from the terminal in parallel.
For such cases it would be useful to have a pipe | that starts program (i+1) only after program i has produced some output.
Edit:
Example:
cat /dev/sda | bzip2 | gpg -c | ssh user#host 'cat > backup'
Here both gpg -c as well as ssh ask for a password.
A workaround for this particular example would be the creation of ssh key pairs, but this is not possible on every system, and I was wondering whether there is a general solution.
Also gpg allows for the passphrase to be passed as command line argument, but this is not suggested for security reasons.
You can use this construction:
(read a; echo "$a"; cat) > file
For example:
$ (read a; echo "$a"; echo cat is started > /dev/stderr; cat) > file
1
cat is started
2
3
Here 1, 2 and 3 were entered from keyboard; cat is started was written by echo.
Contents of file after execution of the command:
$ cat file
1
2
3
I am now using:
#!/bin/bash
sudo echo "I am root!"
sudo cat /dev/disk0 | bzip2 | gpg -c | (read -n 1 a; (echo -n "$a"; cat) | ssh user#host 'cat > backup')
The first sudo will prevent the second from asking the password again. As suggested above, the read postpones the starting of ssh. I used -n 1 for read since I don't want to wait for newline, and -n for echo to surpress the newline.
for one you can give gpg the password with the --passphrase option.
For ssh the best solution would be to login by key. But if you need to do by password the expect command will be good. Here's a good example: Use expect in bash script to provide password to SSH command
Expect also allows you to have some input - so if you don't want to hardcode your passwords this might be the way to go.
I've needed something similar a few times before, where the first command in the pipeline requires a password to be entered, and the next command doesn't automatically cater for this (like the way that less does).
Similar to Igor's response, I find the use of read inside a subshell useful:
cmd1 | ( read; cat - | cmd2 )
Related
I have the following script in bash:
ssh user#1.1.1.1 "echo 'start'
mkdir -p /home/user/out
cp /tmp/big_file /home/user/out
echo 'syncing flash'
sync
while [[ $(cat /proc/meminfo | grep Dirty | awk '{print $2}') -ne 0 ]] ; do
echo \"$(cat /proc/meminfo)\"
sleep 1
sync
done
echo 'done'"
I have my host PC and a target PC which I am copying to. Before I run this script I have already scp'd a big file into /tmp on the target.
When I run this script it copies the file /tmp/big ok, but when it enters the loop to sync the flash and I wait for meminfo Dirty to get to zero what I see is always Dirty: 74224 kB repeated in the loop.
However in a different ssh session logged in to the target I have it running:
watch -n1 "cat /proc/meminfo | grep Drity"
And I see this count down from ~74000kb to 0kB.
The difference is that the ssh session doing the watch is logged in as root and the ssh is logged in a user.
So I did the same test with the ssh shell logged in as user and I saw always 0kb in Drity...
Does this imply that the user can't read meminfo relating to the whole system? - how can I tell when the flash has sync'd as a non-root user?
Since the argument to ssh is in double quotes, variables and command substitutions are expanded locally on the client before sending the command, they're not done on the remote machine. Since they're substituted on the client, you'll obviously get the same result each time through the loop (because the client isn't looping).
You should either escape the $ characters so they're sent to the server, or put the command inside single quotes (but the latter makes it difficult to include single quotes in the command).
ssh user#1.1.1.1 "echo 'start'
mkdir -p /home/user/out
cp /tmp/big_file /home/user/out
echo 'syncing flash'
sync
while [[ \$(awk '/Dirty/ {print \$2}' /proc/meminfo) -ne 0 ]] ; do
cat /proc/meminfo
sleep 1
sync
done
echo 'done'"
There's also no need for cat /proc/meminfo and grep Dirty in the command substitution. awk can do pattern matching and take a filename argument.
I am attempting to run multiple commands via a bash script on a remote server. specifically, the for loop to be run on the remote server is giving me issues. I suspect it is because I don't know how to properly escape characters or use $().
Below is the code.
ssh (user)#(server) <<EOF
sudo su - (username)
whoami
'for e in $(`ls -lrt /usr/jboss/jbosseap | awk '{print $9}' | grep multichannel`);
do
echo "$e";
done'
Removing user and server names for obvious reasons. Just concentrate on the for loop. when I run that for loop command line (without the $()) its works fine. Just not sure how to nest it in a remote call.
Thanks very much for any and all help!
If you've got a complex script that you're trying to run over ssh you're going to be better off putting that script in a file and piping that file into ssh like:
cat remote_script.sh | ssh user#host
or:
cat remote_script.sh | ssh user#host sudo -u username
And now you don't have to worry about N levels of escaping.
You can run it as below .
here file "list " includes your list of nodes and script should be present in all nodes
for i in $(cat list ) ;do ssh -o StrictHostKeyChecking=no $i "/path/your_script" ;done
I have a list of IP addresses, and my end goal is to ssh into each one, and reset them one-at-a-time. I was asked to use Linux / Bash, which I am not extremely familiar. My code right now will take the first IP from the list, and connect to it, but it never moves on past that point. I believe the issue is somewhere between the while read oneip3 and do code. Any help is greatly appreciated.
The way I run this script is as follows: (I have a list of IP addresses in a separate text file):
./runscript.txt ip_list.txt
while read oneip3
do
(sleep 5
echo "yes\r"
sleep 3
echo -e "password\r"
sleep 3
echo -e "reset\r"
sleep 3
echo -e "yes\r"
sleep 20
echo -e "\r"
) | ssh -t -t -oHostKeyAlgorithms=+ssh-dss admin#$oneip3
done < $1
You didn't provide SSH argument. So it opens an interactive shell.
It is a good reason to be stuck on the first machine (maybe there is other reason...)
Try this to debug
... | ssh -t -t -oHostKeyAlgorithms=+ssh-dss "admin#$oneip3" pwd
Other remarks in comment about StrictHostKeyChecking seams good too (if you are really concern by security, you can deploy all needed keys by hand firstly)
I'm getting the following mail every time I execute a specific cronjob. The called script runs fine when I'm calling it directly and even from cron. So the message I get is not an actual error, since the script does exactly what it is supposed to do.
Here is the cron.d entry:
* * * * * root /bin/bash -l -c "/opt/get.sh > /tmp/file"
and the get.sh script itself:
#!/bin/sh
#group and url
groups="foo"
url="https://somehost.test/get.php?groups=${groups}"
# encryption
pass='bar'
method='aes-256-xts'
pass=$(echo -n $pass | xxd -ps | sed 's/[[:xdigit:]]\{2\}/&/g')
encrypted=$(wget -qO- ${url})
decoded=$(echo -n $encrypted | awk -F '#' '{print $1}')
iv=$(echo $encrypted | awk -F '#' '{print $2}' |base64 --decode | xxd -ps | sed 's/[[:xdigit:]]\{2\}/&/g')
# base64 decode input and save to file
output=$(echo -n $decoded | base64 --decode | openssl enc -${method} -d -nosalt -nopad -K ${pass} -iv ${iv})
if [ ! -z "${output}" ]; then
echo "${output}"
else
echo "Error while getting information"
fi
When I'm not using the bash -l syntax the script hangs during the wget process. So my guess would be that it has something to do with wget and putting the output to stdout. But I have no idea how to fix it.
You actually have two questions here.
Why it prints stdin: is not a tty?
This warning message is printed by bash -l. The -l (--login) options asks bash to start the login shell, e.g. the one which is usually started when you enter your password. In this case bash expects its stdin to be a real terminal (e.g. the isatty(0) call should return 1), and it's not true if it is run by cron—hence this warning.
Another easy way to reproduce this warning, and the very common one, is to run this command via ssh:
$ ssh user#example.com 'bash -l -c "echo test"'
Password:
stdin: is not a tty
test
It happens because ssh does not allocate a terminal when called with a command as a parameter (one should use -t option for ssh to force the terminal allocation in this case).
Why it did not work without -l?
As correctly stated by #Cyrus in the comments, the list of files which bash loads on start depends on the type of the session. E.g. for login shells it will load /etc/profile, ~/.bash_profile, ~/.bash_login, and ~/.profile (see INVOCATION in manual bash(1)), while for non-login shells it will only load ~/.bashrc. It seems you defined your http_proxy variable only in one of the files loaded for login shells, but not in ~/.bashrc. You moved it to ~/.wgetrc and it's correct, but you could also define it in ~/.bashrc and it would have worked.
in your .profile, change
mesg n
to
if `tty -s`; then
mesg n
fi
I ended up putting the proxy configuration in the wgetrc. There is now no need to execute the script on a login shell anymore.
This is not a real answer to the actual problem, but it solved mine.
If you run into this problem check if you are getting all the environment variables set as you expect. Thanks to Cyrus for putting me to the right direction.
There are other threads with this same topic but my issue is unique. I am running a bash script that has a function that sshes to a remote server and runs a sudo command on the remote server. I'm using the ssh -t option to avoid the requiretty issue. The offending line of code works fine as long as it's NOT being called from within the while loop. The while loop basically reads from a csv file on the local server and calls the checkAuthType function:
while read inputline
do
ARRAY=(`echo $inputline | tr ',' ' '`)
HOSTNAME=${ARRAY[0]}
OS_TYPE=${ARRAY[1]}
checkAuthType $HOSTNAME $OS_TYPE
<more irrelevant code>
done < configfile.csv
This is the function that sits at the top of the script (outside of any while loops):
function checkAuthType()
{
if [ $2 == linux ]; then
LINE=`ssh -t $1 'sudo grep "PasswordAuthentication" /etc/ssh/sshd_config | grep -v "yes\|Yes\|#"'`
fi
if [ $2 == unix ]; then
LINE=`ssh -n $1 'grep "PasswordAuthentication" /usr/local/etc/sshd_config | grep -v "yes\|Yes\|#"'`
fi
<more irrelevant code>
}
So, the offending line is the line that has the sudo command within the function. I can change the command to something simple like "sudo ls -l" and I will still get the "stdin is not a terminal" error. I've also tried "ssh -t -t" but to no avail. But if I call the checkAuthType function from outside of the while loop, it works fine. What is it about the while loop that changes the terminal and how do I fix it? Thank you one thousand times in advance.
Another option to try to get around the problem would be to redirect the file to a different file descriptor and force read to read from it instead.
while read inputline <&3
do
ARRAY=(`echo $inputline | tr ',' ' '`)
HOSTNAME=${ARRAY[0]}
OS_TYPE=${ARRAY[1]}
checkAuthType $HOSTNAME $OS_TYPE
<more irrelevant code>
done 3< configfile.csv
I am guessing you are testing with linux. You should try add the -n flag to your (linux) ssh command to avoid having ssh read from stdin - as it normally reads from stdin the while loop is feeding it your csv.
UPDATE
You should (usually) use the -n flag when scripting with SSH, and the flag is typically needed for 'expected behavior' when using a while read-loop. It does not seem to be the main issue here, though.
There are probably other solutions to this, but you could try adding another -t flag to force pseudo-tty allocation when stdin is not a terminal:
ssh -n -t -t
BroSlow's approach with a different file descriptor seems to work! Since the read command reads from fd 3 and not stdin,
ssh and hence sudo still have or get a tty/pty as stdin.
# simple test case
while read line <&3; do
sudo -k
echo "$line"
ssh -t localhost 'sudo ls -ld /'
done 3<&- 3< <(echo 1; sleep 3; echo 2; sleep 3)