I've been working on a bash script that automatically runs certain scripts on remote machines and saves the logs to certain folders. As of now I have been copying the local script to the remote machine, executing it into a remote log, copying the remote log into a local folder, and then deleting the remote log and remote copy of the script.
This works, but I know it can work better if I can avoid doing all the in between steps. The one caveat is I need this to be automatic and passwordless (meaning no user input at all). One of the scripts needs to be ran as root or it won't display all the necessary information and will userlock the machine temporarily.
The code I am currently using to execute the remoteScript into a log that I later retrieve with scp is below.
sshpass -f password.txt ssh user#1.1.1.1 "echo $password | sudo -S /home/user/remoteScript.sh > remoteLog.txt"
And in my testing, execution of local script on remote machine into local log file works like below
sshpass -f password.txt ssh user#1.1.1.1 "bash -s" < /home/user/localScript.sh >> localLog.txt
How could I combine the elements of the two code examples above in order to make a local script run on a remote machine with root privilege and log the output into a local text file?
Some things I have tried that do not work include:
sshpass -f password.txt ssh user#1.1.1.1 "bash -s" < "echo $password | sudo -S /home/user/script.sh >> log.txt"
sshpass -f password.txt ssh user#1.1.1.1 "echo $password | sudo -S /home/user/script.sh" >> log.txt
and notably
sshpass -f password.txt ssh user#1.1.1.1 echo $password | sudo -S /home/user/script.sh >> log.txt
which just executes the local script with root privilege on the local machine.
I have tried many variations of the above commands and I believe its some sort of piping or flow issue but I cannot figure it out. Is there anyway to do this?
Machines are Ubuntu 16.04 and you cannot ssh in already as root.
Thanks in advance
A) It might be worth looking into an orchestration/config management solution (e.g. ansible). It's a steep learning curve at first, but initial outlay will pay off on spades down the line if you're managing multiple servers.
B) Setup password-less sudo for the scripts you want to execute, so you don't have to pass around the password in plaintext, and can run without any input. In sudoers:
user ALL=(ALL) NOPASSWD:/home/user/script.sh
C) Setup an SSH key, so you don't need to use a password at all.
But in nutshell, the code you're looking for is something like:
cat /home/user/localScript.sh | ssh user#1.1.1.1 "sudo bash" > log.txt
Which executes a non-interactive bash shell as root on the remote machine, which will take commands to execute on standard in, and the standard output will come back over the ssh channel for you to write to your local log.
Look into &> or 2>&1 if you want standard error too.
Is there any way I can combine the following commands into one command? I do not want to login in each time for each command.
sshpass -p 'somepwd' ssh user#server "mkdir -p /home/user/test"
sshpass -p 'somepwd' scp file.sh user#server:/home/user/test
sshpass -p 'somepwd' scp /test/somefile.txt user#server:/home/user/test
sshpass -p 'somepwd' ssh user#server -C "cd /home/user/test;./file.sh"
I did check the answer for combing multiple commands when using ssh and scp; Based on that I would still need 3 logins, one for first ssh and mkdir, one for scp and one for ssh and running the shell script.
Is there a better solution?
Use public/private keys instead of password authentication. Not only will this simplify the use of ssh, it is much more secure, especially after you disallow password authentication on the server you are connecting to. Using password authentication means you will get hacked, or your server has already been compromised and you don't know it yet. The rest of this answer assumes you have set up public/private keys.
I see you have files in /test. Don't put your work in the root directory, this invites security issues. Instead, work in your home directory unless you are experienced with setting up permissions properly.
Because file.sh is in your current directory (whatever that is) and you want a file from /test/ you cannot use rsync. rsync would be a good choice if all your files lived in the same directory.
Here is what we are left with; I have not messed with the location of /test/ because I don't know enough about the task:
ssh user#server "mkdir -p /home/user/test"
scp file.sh user#server:/home/user/test
scp /test/somefile.txt user#server:/home/user/test
ssh user#server -C "cd /home/user/test;./file.sh"
With GNU tar and ssh:
tar -c file.sh test/somefile.txt | sshpass -p 'somepwd' ssh user#server -C "tar -C / --transform 's|test/||;s|^|/home/user/test/|' --show-transformed-names -xv; cd /home/user/test; ./file.sh"
For more secure methods to pass the password with sshpass, see man sshpass.
I can copy file via SSH by using SCP like this:
cd /root/dir1/dir2/
scp filename root#192.168.0.19:$PWD/
But if on remote server some directories are absent, in example remote server has only /root/ and havn't dir1 and dir2, then I can't do it and I get an error.
How can I do this - to copy file with creating directories which absent via SSH, and how to make it the easiest way?
The easiest way mean that I can get current path only by $PWD, i.e. script must be light moveable without any changes.
This command will do it:
rsync -ahHv --rsync-path="mkdir -p $PWD && rsync" filename -e "ssh -v" root#192.168.0.19:"$PWD/"
I can make the same directories on the remote servers and copy file to it via SSH by using SCP like this:
cd /root/dir1/dir2/
ssh -n root#192.168.0.19 "mkdir -p '$PWD'"
scp -p filename root#192.168.0.19:$PWD/
I have a bash script I'm using to connect to a remote server via ssh. That works, however I want the script to immediately pass the command of cd /some/dir after connecting. That doesn't seem to be working. Here's my code:
#!/bin/bash
echo "SSHing.."
ssh -i ~/.ssh/some-site.pem xxx#yyy.com
cd /some/dir
read
How can I have the cd command be executed right after SSH connection is established?
There are two easy ways to execute commands via SSH from inside the script:
1) ssh user#host 'command'
2)
ssh user#host <<<EOF
command1
command2
<...>
commandn
EOF
Normally you'd just edit your ~/.profile on the remote machine.
If that is not an option, you could do something like this:
ssh -t theserver.com 'cd /some/dir && bash -i'
You can use the following command
ssh user#watevr <the_cmd_to_be_executed>
You can try this :
ssh abc#hostname :/pathto/specific directory
I know it is not recommended, but is it at all possible to pass the user's password to scp?
I'd like to copy a file via scp as part of a batch job and the receiving server does, of course, need a password and, no, I cannot easily change that to key-based authentication.
Use sshpass:
sshpass -p "password" scp -r user#example.com:/some/remote/path /some/local/path
or so the password does not show in the bash history
sshpass -f "/path/to/passwordfile" scp -r user#example.com:/some/remote/path /some/local/path
The above copies contents of path from the remote host to your local.
Install :
ubuntu/debian
apt install sshpass
centos/fedora
yum install sshpass
mac w/ macports
port install sshpass
mac w/ brew
brew install https://raw.githubusercontent.com/kadwanev/bigboybrew/master/Library/Formula/sshpass.rb
just generate a ssh key like:
ssh-keygen -t rsa -C "your_email#youremail.com"
copy the content of ~/.ssh/id_rsa.pub
and lastly add it to the remote machines ~/.ssh/authorized_keys
make sure remote machine have the permissions 0700 for ~./ssh folder and 0600 for ~/.ssh/authorized_keys
If you are connecting to the server from Windows, the Putty version of scp ("pscp") lets you pass the password with the -pw parameter.
This is mentioned in the documentation here.
curl can be used as a alternative to scp to copy a file and it supports a password on the commandline.
curl --insecure --user username:password -T /path/to/sourcefile sftp://desthost/path/
You can script it with a tool like expect (there are handy bindings too, like Pexpect for Python).
You can use the 'expect' script on unix/terminal
For example create 'test.exp' :
#!/usr/bin/expect
spawn scp /usr/bin/file.txt root#<ServerLocation>:/home
set pass "Your_Password"
expect {
password: {send "$pass\r"; exp_continue}
}
run the script
expect test.exp
I hope that helps.
You may use ssh-copy-id to add ssh key:
$which ssh-copy-id #check whether it exists
If exists:
ssh-copy-id "user#remote-system"
Here is an example of how you do it with expect tool:
sub copyover {
$scp = Expect->spawn("/usr/bin/scp ${srcpath}/$file $who:${destpath}/$file");
$scp->expect(30,"ssword: ") || die "Never got password prompt from $dest:$!\n";
print $scp 'password' . "\n";
$scp->expect(30,"-re",'$\s') || die "Never got prompt from parent system:$!\n";
$scp->soft_close();
return;
}
Nobody mentioned it, but Putty scp (pscp) has a -pw option for password.
Documentation can be found here: https://the.earth.li/~sgtatham/putty/0.67/htmldoc/Chapter5.html#pscp
Once you set up ssh-keygen as explained above, you can do
scp -i ~/.ssh/id_rsa /local/path/to/file remote#ip.com:/path/in/remote/server/
If you want to lessen typing each time, you can modify your .bash_profile file and put
alias remote_scp='scp -i ~/.ssh/id_rsa /local/path/to/file remote#ip.com:/path/in/remote/server/
Then from your terminal do source ~/.bash_profile. Afterwards if you type remote_scp in your terminal it should run the scp command without password.
Here's a poor man's Linux/Python/Expect-like example based on this blog post: Upgrading simple shells to fully interactive
TTYs. I needed this for old machines where I can't install Expect or add modules to Python.
Code:
(
echo 'scp jmudd#mysite.com:./install.sh .'
sleep 5
echo 'scp-passwd'
sleep 5
echo 'exit'
) |
python -c 'import pty; pty.spawn("/usr/bin/bash")'
Output:
scp jmudd#mysite.com:install.sh .
bash-4.2$ scp jmudd#mysite.com:install.sh .
Password:
install.sh 100% 15KB 236.2KB/s 00:00
bash-4.2$ exit
exit
Make sure password authentication is enabled on the target server. If it runs Ubuntu, then open /etc/ssh/sshd_config on the server, find lines PasswordAuthentication=no and comment all them out (put # at the start of the line), save the file and run sudo systemctl restart ssh to apply the configuration. If there is no such line then you're done.
Add -o PreferredAuthentications="password" to your scp command, e.g.:
scp -o PreferredAuthentications="password" /path/to/file user#server:/destination/directory
make sure you have "expect" tool before, if not, do it
# apt-get install expect
create the a script file with following content. (# vi /root/scriptfile)
spawn scp /path_from/file_name user_name_here#to_host_name:/path_to
expect "password:"
send put_password_here\n;
interact
execute the script file with "expect" tool
# expect /root/scriptfile
copy files from one server to other server ( on scripts)
Install putty on ubuntu or other Linux machines. putty comes with pscp. we can copy files with pscp.
apt-get update
apt-get install putty
echo n | pscp -pw "Password#1234" -r user_name#source_server_IP:/copy_file_path/files /path_to_copy/files
For more options see pscp help.
Using SCP non interactively from Windows:
Install the community Edition of netcmdlets
Import Module
Use Send-PowerShellServerFile -AuthMode password -User MyUser -Password not-secure -Server YourServer -LocalFile C:\downloads\test.txt -RemoteFile C:\temp\test.txt for sending File with non-interactive password
In case if you observe a strict host key check error then use -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null options.
The complete example is as follows
sshpass -p "password" scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root#domain-name.com:/tmp/from/psoutput /tmp/to/psoutput
You can use below steps. This works for me!
Step1-
create a normal file suppose "fileWithScpPassword" which contains the ssh password for the destination server.
Step2- use sshpaas -f followed by password file name and then normal scp command.
sshpass -f "fileWithScpPassword" scp /filePathToUpload user#ip:/destinationPath/
One easy way I do this:
Use the same scp cmd as you use with ssh keys i.e
scp -C -i <path_to opens sshkey> <'local file_path'> user#<ip_address_VM>: <'remote file_path’>
for transferring file from local to remote
but instead of providing the correct <path_to_opensshkey>, use some garbage path. Due to wrong key path you will be asked for password instead and you can simply pass the password now to get the work done!
An alternative would be add the public half of the user's key to the authorized-keys file on the target system. On the system you are initiating the transfer from, you can run an ssh-agent daemon and add the private half of the key to the agent. The batch job can then be configured to use the agent to get the private key, rather than prompting for the key's password.
This should be do-able on either a UNIX/Linux system or on Windows platform using pageant and pscp.
All the solutions mentioned above can work only if you the app installed or you should have the admin rights to install except or sshpass.
I found this very useful link to simply start the scp in Background.
$ nohup scp file_to_copy user#server:/path/to/copy/the/file > nohup.out 2>&1
https://charmyin.github.io/scp/2014/10/07/run-scp-in-background/
I found this really helpful answer here.
rsync -r -v --progress -e ssh user#remote-system:/address/to/remote/file /home/user/
Not only you can pass there the password, but also it will show the progress bar when copying. Really awesome.