I am struggling to get a simple cronjob running sftpclone working.
Details:
I can access the SFTP server via SSH key, i.e.:
# sftp username#my.backup.server.com
results in immediate access to the command line of the server.
I can also perform a successful backup via the command:
# sftpclone /var/mytest/ username#my.backup.server.com:test
But the same command doesn't seem to work if included in a cronjob. I tried adding a simple cronjob (i.e. with the command as in the example above), and I kept adding more details, such as the explicit user (root) and the location of the key to use. The final job is:
*/10 * * * * root /usr/local/bin/sftpclone /var/mytest/ username#my.backup.server.com:test -k /root/.ssh/id_rsa
And it still doesn't work. I am unsure as to how to troubleshoot the problem. Other cronjobs work fine, and the same command works perfectly in the command line:
/usr/local/bin/sftpclone /var/mytest/ username#my.backup.server.com:test -k /root/.ssh/id_rsa
Any help would be appreciated.
Related
I was having the same issue as this person and this person, where my command worked locally but inside of a cron task, nothing happened.
Here is what I am trying to run:
/usr/bin/mysqldump -u root -ppassword database_name > ~/Documents/dump.sql
This works locally when I type I straight into my terminal! It might throw an error saying:
mysqldump: [Warning] Using a password on the command line interface can be insecure
But it still it works. It even works if I put in the user root before the command.
Now, here is the CRON (just trying to run it every minute to test if it works)
*/1 * * * * root /usr/bin/mysqldump -u root -ppassword database_name > ~/Documents/dump.sql
And... nothing happens. What I saw from this question and [this person] was that it was a matter of syntax. I cannot figure out WHY this CRON will not run!
You should try using $HOME rather than ~ as tilde expansion is not reliable in a cron job.
See this as an example.
Here is more information about tilde expansion.
After much research, I couldn't find a solution but post this question.
I have a computer A and B both Ubuntu desktop. I want to copy file from A to B. Steps I followed.
1. ssh-keygen in computer A
2. Left password blank
3. Copied id_rsa.pub to computer B ~/.ssh/ from computer A
4. Renamed id_rsa.pub to authorized_keys in computer B
5. In computer A I did scp -i ~/.ssh/id_rsa -r /var/www/abc abc#ip:/home/abc/
If I do step 4 in commandline its working fine. But when I did same in crontab
22 10 * * * root scp -i ~/.ssh/id_rsa -r /var/www/abc abc#ip:/home/abc
Its doing nothing.
I have tried virtually every answer found related to the problem. The answer just came accidentally.
I typed username instead of root and it worked. I don't know how but it worked. Hope this will help people like me.
2 10 * * * root /usr/bin/scp -i /home/username/.ssh/id_rsa -r /var/www/abc abc#ip:/home/abc
2 10 * * * username /usr/bin/scp -i /home/username/.ssh/id_rsa -r /var/www/abc abc#ip:/home/abc
This is my solution. Made in Raspberry with Jessie OS.
Fix connection with the server with public key with no passphrase. You can find tutorials everywhere.
The thing is do it as the same user that shall create the crontab.
In my case I made the keys as PI (user on my Raspberry). Make sure you can login without password on your server.
Then I created my script that uploads all txt-files in a directory to the server every 5th minute.
ex:
"#!/bin/bash
scp /mnt/www/hus/*.txt xxxxxx.se#ssh.xxxxx.se:/www/images/hustemp"
Save it as xxxxxxx.sh in your home dir and make it executable (chmod +x xxxxxxx.sh).
Then itś time to create the cronjob. I think you have to be in your home dir.Just run crontab -e (no sudo in front)and edit to what you want. In my case:
*/5 * * * * /home/pi/upload.sh
It works perfect!
Good Luck
Anders
Why don't you try putting the the scp command in a bash script and put the bash script in the cron , also remember to put the shebang in your sh script as follows : #! /bin/bash (generally the path , confirm this by typing which bash in your shell). Also chmod a+x your sh script to make it executable and try the sh script from bash as ./script.sh and then put it in the crontab.
Why did the scp command not work in crontab?
The following post does a good job explaining the different kinds of problems one faces with cron jobs -
https://askubuntu.com/questions/23009/reasons-why-crontab-does-not-work
In your case it's an environment problem. Crontab's environment is different from that of bash's.
Hope this helps.
In crontab, you have a mere execution of the command line without all the goodies of an interactive shell, that is a populated PATH variable, and all other bash tricks such as ~ interpretation (unsure for that last one).
So the rule is always use full paths in crontab:
22 10 * * * root /usr/bin/scp -i /home/username/.ssh/id_rsa -r /var/www/abc abc#ip:/home/abc
For those struggling with this issue, all the answers above didn't solve my problem. Actually, you have to do the scp once with the root account in the terminal, so that you get this message:
The authenticity of host 'XXXX (123.123.123.123)' can't be established.
ECDSA key fingerprint is SHA256:XXXXXXXXXXXxxxxXXXxxXXXxxXXxXxxXXX.
Are you sure you want to continue connecting (yes/no/[fingerprint])?
Then you type "yes" and your next crons will work like a charm.
Create a shell script entering the scp command in the root
Make the script executable
Put the script in crontab
PATH=/usr/bin
32 18 * * * cd /root/ ; (time ./infra.sh)
Step 5 doesn't work,maybe Step 3 and 4 doesn't work well.
3. Copied id_rsa.pub to computer B ~/.ssh/ from computer A
4. Renamed id_rsa.pub to authorized_keys in computer B
You should use the command "ssh-copy-id" to copy .pub file.
I set the following cron tab when I'm under root user.
* * * * * wget -q --spider http://mytargeturl.com/my/url/
The codes are under the same server but owned by another user (and I couldn't set a crontab with that user). I have to request the page with wget because of MVC link system complexity.
When I run:
crontab -l -u root
I can see this crontab setting.
Why would be the reason that crontab doesn't work?
Thanks.
Your syntax looks fine and this should work. Check /var/log/cron to make sure that it is indeed running, and if so, consider logging the command's output to a file and then inspect the file to pinpoint where the problem may be.
I'm defining something like this in my crontab:
* * * * * ssh -tt otherhost whoami
And I'm getting the following output:
tcgetattr: Invalid argument
me
Running ssh with fewer -ttoptions leads to other errors besides tcgetattr.
The solution posted in why is the `tcgetattr` error seen when ssh is used for dumping the backup file on another server? doesn't really work well because in this case I'm using several ssh connections to run monitoring scripts on different hosts and I need to capture output sent to stderr and email it.
Any ideas on how to workaround this?
You could use something like this:
ssh -tt otherhost "your_monitoring_script 2>&1" 2> /dev/null
That way the errors from ssh go in the bucket, but the errors from your script are shown in stdout. For that to work you should mark errors from your script as "ERROR:" so that you can find them back if your script provides lots of output.
I have a web server (odin) and a backup server (jofur). On jofur, I can run the following code to rsync my web directories (via key authentication) from odin to jofur:
rsync -avz -e ssh backups#odin.mydomain.net:/home/backups /home/myuser/odin
If I enter this into the command line, everything rsyncs perfectly:
myuser#jofur:~$ rsync -avz -e ssh backups#odin.mydomain.net:/home/backups /home/myuser/odin
receiving incremental file list
sent 23 bytes received 1921 bytes 1296.00 bytes/sec
total size is 349557271 speedup is 179813.41
I want this to run every morning, so I edited my crontab to read this:
0 4 * * * rsync -avz -e ssh backups#odin.mydomain.net:/home/backups /home/myuser/odin
This doesn't work. The following message is deposited in /var/mail/myuser:
Could not create directory '/home/myuser/.ssh'.
Host key verification failed.
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(605) [Receiver=3.0.9]
I'm not sure what this error means. I'm wary of futzing blindly with permissions because I don't want to leave any backdoors open. Any suggestions?
Its hard to tell whether cron is using the wrong rsync binary or whether rsync requires some variable which is not being set in cron. Please set the stdout/stderr as shown below and pass on the output of the log file
Also, try doing a "which rsync" from the command line ; this will tell you which rsync you are using from the command line.
0 4 * * * rsync -avz -e ssh backups#odin.mydomain.net:/home/backups /home/myuser/odin > /tmp/cron_output.log 2>&1
EDIT :
Can you create a shell script called SOME_DIR/cron_job_rsync.sh which contains the following. Make sure you set the execute bit.
#!/bin/sh
/usr/sbin/rsync -avz -e ssh backups#odin.mydomain.net:/home/backups /home/myuser/odin
And modify the cronjob as shown below
0 4 * * * SOME_DIR/cron_job_rsync.sh >/tmp/cron_output.log 2>&1
I had a similar issue. Mine was the HOME directory was encrypted.
If your user is logged, it works the known_hosts.
But when it's a cron, the cron uses the right user BUT it does not have access to your $HOME/~/.ssh directory because is encrypted :-(
i got the same error just like you.
I finally found user home directory is an 'mount point', when logged in, it changed.
You can use the shell command 'mount' to check if you have the same way to use home directory.
So, i logged in and 'cd /', then do
```
cp -ar ${HOME}/.ssh /tmp/
sudo umount ${HOME}
mv /tmp/.ssh ${HOME}
```
There is may failed, because you need to check the ${HOME} if you have the right to write, if not, try sudo or add writable to ${HOME}.
After that, every thing being fine.
Please follow the below steps to avoid the error
http://umasarath52.blogspot.in/2013/09/solved-rsync-not-executing-via-cron.html
I resolved this issue by communicating with the administrators for my server. Here is what they told me:
For advanced security and performance, we use 1H (Hive) which
utilizes a chrooted environment for users. Libraries and binaries
should be copied to the chrooted environment to make them accessible.
They sent me a follow up email telling me that the "Relevent" packages have been installed. At that point, the problem was resolved. Unfortunately, I didn't get any additional information from them. The host was Arvixe, but I'm guessing that anyone using 1H (Hive) will encounter a similar problem. Hopefully this answer will be helpful.
Use the rrsync script together with a dedicated ssh key as follows:
REMOTE server
mkdir ~/bin
gunzip /usr/share/doc/rsync/scripts/rrsync.gz -c > ~/bin/rrsync
chmod +x ~/bin/rrsync
LOCAL computer
ssh-keygen -f ~/.ssh/id_remote_backup -C "Automated remote backup" #NO passphrase
scp ~/.ssh/id_remote_backup.pub devel#10.10.10.83:/home/devel/.ssh
REMOTE computer
cat id_remote_backup.pub >> authorized_keys
Prepend to the newly added line the following
command="$HOME/bin/rrsync -ro ~/backups/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding
So that the result looks like
command="$HOME/bin/rrsync -ro ~/backups/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa AAA...vp Automated remote backup
LOCAL
Put in your crontab the following script with x permission:
#!/bin/sh
echo ""
echo ""
echo "CRON:" `date`
set -xv
rsync -e "ssh -i $HOME/.ssh/id_remote_backup" -avzP devel#10.10.10.83:/ /home/user/servidor
Source: http://www.guyrutenberg.com/2014/01/14/restricting-ssh-access-to-rsync/
I did several steps to make it work.
Check your paths. For every command you'll use check which [command] and use that full path for the crontab
Open crontab as the user you want to run it with so it has access to that users ssh-key
Add (remember to user which) ssh-agent && [your ssh-command] so it can connect over ssh.
When authentication still fails at this point. Try to generate a passwordless ssh-key. This way you can skip the password prompting.
For debugging it is useful to add -vvv to the ssh command in rsync. It makes it clear what goes wrong.
Using the correct keyring solved the issue for me. Add the following line to your crontab:
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
In total, your crontab (edited by calling crontab -e from your terminal) should now look as follows:
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
0 4 * * * rsync -avz backups#odin.mydomain.net:/home/backups /home/myuser/odin
Background: It turns out that some Linux distributions use a keyring to protect your public-private key pair - so the key pair is password-protected without ever noticing you. Consequently, rsync is not able to open your ssh key for authentication.
Note that I also omitted the -e ssh; I think it is not necessary here.
Further Troubleshooting: rsync does not provide a lot of debugging output. What helped me identify the problem was to put a dummy scp command, which is much more verbose, in my crontab. A crontab entry for troubleshooting may look something like:
* * * * * scp -v backups#odin.mydomain.net:/home/backups/dummy.txt /home/myuser/odin/dummy.txt >> /home/myuser/odin/dummy.txt.log 2>&1
The above command will run every minute (great for developing) and it will copy a file /home/backups/dummy.txt to your local machine. All logs (stdout and stderr) are written to to /home/myuser/odin/dummy.txt.log. Inspect these logs to see where the error precisely comes from.
Reference: The troubleshooting explained above lead me to the solution: https://unix.stackexchange.com/a/332353/395749