Run rsync via cron on Mac OS Catalina - cron

There are many posts about this on the web but I have not found a working solution for my issue.
I'm on Mac OS Catalina. I have written a script to rsync a remote directory, via ssh, to my local computer:
#!/bin/bash
/usr/bin/rsync -arvzhe --delete 'ssh -i /Users/myusername/.ssh/id_rsa' user#XX.XXX.XX.XX:/home/storage/app/public /Users/myusername/public/
But it fails with the error:
Permission denied (publickey).
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/rsync/rsync-54/rsync/io.c(453) [receiver=2.6.9]
The rsync call works fine when calling manually from the Terminal.
I do have a private key file in /Users/myusername/.ssh/id_rsa
Thanks.

You have to give your terminal app full disk access. (It will still have to obey Unix-level permissions after you do, but right now you are bumping into another layer of security that Mac OS has added.)
To fix, go to: System Preferences => Security and Privacy => Privacy, select "Full Disk Access" and check the "Terminal" box on the right.

Related

I mass executed a permission command in ssh and now I am locked out

I am working in an aws ssh terminal. I kept getting permission denied to write to a lot of files and it was bugging me. I tried to execute this command in the root directory, on vscode remote:
chmod -R o+rw .
It ran the command for a lot of files in the log. Then after a bit it stopped responding. I tried logging back in, but it gave me this error:
kex_exchange_identification: Connection closed by remote host
Connection closed by (host ip address) port 22
one of my remote coworkers seems to have access, but me and my in house coworker (working off of the same user) cannot connect. We have tried connection to other users on the terminal but the response.
First of all sorry that you got locked out, Make sure your friend doesn't log out of the machine.
From the error message
kex_exchange_identification: Connection closed by remote host
It happens when private keys (files) are readable by all, i.e. have the wrong permissions.
which you did via command
chmod -R o+rw .
so below files in /etc/ssh/ usually needs to have 600 permissions
ssh_host_key
ssh_host_dsa_key
ssh_host_rsa_key
In addition to this, there might be a problem with your authorized_keys file as well
chmod 700 /root/.ssh
chmod 600 /root/.ssh/authorized_keys
restorecon -R -v /root/.ssh
In this question adding-a-public-key-to-ssh-authorized-keys-does-not-log-me-in-automatically there are whole bunch of options you can try.
If nothing works, launch another EC2 instance and mount this problematic instance as another volume on the newly launched machine, at least you'll be able to get your code/data.

NT_STATUS_ACCESS_DENIED Error in Samba share

So, I tried to set up a public SMB share with Samba on CentOS 7. Now, I have it set up, and I have a headache. But, sweet victory. I'm posting this here for all y'all so that you don't need to waste your time. It's actually easy, you just need to know the hoops you need to jump through. I'll also edit the Samba wiki.
The first problem was that it wouldn't connect at all, except locally:
Remote Connection (my Linux desktop):
-------
[root#my-desktop ~]# smbclient //sambaserver/PublicDocs -N
Error connecting to 192.168.100.97 (No route to host)
Connection to cgybkp01 failed (Error NT_STATUS_HOST_UNREACHABLE)
On Windows 8, using Windows Explorer, after typing "\\sambaserver" into the address bar, the progress bar would wait, wait, wait, then time out. The error message was:
Remote Connection (my Windows 8 desktop):
Windows cannot access \\sambaserver
Check the spelling of the name. Otherwise, there might be a problem with your network. To
try to identify and resolve network problems, click Diagnose.
This ended up being a problem with firewalld. To unblock Samba, I needed to add this line to /etc/firewalld/zones/public.xml :
<service name="samba"/>
Perfect, now I can connect!
But, I was actually mounting an NFS share, so I had one more issue, with SELinux. Now, when I attempt to connect with smbclient...
smbclient //sambaserver/PublicDocs -N
I can connect, but when I try to ls, I get the error: "NT_STATUS_ACCESS_DENIED" in CentOS 7. So, how do I connect?
The first thing everyone recommended that I try was file permissions. If you're not familiar with file permissions in Linux, I'd recommend trying those first. But for me, that didn't work, because SELinux was blocking me.
To see all of the SELinux options for Samba, type:
getsebool -a | grep samba
getsebool -a | grep smb
The one I needed to change was samba_share_nfs, because I was sharing an NFS mounted directory:
setsebool -P samba_share_nfs on
CentOS maintains a list of these booleans here.

rsync from linux to windows without smb mount

I would like to use rsync to mirror a git repo from a linux machine to several Windows servers. What I'm seeing online so far are solutions that require you mount the Windows side using samba. ideally I would like to avoid that because it would require managing several mounts, not to mention security concerns. I've verified that I can at least ssh onto my test server. I have cwRsync installed on the windows server. I try the following command, but rsync doesn't seem to like the space in the path to the rsync executable.
rsync -rlH --rsync-path="c:/Program\ Files/cwRsync/bin/rsync.exe" . Administrator#cool03-stage:d:/rsync_test/.
'c:/Program\' is not recognized as an internal or external command,
operable program or batch file.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]
After adding C:\Program Files\cwRsync\bin to my PATH on the Windows server, I can run this:
rsync -rlH -e ssh . Administrator#cool03-stage:/cygdrive/d/rsync_test/.
Seems to work.

rsync ssh fails to connect

When trying to connect via rsync to download a file using the following command:
rsync -Pav http://some.domainname.com/file.tar.gz
The following error is given to me
ssh: Could not resolve hostname http: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(605) [Receiver=3.0.9]
If i try and download the same file using the ftp commands instead it works fine. If i try and use the rsync command from a pc on another network instead this also works fine, so it seems to be something environmental.
The customers client pc sits behind a proxy server, but all the proxy address details have been added to the system, and i'm able to browse, ftp, and ssh onto a test server without any issues. Is there anything that may be blocking rsync from running?
You are mixing curl/wget and rsync.
wget http://some.domainname.com/file.tar.gz
or
rsync a_user#some.domainename.com:/path/to/webdir/file.tar.gz /local/dest/dir

Establishing ssh connection from within RStudio on linux

I am trying to pull a file from another computer into R environment in RStudio on Centos 6
I've tried it in plain R first and when I issue
readLines(pipe( 'ssh root#X.X.X.X "cat /path/somefile.sh"' ))
it correctly asks me for the password of my ssh key and reads the contents.
However if the same command is executed from RStudio all I get is:
ssh_askpass: exec(rpostback-askpass): No such file or directory
Permission denied, please try again.
ssh_askpass: exec(rpostback-askpass): No such file or directory
Permission denied, please try again.
ssh_askpass: exec(rpostback-askpass): No such file or dire
Permission denied (publickey,gssapi-with-mic,password).
I suspect that the reason is because rstudio on centos actually uses rstudio-server user (and gui is provided in a browser). Does anyone know how to properly access ssh'd resources from it ?
UPD: after executing
Sys.setenv(PATH = paste0(Sys.getenv('PATH'), ':/usr/lib/rstudio-server/bin/postback'))
as suggested below it won't output askpass errors, but it still does not work. Now it seems that the console is waiting for the command to execute indefinitely
rpostback-askpass is part of RStudio. It may help to add its location (/usr/lib/rstudio-server/bin/postback on my system) to PATH so that ssh can find it:
Sys.setenv(PATH = paste0(Sys.getenv('PATH'), ':/usr/lib/rstudio-server/bin/postback'))
UPDATE RCurl has scp function for copying files over ssh connection. See this answer for details. If you are running your scripts with RStudio, you can use its API to enter the ssh password interactively with hidden input:
pass <- .rs.askForPassword("password?")
and rstudioapi can help to determine whether the script is launched by RStudio or not.

Resources