Rsync / Linux : prevent new directory being created inside destination directory - linux

Pardon my novice-ness, please.
A question regarding the usage of Rsync.
If I specify, as an example, this command:
rsync -av Downloads Downloads2
the result is that the directory "Downloads2" is created with "Downloads" as a subdirectory, and, within /Downloads2/Downloads are all of the contents of the original /Downloads directory.
I am wondering if there is an option which will cause rsync to copy the contents of /Downloads to /Downloads2 without creating the /Downloads2/Downloads structure?

As you can see from "man rsync", adding a trailing slash on the source directory will prevent that directory being created:
rsync -avz foo:src/bar/ /data/tmp
A trailing slash on the source changes this behavior to avoid creating
an additional directory level at the destination. You can think of a
trailing / on a source as meaning "copy the contents of this directory"
as opposed to "copy the directory by name", but in both cases the
attributes of the containing directory are transferred to the contain‐
ing directory on the destination. In other words, each of the follow‐
ing commands copies the files in the same way, including their setting
of the attributes of /dest/foo:
rsync -av /src/foo /dest
rsync -av /src/foo/ /dest/foo

Related

rsync only certain types of files

I know there has been a huge discussion about this but I have not found something this specific.
Im trying to copy all .key files in /home// directory
This does not work
/usr/bin/rsync -auPA --include="*/*.key" --exclude="*" /home/* /tmp/test
This works but it copies over unwanted empty directories like /home/uname/Documents
/usr/bin/rsync -auPA --include="*/" --include="*.key" --exclude="*" /home /tmp/test
Basically what i need for rsync to do is to copy only files with .key extension and only create necessarily folders that contain .key files
I think you are looking for the -m option. From the man page:
-m, --prune-empty-dirs
This option tells the receiving rsync to get rid of empty directories from the file-list, including nested directories that
have no non-directory children. This is useful for avoiding the creation of a bunch of useless directories when the sending
rsync is recursively scanning a hierarchy of files using include/exclude/filter rules.
Note that the use of transfer rules, such as the --min-size option, does not affect what goes into the file list, and thus
does not leave directories empty, even if none of the files in a directory match the transfer rule.
Because the file-list is actually being pruned, this option also affects what directories get deleted when a delete is active.
However, keep in mind that excluded files and directories can prevent existing items from being deleted due to an exclude both
hiding source files and protecting destination files. See the perishable filter-rule option for how to avoid this.
You can prevent the pruning of certain empty directories from the file-list by using a global "protect" filter. For instance,
this option would ensure that the directory "emptydir" was kept in the file-list:
--filter ’protect emptydir/’
Here’s an example that copies all .pdf files in a hierarchy, only creating the necessary destination directories to hold the
.pdf files, and ensures that any superfluous files and directories in the destination are removed (note the hide filter of
non-directories being used instead of an exclude):
rsync -avm --del --include=’*.pdf’ -f ’hide,! */’ src/ dest
If you didn’t want to remove superfluous destination files, the more time-honored options of "--include='*/' --exclude='*'"
would work fine in place of the hide-filter (if that is more natural to you).

rsync, ignore root synchronised directory hidden files

I'm trying to make snapshot backups of my user using rsync, the base user folder has a ton of hidden files and configurations that I'm not interested in backing up, however I am interested in the hidden folders inside of it's subdirectories.
Here is the layout
Source Directory
/Base
.ignore
.ignore2
.ignore3
/dir1
.keep
normalfiles
/dir2
.keep
normalfiles
Desired Backup Directory
/Backup
/dir1
.keep
normalfiles
/dir2
.keep
normalfiles
How can I ignore the first hidden files AND directories while preserving them in the subdirectories.
Solution
rsync -vrn --exclude="/.*" base/ base2/
By specifying the / before the file match /.* I managed to achieve what I was after. I have found this to be pretty useful when making snapshot backups of some mac's in my house. They tend to store tons of config information in the root folder for each user that isn't needed for my backups.
The hidden files for my own project configs and rc's is saved though since they aren't stored in the root directory.
I can not found "hidden folders inside of it's subdirectories" in your Code sample. Do you mean hidden files?
Here is a try:
rsync -nrv --include="/*/" --include="*/.*" --exclude="*" Base/ Base2/
-n simulate
-r recursive
-v verbose

Shell script to delete files and folders that don't exist in another folder

Using a shell script I wish to delete all files and folders from /folder2/ that do not exist in /folder1/. Files only need to be matched by name.
I must add that the content of both folders shouldn't necessarily match after this operation because it's possible that /folder1/ contains files that do not in exist in /folder2/. So after executing the shell script all files and folders found in /folder2/ can also be found in /folder1/ but not vice versa.
The following works for me:
rsync -r --delete --existing --ignore-existing /path/to/folder1/ /path/to/folder2/
rsync will delete all files and folders from folder2 that are not found in folder1 recursively. Also, rsync will skip creating files on the destination. This answer was found here: https://serverfault.com/a/713577

Slashes and the rsync command

I am trying to do something along the same lines as what is asked in this question: How do I synchronize in both directions?
However, what I can't figure out is whether or not I should add slashes to the end of the file path. Basically I'm trying to create an alias command that syncs the contents of two directories that go by the same name but are on two different servers. What I don't want is for one directory to be copied into the other (which I am aware is a possibility depending on how the slashes at the end are done).
What I have currently is:
alias syncDirectories1 = 'rsync -tvur name#host:/Users/me/directory/ /Users/me/directory/'
alias syncDirectories2 = 'rsync -tvur /Users/me/directory/ name#host:/Users/me/directory/'
For what I am trying to accomplish, should there be slashes at the end of both file paths?
It's described in the rsync(1) manpage:
A trailing slash on the source changes this behavior to avoid creating an additional directory level at the destination. You can think of a
trailing / on a source as meaning "copy the contents of this directory" as opposed to "copy the directory by name", but in both cases the
attributes of the containing directory are transferred to the containing directory on the destination. In other words, each of the following commands copies the files in the same way, including their setting of the attributes of /dest/foo:
rsync -av /src/foo /dest
rsync -av /src/foo/ /dest/foo
As to the destination, I don't think it has any major consequences. There is a difference if the source is a file and destination doesn't exist — this will make a copy of SRC called DEST:
rsync SRC DEST
, whereas, this will create directory DEST and copy the SRC file into it:
rsync SRC DEST/
I tested it with rsync 3.1.3 on Arch Linux, the results are below:
1. rsync -avPzu test login#remote:/home/login/test "test" directory is copied inside of existing "test" on remote (structure is then test/test/...)
2. rsync -avPzu test login#remote:/home/login/test/ same as above
3. rsync -avPzu test/ login#remote:/home/login/test content of "test" directory is synchronized with the remote "test" directory
4. rsync -avPzu test/ login#remote:/home/login/test/ same as above
5. rsync -avPzu test login#remote:/home/login/ same as above
6. rsync -avPzu test login#remote:/home/login same as above
The methods 3-6 are the correct ones in this case, contrary to the accepted answer.
keep the slashes on the source, and remove them from the destination.
Like this:
alias syncDirectories1 = 'rsync -tvur name#host:/Users/me/directory/ /Users/me/directory'
alias syncDirectories2 = 'rsync -tvur /Users/me/directory/ name#host:/Users/me/directory'
I had different results from accepted answer on Ubuntu 17.04. It seems that the destination / does not have an effect.
I did the following four commands:
rsync -av src nslash # No slashes on either src or dest
rsync -av src dslash/ # Slash on Dest only
rsync -av src/ sslash # Slash on src only
rsync -av src/ sdslash/ # Slash on both src and dest
Conclusions:
once there is a / on the src, then the contents of the src only will be copied over to the dest, regardless of whether the dest has a slash or not.
no slash on the src, then the src and its content will be copied to dest.
Slashes on dest has no effect.

Wget - output directory prefix

Currently I try to use:
"wget --user=xxx --password=xxx -r ftp://www.domain.com/htdocs/"
But this saves output files to current directory in this fashion:
curdir/www.domain.com/htdocs/*
I need it to be:
curdir/*
Is there a way to do this, I only see a way to use output prefix, but i think this will just allow me to define directory outside current dir?
You can combine --no-directories if you want all your files inside one directory or --no-host-directories to have subdirectories but no subdirectories per host with your --directory-prefix option.
2.6 Directory Options
‘-nd’
‘--no-directories’
Do not create a hierarchy of directories when retrieving recursively. With this option turned on, all files will get saved to the current directory, without clobbering (if a name shows up more than once, the filenames will get extensions ‘.n’).
‘-nH’
‘--no-host-directories’
Disable generation of host-prefixed directories. By default, invoking Wget with ‘-r http://fly.srk.fer.hr/’ will create a structure of directories beginning with fly.srk.fer.hr/. This option disables such behavior.
‘-P prefix’
‘--directory-prefix=prefix’
Set directory prefix to prefix. The directory prefix is the directory where all other files and subdirectories will be saved to, i.e. the top of the retrieval tree. The default is ‘.’ (the current directory).
(From the wget manual.)

Resources