How to rsync all folders present on the destination folder - linux

I have two folders FolderA and FolderB as below. I want to rsync the common subfolders. For example, I can do rsync -avzP /path/to/FolderB/* /path/to/FolderA/, which will keep SubFolder1 and SubFolder3 mirrored. My question is how I can achieve the same if FolderB is the destination without explicitly --include or --exclude individual subfolders (e.g., in case there are too many of them).
FolderA
|--SubFolder1
|--SubFolder2
|--SubFolder3
|--SubFolder4
FolderB
|--SubFolder1
|--SubFolder3

You can update all files in FolderA using the contents FolderB as the source with your normal:
rsync -uav /path/to/FolderB/ /path/to/FolderA
(note: the trailing '/' after FolderB/ is mandatory to copy the contents of FolderB rather than FolderB itself)
To do it in reverse and update FolderB from FolderA and not copy SubFolder2 and SubFolder4 with the --existing option which will "skip creating new files on receiver", but that will also prevent new files and directories within SubFolder1 and SubFolder3 from being created as well.
You best option to not copy SubFolder2 and SubFolder4 while allowing new files and directories within SubFolder1 and SubFolder3 be created is to use the --filter option. See rsync(1) - Linux manual page.
A typical way to use --filter to exclude SubFolder2 and SubFolder4 on a copy from FolderA to FolderB would be:
rsync -uav --filter -_SubFolder2/ --filter -_SubFolder4/ /path/to/FolderA/ /path/to/FolderB
That will allow you to copy the complete contents of /path/to/FolderA/ to /path/to/FolderB/ without including SubFolder2 and SubFolder4.
Edit Per-Comment On Large Number of SubFolders
If you have a large number of folders under FolderA that you do not want to sync under FolderB, then your other option is to create a text file holding the absolute path to only those SubFolderX under FolderA you want to rsync to FolderB and then use the --no-R and --files-from=folderlist options to only rsync the wanted SubFolders. This will eliminate having to specify a large number of --filter options on the command line.
For example, you can create your folderlist with:
find /path/to/FolderA -maxdepth 1 -type d > folderlist
(note: specify the absolute path above and find will produce the folderlist file containing absolute paths)
Now edit your folderlist file and remove the parent directory (e.g. /path/to/FolderA) and any SubFolders you don't want to sync under FolderB. You can now use the folderlist file to control which SubFolders under FolderA are sync'ed to FolderB without having to include a long list of filters on the command line. Your command line then becomes
rsync -uai -r --no-R --files-from=folderlist / /path/to/FolderB
(note: the '/' as source serves as a base for the paths contained in folderlist. You can change the -i option to control the level of information dumped to the screen, e.g. -v, etc... or remove it altogether to suppress any reporting other than errors)
(also note: when using --files-from, -a does not imply -r (recursive), so you will need to explicitly add -r if you need a recursive transfer)

Related

rsync, ignore root synchronised directory hidden files

I'm trying to make snapshot backups of my user using rsync, the base user folder has a ton of hidden files and configurations that I'm not interested in backing up, however I am interested in the hidden folders inside of it's subdirectories.
Here is the layout
Source Directory
/Base
.ignore
.ignore2
.ignore3
/dir1
.keep
normalfiles
/dir2
.keep
normalfiles
Desired Backup Directory
/Backup
/dir1
.keep
normalfiles
/dir2
.keep
normalfiles
How can I ignore the first hidden files AND directories while preserving them in the subdirectories.
Solution
rsync -vrn --exclude="/.*" base/ base2/
By specifying the / before the file match /.* I managed to achieve what I was after. I have found this to be pretty useful when making snapshot backups of some mac's in my house. They tend to store tons of config information in the root folder for each user that isn't needed for my backups.
The hidden files for my own project configs and rc's is saved though since they aren't stored in the root directory.
I can not found "hidden folders inside of it's subdirectories" in your Code sample. Do you mean hidden files?
Here is a try:
rsync -nrv --include="/*/" --include="*/.*" --exclude="*" Base/ Base2/
-n simulate
-r recursive
-v verbose

How do I copy differing content files from one directory to another?

There exists two directories: a/ and b/.
I'd like to copy all the files(recursively) from a/ into b/.
However, I only want to copy over an a file if its content is different than the already existing b file. If the corresponding b file does not exist, then you would still copy over the a file.
*by "corresponding file", I mean a files with the same name and relative path from their parent directories.
note:
The reason I don't want to overwrite a b file with the same exact contents, is because the b directory is being monitored by another program, and I don't want the file date to change causing the program to do more work than required.
I'm essentially looking for a way to perform a cp -rf a/ b/ while performing a diff check on each file. If the file's are different, perform the copy; otherwise skip the copy.
I see that cp has an update flag:
-u, --update
copy only when the SOURCE file is newer than the destination file or when the
destination file is missing
but this will not work because I'm not concerned about newer files; I'm concerned about different file contents.
Any shell language will do.
I've been attempting to get this to work by injecting my diff check into a find command:
find a/ ??? -exec cp {} b \;
This doesn't seem like an uncommon thing to do between two directories, so I'm hoping there is an elegant command line solution as aposed to me having to write a python script.
You can achieve this using rsync. Files or directories will be updated only if there is any new update in source folder.
$rsync -av --progress sourcefolder destinationfolder

How to create tar archive without some folders?

How I can create tar archive without some folders?
Now I'm creating tar archive of folder and deleting some folders from it. But it is takes long time.
Structure:
www
- sub f 1
- sub f 2
- sub f 3
need create archive only with folders (sub f 1) and (sub f 2)
(It is only for example, my really structure have more sub levels)
Thanks!
Does this works ?
tar -cf backup.tar --exclude "www/subf3" www
// is your directory with spaces?
In addition of the --exclude option suggested by Ajreal, you could use some other tar option (from the --help output):
--exclude=PATTERN exclude files, given as a PATTERN
--exclude-backups exclude backup and lock files
--exclude-caches exclude contents of directories containing
CACHEDIR.TAG, except for the tag file itself
--exclude-caches-all exclude directories containing CACHEDIR.TAG
--exclude-caches-under exclude everything under directories containing
CACHEDIR.TAG
--exclude-tag=FILE exclude contents of directories containing FILE,
except for FILE itself
--exclude-tag-all=FILE exclude directories containing FILE
--exclude-tag-under=FILE exclude everything under directories
containing FILE
--exclude-vcs exclude version control system directories
-X, --exclude-from=FILE exclude patterns listed in FILE
and you could also use tardy, a tar post-processor (which is packaged in Debian)

Copy files excluding some folder in linux

I want to create script that copy my project and make it zip archive. I want to exclude all folder named .svn in all sub directories. Any suggestion?
I'd use rsync's FILTER RULES for this:
Create an .rsync-filter file (in the origin directory) containing, e.g.
-.svn/
Now run rsync like an exalted copy:
rsync -aFF origin/ destination/
You can do this using rsync. Although this is designed to synchronise directories across servers, it can also be used to copy directories on a single machine.
rsync has a --exclude option to exclude files and directories by pattern. See http://www.samba.org/ftp/rsync/rsync.html for help and examples.
Just call the zip utility on your project’s folder and use the -r option for recursive plus the -x option to exclude files / folders by pattern.
zip -r target-filename.zip source-folder -x \*exclude-pattern\*
exclude-pattern in your case would be .svn
See also man zip

rsync to copy only particular folders

I have a dir with files and dirs in it. What I want is to make rsync copy of several folders from that dir. For example, lets say I have this:
/home/user
-- drwxr-xr-x folderA
-- drwxr-xr-x folderB
-- drwxr-xr-x folderC
-- -rw-r--r-- file.1
-- -rw-r--r-- file.2
I want to copy folderA and folerB using rsync. I have created file rsync_folders.txt
+ /folderA/**
+ /folderB/**
My problem is that rsync will always copy file unless it matches exclude pattern. But if I add
- /**
nothing is copied because rsync first matches against exclude patterns.
Any ideas?
Note: I cannot list all folders and files I want to exclude. It will be changing from time to time.
Either use rsync -av src1 src2 src3 ... dst or put all the folders you want to rsync in a text file (each folder in a separate line) and use rsync -arv --files-from=sources.txt dst.
Note that by default -a implies --recursive but not when --files-from is used, so in this case -r must be specified explicitly.

Resources