I have two questions with respect to rsync:
1: I have a bunch of files which are incremented by day of the year. Ex: file.txt.81, file.txt.82, etc. Now, these files are in different directories:
data1/file.txt.81
data1/file.txt.82
data2/file2.txt.81
data2/file2.txt.82
How can I have rsync get only the *.82 files and not even touch the other files
2: Now I have a similar data directory structure as above. How can I rsync all files that have been modified on or after a specific day?
Thanks
Here is the answer for #1rsync -avz --include "**/" --include=*.82 --exclude=* /path/from /path/to
This will recursively (-a) include the directories and search them for anything matching .82 and exclude everthing else. You can find more info on this in man rsync and look for "exclude patterns"
For #2 I would find some way to do it with find and mtime. To find files modified in past 60 minutes with the name *.82 this should work:
sudo find /path/from -mmin 60 -type f -name *.82
EDITED: too many backticks
Related
The current find command is utilized to find and delete outdated files and directories. The expired data is based on a given properties file and the destination.
If the properties file says…
"/home/some/working/directory;.;180"
…then we want files and empty subdirectories deleted after 180 days.
The original command is…
"find ${var[0]} -mtime +${var[2]} -delete &"
…but I now need to modify now that we've discovered is has deleted symbolic links that existed in specified sub-directories after the given expiration date in the properties file. The variable path and variable expiration time are designated in the properties file (as previously demonstrated).
I have been testing using…
"find -L"
…to follow the symbolic links to make sure this clean up command reaches the destinations, as desired.
I have also been testing using…
"\! -type l"
…to ignore deleting symbolic links, so the command I've been trying is…
"find -L ${var[0]} ! -type l -mtime +${var[2]} -delete &"
…but I haven't achieved the desired results. Help please, I am still fresh into Linux and my research hasn't lead me to a desired answer. Thank you for your time.
Change
\! -type l
to
\! -xtype l
find -L ${var[0]} \\! -xtype l -mtime +${var[2]} -delete &
I'm trying to delete all files in a folder structure (recursively) except the youngest one for each month.
In other words.... only keep the first ones from each month in each folder.
On a Linux system (bash) ... ;-) (or even more precise on a Synology NAS)
May thanks for your help !
Alex
Please be careful! I take no responsibility!
Try find:
Remove files which are older than 7 days:
find . -type f -ctime +7 -delete
lets say I have a my_dirs/ directory, insdie that directory I have several parallel subdirectories which has several files and I want to delete all of them except the ones that have the substring '.regions'
this is my parent directory content:
this is what I tried:
shopt -s extglob
rm -r !(./**/*.regions*)
but I got an error message: cannot be deleted «! (./**/*. region *) »: The file or directory does not exist.
how can I do that?
First of all, always be careful when deleting multiple files.
The command to achieve what you want would be:
find my_dirs -type f ! -name "*.regions*" -delete
"-delete" must be last, otherwise it will delete everything it finds
This will explore all subdirectories in my_dirs, find the files (-type f) that not (!) contain ".regions" ("*.regions*") on their name, and delete (-delete) them.
I recommend running this first: find my_dirs -type f ! -name "*.regions*",
so it won't delete anything and you can check the files are correct.
Edit: Added -type f so it only targets files per Philippe's suggestion.
I'm trying to write a script that copy all the files of one dir (with subdirs) to the root of another dir.
So Imagine I have this file structure:
/
pic.JPG
PIC5.JPG
FOLDER
pic2.JPG
pic3.JPG
FOLDER2
pic4.JPG
I want all the .JPG files from that directory and copy them over to another destination. But I don't want the directory structure, just the files.
This is what I've got:
"sudo rsync -aq --include '*/' --include '*.JPG' --exclude '*\' /source/picturesRoot/ /destination/flatView/
But it also copies the directories :(
I found this link on stackoverflow:
rsync : Recursively sync all files while ignoring the directory structure
I looked at the solution and didn't see much difference with my command, apart from the * and . in the path. I tried it but it didn't work.
I hope somebody can help me, thanks.
This answer cannot work for you because your pictures are not at the same level in directories. There is no option in rsync to skip the creation of directory structure. In the link you gave, it's working because the user explicitly select source files with *.
You can try something with find and rsync. Find will find files and rsync copy them.
Here is a solution :
find /source/picturesRoot -type f -name "*.JPG" -exec rsync -a {} /destination/flatView/ \;
Be careful, if two files have the same name just one will be in destination directory.
i'm trying to elaborate a command that will find files that haven't been modified in over 6 months and zip them with one command. Afterwards i want to delete all those files and i just archived.
My current command to find the directories with the files is
find /var/www -type d -mtime -400 ! -mtime -180 | xargs ls -l > testd.txt
This gave me all the directories including the files that are older than 6 months
Now i was wondering if there was a way of zipping all the results and deleting them afterwards. Something amongst the line of
find /var/www -type f -mtime -400 ! -mtime -180 | gzip -c archive.gz
If anyone knows the proper syntax to achieve this i'd love to know. Thakns!
Edit, after a few tests this command results in a corrupted file
find /var/www -mtime -900 ! -mtime -180 | xargs tar -cf test4.tar
Any ideas?
Break this into several distinct steps that you can implement and thoroughly test separately:
Build a list of files to be archived and then deleted, saved to a temp file
Use the list from step 1 to add the files to .tar.gz archives. Give the archive file a name following a specific pattern that won't appear in the files to be archived, and put it in a directory outside the hierarchy of files being archived.
Read back the files from the .tar.gz and compare them (or their hashes) to the original files to ENSURE that you got them all without corruption
Use the list from step 1 to delete the files. Do not use a wildcard for deletion. Put in some guard code to prevent deletion of any file matching the name pattern of the archive .tar.gz file(s) created in step 2.
When testing a script that can do irreversible damage, always code the dangerous command with a leading echo and leave it that way until you are sure everything works. Only then remove the echo.
Consider zip, it should meet your requirements.
find ... | zip -m# archive.zip
-m (move) deletes the input directories/files after making the specified zip archive.
-# takes the list of input files from standard input.
You may find more options which are useful to you in the zip manual, e. g.
-r (recurse) travels the directory structure recursively.
-sf (show-files) shows the files that would be operated on, then exits.
-t or --from-date operates on files not modified prior to the specified date.
-tt or --before-date operates on files not modified after or at the specified date.
This could possibly make findexpendable.
zip -mr --from-date 2012-09-05 --before-date 2013-04-13 archive /var/www