Similar to this question I want to make a task scheduler script to copy NEW files (last 24h) to a new folder.
I try to use this code:
find /volume1/start/ -mtime -1 -type f -exec cp -r {} /volume1/target/ \;
but it delivers a 1kb filename.pdf#SynoEAStream file instead of the file itself.
How can I fix that?
Ok actually it seems to work as supposed I actually just had the wrong file modification date in the files where it didn't work. The script additionally copies some "useless" #SynoEAStream files whicht I now avoid by only looking for pdf-files which ist what I wanted.
find /volume1/TestScan/start/ -mtime -1 -type f -iname '*.pdf' -exec rsync -r {} /volume1/TestScan/ziel/ \;
Maybe its helpful for someone
Related
I have some automated downloads in a proprietary linux distro.
They go to a temp scratch disk. I want to move them when they're finished to the main RAID array. The best way I can see to do this is to check the folders on the disk to see if the contents have changed in the last minute. If not then its probably finished downloading and then move it.
Assuming there could be hundreds of folders or just one in this location and its all going to the same place. Whats the best way to write this?
I can get a list of folder sizes with
du -h directory/name
The folders can contain multiple files anywhere from 1.5mb to 10GB
Temp Loc: /volume2/4TBScratch/Processing
Dest Loc when complete: /volume1/S/00 Landing
EDIT:
Using this:
find /volume2/4TBScratch/Processing -mindepth 1 -type d -not -mmin +10 -exec mv "{}" "/volume1/S/00 Landing" \;
find: `/volume2/4TBScratch/Processing/test': No such file or directory
4.3#
yet it DOES copy the relevant folders and all files. But the error worries me that something might go wrong in the future.... is it because there is multiple files and it's running the same move command for EACH file or folder in the root folder? But since it moves it all on the first iteration it cant find it on the next ones?
EDIT2:
Using Rsync
4.3# find /volume2/4TBScratch/Processing -mindepth 1 -type d -not -mmin +10 -exec rsync --remove-source-files "{}" "/volume1/S/00 Landing" \;
skipping directory newtest
skipping directory erw
RESOLVED: EDIT3
Resolved with the help in the comments below. Final script looks like this:
find /volume2/4TBScratch/Processing -mindepth 1 -type d -not -mmin +10 -exec rsync -a --remove-source-files "{}" "/volume1/S/00 Landing" \;
find /volume2/4TBScratch/Processing -depth -type d -empty -delete
rsync to move folders and files but leaves empty root dir
the next command finds empty folders and removes them.
Thanks all!
You can use GNU find with options -size for detecting files/folders of certain size and use mv with the -exec option to move to destination directory. The syntax is
find /volume2/4TBScratch/Processing -type d -maxdepth 1 -size -10G -exec mv "{}" "/volume1/S/00 Landing" \;
Using rsync
find /volume2/4TBScratch/Processing -type d -maxdepth 1 -size -10G -exec rsync --remove-source-files "{}" "/volume1/S/00 Landing" \;
The size with a - sign to indicate less than the mentioned size which in this case is 10GB. A note on each of the flags used
-type d -> For identifying only the folders from the source path.
-maxdepth 1 -> To look only on the current source directory and not
being recursive.
-exec -> Execute command following it.
Alternatively, if you want to find files that are last modified over a certain time(minutes), find has an option for -mmin which can be set to a value. E.g. -mmin -5 would return files modified five minutes ago.
So suggest adding it to your requirement, for x as you need and see if the directories are listed, then you can add the -exec option for moving the directories
find /volume2/4TBScratch/Processing -type d -maxdepth 1 -mmin -2 -size -10G
Refer to the GNU documentation for finding files according to size on how this works.
Note:- The double quotes("") are added to avoid Bash from splitting the names containing spaces.
I have a folder that contains so many files/folders inside it that even basic commands like du and find are crashing. I'd like to clean up some old files from it... but obviously I can't do that with the find command...
# find /opt/graphite/storage/whisper -mtime +30 -type f -delete
Aborted (core dumped)
What command or trick can I use to delete files from that folder since find isn't working?
I believe the best way to go is using a simple for-loop: the problem is that find loads all found information in memory, and only once this is done, it starts deleting.
However, a loop can solve this:
for f in $(ls -a)
do
if <check_last_modification_date>($f)
then rm -r $f
fi
done
Concerning the last modification date check, there are plenty of ways to do this, as explained here.
For find command using the -exec option worked for me to delete the files.
# find /opt/graphite/storage/whisper -mtime +30 -type f -exec rm -f {} \;
Will you be able to help me to write a script, I just want to find log files over 2GB and copy them to archive folder in same directory.I just write a find command it is not working, appreciate if someone could help me.
ex - main log folders - /vsapp/logs/
- app1,app2,app3
there are lot of logs in the app1, app2 and app3 folders.
so i want to find the logs in the logs folder which is over 2GB, and copy them to archive folder with the different name with today's date.
ex - abcd.log -----copy to -----> abcd.log-08-22-2016
My command at the moment which is not working
find $i/* -type f -size +2G -exec cp '{}' $i/$arc/{}-$date
You can do:
find /src -type f -name '*.log' -size +2G -exec cp {} /dest/{}-$(date -I) \;
Additions/Modifications i made:
-name '*.log' searches only for log files, as we are only interested in those. You can look for files with any names too if unsure, just omit -name '*.log in that case
$(date -I) is command substitution the output will be today's date in format YYYY-mm-dd, you can also define a custom format, check man date
End the -exec action of find with \;
I want to copy the recently updated multiple file into another directory.
I am having 1.xml,2.xml,3.xml.... in this directory recently someone updated file or added new file into the directory,So i want to copy those files into the destination directory ..Its like synchronization of 2 directories.
For that I have tried below commend
find home/deployment/server/services/ -type f -mtime 1 | xargs cp /home/application/
and below one also
find home/deployment/server/services/ -type f -mtime 1 -exec cp /home/application/
I am not getting any file into destination after updating 1.xml file,So I have added new file 4.xml even that also not updating in destination directory.
How to process recently updated or newly added multiple files.
Thanks in advance.
Short answer:
use xargs to mv the "find" directory into another directory
Long answer: As I recall (not tested) for exec syntax is
find . -type f --mtime 1 -exec cp {} /destination/path/ +
"{}" is an argument which came from command "find"
For xargs
find . -type f --mtime 1 | xargs -0 -I {} cp {} /destination/path/
I do this often but use \; instead of + and usually -cnewer rather than -mtime.
\; executes the cp command on files individually instead of as a group.
+ executes as a group with as many paths as xterm will take. It may do this multiple time if there are a lot of files.
the \ in front of the ; option is required or bash will think it is the end of the command.
find ./ -mtime -1 -exec cp {} /path/ \; -print
Use the -print at the end to get a list of the files that were copied.
In linux shell, When I run
ls -al -t
that show the time of files.
How to cp/rm files by time? just like copy all the files that created today or yesterday. Thanks a lot.
Depending on what you actually want to do, find provides -[acm]time options for finding files by accessed, created or modified dates, along with -newer and -min. You can combine them with -exec to copy, delete, or whatever you want to do. For example:
find -maxdepth 1 -mtime +1 -type f -exec cp '{}' backup \;
Will copy all the regular files in the current directory more than 1 day old to the directory backup (assuming the directory backup exists).
Simple Example
find /path/to/folder/ -mtime 1 -exec rm {} \; // Deletes all Files modified yesterday
For more examples google for bash find time or take a look here