I want to copy all the log files from a directory which does not contain log files, but it contains other subdirectories with log files. These subdirectories also contain other subdirectories, so I need something recursive.
I tried
cp -R *.log /destination
But it doesn't work because the first directory does not contains log files. The response can be also a loop in bash.
find /path/to/logdir -type f -name "*.log" |xargs -I {} cp {} /path/to/destinationdir
Explanation:
find searches recursively
-type f tells you to search for files
-name specifies the name pattern
xargs executes commands
-I {} indicates an argument substitution symbol
Another version without xargs:
find /path/to/logdir -type f -name '* .log' -exec cp '{}' /path/to/destinationdir \;
Imagine a folder structure like this:
/411/folder1/
/411/archive/
/211/archive/
/211/folder1/
/211/folder2/
In each subfolder there will be files and more subfolders with files/folders. I'm interested in to zip all the files in the subfolders called 'archive' and ignore all other folders in the structure.
If I use the command: find * -type d -name 'archive'
The output will be like:
928/archive
973/archive
990/archive
What I'm interested in is to have an output like:
928/archive/file1.jpg
928/archive/file2.jpg
928/archive/folder1/file3.jpg
And so on so that I can use the commander: find * -type d -name 'archive' [with some more/other options] | zip all_archive_files.zip -#
How can this be done?
you can match on the whole path using -path or regex: for example
find . -regex './[0-9]+/archive/.*' -type f -exec zip all.zip {} \;
This is crude, but you could:
find . -type d -name 'archive' -exec find {} \; | zip stuff.zip -#
The normal way to run zip takes filenames from the command line and with the -r flag the command will recurse into directories by itself. Consider
find . -name 'archive' -type d -exec zip -r all_archive_files.zip {} +
The -exec option will run zip -r all_archive_files.zip ... where ... is replaced by a list of all the files that find found. Run it with echo between -exec and zip to see what it will do:
$ find . -name 'archive' -type d
411/archive
412/archive
488/archive
512/archive
$ find . -name 'archive' -type d -exec echo zip -r all_archive_files.zip {} +
zip -r all_archive_files.zip 411/archive 412/archive 488/archive 512/archive
Let's say I am in the directory /home/videos and want to iterate recursively through all of the directories underneath it. If the directory name contains "images" I want to delete the directory and all of its contents. Also, can this be done for files? Let's say in each directory go through every file and check if its name ends with ".mp3" and delete it?
Thanks
find . -name "*images*" -type d -exec rm -r {} \;
find . -name "*.mp3" -type f -exec rm -rf {} \;
-exec rm -rf {} \; : Delete all files matched by file pattern.
-type f : Only match files and do not include directory names.
-type d : matches only directory names
Here is a nice tutorial on this. http://www.cyberciti.biz/faq/linux-unix-how-to-find-and-remove-files/
So you can do something like
find /home/videos -type d -name "*images*" -exec rm -rf {} \;
and
find /home/videos -type f -name "*.mp3" -exec rm -rf {} \;
I have the following directories:
P922_101
P922_102
.
.
Each directory, for instance P922_101 has following subdirectories:
140311_AH8MHGADXX 140401_AH8CU4ADXX
Each subdirectory, for instance 140311_AH8MHGADXX has the following files:
1_140311_AH8MH_P922_101_1.fastq.gz 1_140311_AH8MH_P922_101_2.fastq.gz
2_140311_AH8MH_P922_101_1.fastq.gz 2_140311_AH8MH_P922_101_2.fastq.gz
And files in 140401_AH8CU4ADXX are:
1_140401_AH8CU_P922_101_1.fastq.gz 1_140401_AH8CU_P922_4001_2.fastq.gz
2_140401_AH8CU_P922_101_1.fastq.gz 2_140401_AH8CU_P922_4001_2.fastq.gz
I want to do 'cat' for the files in the subdirectories in the following way:
cat 1_140311_AH8MH_P922_101_1.fastq.gz 2_140311_AH8MH_P922_101_1.fastq.gz
1_140401_AH8CU_P922_101_1.fastq.gz 2_140401_AH8CU_P922_101_1.fastq.gz > P922_101_1.fastq.gz
which means that files ending with _1.fastq.gz should be concatenated into a single file and files ending with _2.fatsq.gz into another file.
It should be run for all files in subdirectories in all directories. Could someone give a linux solution to do this?
Since they're compressed, you should probably use gzip -dc (decompress and write to stdout) -
find /somePath -type f -name "*.fastq.gz" -exec gzip -dc {} \; | \
tee -a /someOutFolder/out.txt
You can use find for this:
find /top/path -mindepth 2 -type f -name "*_1.fastq.gz" -exec cat {} \; > one_file
find /top/path -mindepth 2 -type f -name "*_2.fastq.gz" -exec cat {} \; > another_file
This will look for all the files starting from /top/path and having a name matching the pattern _1.fastq.gz / _2.fastq.gz and cat them into the desired file. -mindepth 2 makes find look for files that are at least under the current directory; this way, files in /top/path won't be matched.
Note that you will probably need zcat instead of cat, for gz files.
As you keep adding details in comments, let's see what else we can do:
Say you have the list of directories in a file directories_list, each line containing one:
while read directory
do
find $directory -mindepth 2 -type f -name "*_1.fastq.gz" -exec cat {} \; > $directory/output
done < directories_list
In Linux, how do I remove folders with a certain name which are nested deep in a folder hierarchy?
The following paths are under a folder and I would like to remove all folders named a.
1/2/3/a
1/2/3/b
10/20/30/a
10/20/30/b
100/200/300/a
100/200/300/b
What Linux command should I use from the parent folder?
If the target directory is empty, use find, filter with only directories, filter by name, execute rmdir:
find . -type d -name a -exec rmdir {} \;
If you want to recursively delete its contents, replace -exec rmdir {} \; with -delete or -prune -exec rm -rf {} \;. Other answers include details about these versions, credit them too.
Use find for name "a" and execute rm to remove those named according to your wishes, as follows:
find . -name a -exec rm -rf {} \;
Test it first using ls to list:
find . -name a -exec ls {} \;
To ensure this only removes directories and not plain files, use the "-type d" arg (as suggested in the comments):
find . -name a -type d -exec rm -rf {} \;
The "{}" is a substitution for each file "a" found - the exec command is executed against each by substitution.
This also works - it will remove all the folders called "a" and their contents:
rm -rf `find . -type d -name a`
I ended up here looking to delete my node_modules folders before doing a backup of my work in progress using rsync. A key requirements is that the node_modules folder can be nested, so you need the -prune option.
First I ran this to visually verify the folders to be deleted:
find . -type d -name node_modules -prune
Then I ran this to delete them all:
find . -type d -name node_modules -prune -exec rm -rf {} \;
Thanks to pistache
To delete all directories with the name foo, run:
find -type d -name foo -a -prune -exec rm -rf {} \;
The other answers are missing an important thing: the -prune option. Without -prune, GNU find will delete the directory with the matching name and then try to recurse into it to find more directories that match. The -prune option tells it to not recurse into a directory that matched the conditions.
This command works for me. It does its work recursively
find . -name "node_modules" -type d -prune -exec rm -rf '{}' +
. - current folder
"node_modules" - folder name
find ./ -name "FOLDERNAME" | xargs rm -Rf
Should do the trick. WARNING, if you accidentally pump a . or / into xargs rm -Rf your entire computer will be deleted without an option to get it back, requiring an OS reinstall.
Combining multiple answers, here's a command that works on both Linux and MacOS
rm -rf $(find . -type d -name __pycache__)
I had more than 100 files like
log-12
log-123
log-34
....
above answers did not work for me
but the following command helped me.
find . -name "log-*" -exec rm -rf {} \;
i gave -type as . so it deletes both files and folders which starts with log-
and rm -rf deletes folders recursively even it has files.
if you want folders alone
find -type d -name "log-*" -exec rm -rf {} \;
files alone
find -type f -name "log-*" -exec rm -rf {} \;
Another one:
"-exec rm -rf {} \;" can be replaced by "-delete"
find -type d -name __pycache__ -delete # GNU find
find . -type d -name __pycache__ -delete # POSIX find (e.g. Mac OS X)
Earlier comments didn't work for me since I was looking for an expression within the folder name in some folder within the structure
The following works for a folder in a structure like:
b/d/ab/cd/file or c/d/e/f/a/f/file
To check before using rm-rf
find . -name *a* -type d -exec realpath {} \;
Removing folders including content recursively
find . -name *a* -type d -exec rm -rf {} \;
find path/to/the/folders -maxdepth 1 -name "my_*" -type d -delete