I would like to be able to find the sub directories in a folder
Then located the .txt files in each sub folder
Then CAT those files in each sub folder into a new .txt file within that folder.
This is what I am using now but it does not work:
find . -type d -exec find .{} -type f -iname .txt \; -exec cat {} >> all.txt \;
any help would be most appreciated.
Thanks
B
Using a Bash for-Loop, this should work:
for sub_dir in $(find ./ -type d -not -name '.'); do
cat $sub_dir/*.txt > $sub_dir/all.txt;
done
Use all the subdirs found by find (excluding the . directory) and issue a cat on each one of them.
find . -name "*.txt" -type f -exec cat {} + > all.txt
Related
Imagine a folder structure like this:
/411/folder1/
/411/archive/
/211/archive/
/211/folder1/
/211/folder2/
In each subfolder there will be files and more subfolders with files/folders. I'm interested in to zip all the files in the subfolders called 'archive' and ignore all other folders in the structure.
If I use the command: find * -type d -name 'archive'
The output will be like:
928/archive
973/archive
990/archive
What I'm interested in is to have an output like:
928/archive/file1.jpg
928/archive/file2.jpg
928/archive/folder1/file3.jpg
And so on so that I can use the commander: find * -type d -name 'archive' [with some more/other options] | zip all_archive_files.zip -#
How can this be done?
you can match on the whole path using -path or regex: for example
find . -regex './[0-9]+/archive/.*' -type f -exec zip all.zip {} \;
This is crude, but you could:
find . -type d -name 'archive' -exec find {} \; | zip stuff.zip -#
The normal way to run zip takes filenames from the command line and with the -r flag the command will recurse into directories by itself. Consider
find . -name 'archive' -type d -exec zip -r all_archive_files.zip {} +
The -exec option will run zip -r all_archive_files.zip ... where ... is replaced by a list of all the files that find found. Run it with echo between -exec and zip to see what it will do:
$ find . -name 'archive' -type d
411/archive
412/archive
488/archive
512/archive
$ find . -name 'archive' -type d -exec echo zip -r all_archive_files.zip {} +
zip -r all_archive_files.zip 411/archive 412/archive 488/archive 512/archive
I'm new to unix shell/bash scripting . My requirement is as follows :
The current directory contains a lot of dynamic folders and the data file is available only in the last sub folder.
I need to move the data file to the home folder and rename the datafile's name as the current directory's name.
Could you please help in writing the bash script for the same.
--update--
I tried the following to move file to the parent directory:
find . -mindepth 2 -type f -print -exec mv {} . \;
After trying out many options , the following worked
find . -mindepth 2 -type f -print -exec mv {} . \;
dirFullPath=`pwd`
fileName=`echo $dirFullPath | awk -F"/" '{print $(NF)}'`
mv *.0 $fileName.tab
Any other better solutions is appreciated, Thanks.!!
I have the following directories:
P922_101
P922_102
.
.
Each directory, for instance P922_101 has following subdirectories:
140311_AH8MHGADXX 140401_AH8CU4ADXX
Each subdirectory, for instance 140311_AH8MHGADXX has the following files:
1_140311_AH8MH_P922_101_1.fastq.gz 1_140311_AH8MH_P922_101_2.fastq.gz
2_140311_AH8MH_P922_101_1.fastq.gz 2_140311_AH8MH_P922_101_2.fastq.gz
And files in 140401_AH8CU4ADXX are:
1_140401_AH8CU_P922_101_1.fastq.gz 1_140401_AH8CU_P922_4001_2.fastq.gz
2_140401_AH8CU_P922_101_1.fastq.gz 2_140401_AH8CU_P922_4001_2.fastq.gz
I want to do 'cat' for the files in the subdirectories in the following way:
cat 1_140311_AH8MH_P922_101_1.fastq.gz 2_140311_AH8MH_P922_101_1.fastq.gz
1_140401_AH8CU_P922_101_1.fastq.gz 2_140401_AH8CU_P922_101_1.fastq.gz > P922_101_1.fastq.gz
which means that files ending with _1.fastq.gz should be concatenated into a single file and files ending with _2.fatsq.gz into another file.
It should be run for all files in subdirectories in all directories. Could someone give a linux solution to do this?
Since they're compressed, you should probably use gzip -dc (decompress and write to stdout) -
find /somePath -type f -name "*.fastq.gz" -exec gzip -dc {} \; | \
tee -a /someOutFolder/out.txt
You can use find for this:
find /top/path -mindepth 2 -type f -name "*_1.fastq.gz" -exec cat {} \; > one_file
find /top/path -mindepth 2 -type f -name "*_2.fastq.gz" -exec cat {} \; > another_file
This will look for all the files starting from /top/path and having a name matching the pattern _1.fastq.gz / _2.fastq.gz and cat them into the desired file. -mindepth 2 makes find look for files that are at least under the current directory; this way, files in /top/path won't be matched.
Note that you will probably need zcat instead of cat, for gz files.
As you keep adding details in comments, let's see what else we can do:
Say you have the list of directories in a file directories_list, each line containing one:
while read directory
do
find $directory -mindepth 2 -type f -name "*_1.fastq.gz" -exec cat {} \; > $directory/output
done < directories_list
I have a directory with unknown number of subdirectories and unknown level of sub*directories within them. How do I copy all the file swith the same suffix to a new directory?
E.g. from this directory:
> some-dir
>> foo-subdir
>>> bar-sudsubdir
>>>> file-adx.txt
>> foobar-subdir
>>> file-kiv.txt
Move all the *.txt files to:
> new-dir
>> file-adx.txt
>> file-kiv.txt
One option is to use find:
find some-dir -type f -name "*.txt" -exec cp \{\} new-dir \;
find some-dir -type f -name "*.txt" would find *.txt files in the directory some-dir. The -exec option builds a command line (e.g. cp file new.txt) for every matching file denoted by {}.
Use find with xargs as shown below:
find some-dir -type f -name "*.txt" -print0 | xargs -0 cp --target-directory=new-dir
For a large number of files, this xargs version is more efficient than using find some-dir -type f -name "*.txt" -exec cp {} new-dir \; because xargs will pass multiple files at a time to cp, instead of calling cp once per file. So there will be fewer fork/exec calls with the xargs version.
So Basically I need to find all files and folders in my github project containing the string 'persons'
find . -type f -print | grep "persons"
find . -type d -print | grep "persons"
The above works for me.
But I also need to rename all the above files and folders with 'members'
Can I do the above with a couple of commands? Instead of manually replacing them one by one
i dont know how to do a git mv oldfilename newfilename rescursively to the above
for dir in `find /DIR -type d -iname '*persons*'` ; do
git mv "${dir}" "${dir/persons/members}"
done
Will do. For the files do it with -type f.
find . -depth -name persons | while read F; do mv $F $(dirname $F)/members; done