I'm trying to find projects in an enormous directory. The projects are always several levels of depth in and have a config file which contains the project name. So basically...
Given a path and string
Return any directory that has a depth of 3 from the and contains a file named "config"
that contains the
I learned that find combined with grep will work... but print out the grepped text and not the path of it's parent directory
find <starting-dir> -maxdepth 3 -mindepth 3 -type d -exec grep '<project-name>' {}/config \;
Just prints out the project name :(
Perhaps there any way to switch back to find's default behaviour of printing out the found file path only if the grep is successful? Or is there another tool I should try to use to solve this?
To get -print, you need to add it explicitly after a succesful -exec.
For example, using grep's -q:
find <starting-dir> \
-maxdepth 3 -mindepth 3 \
-type d \
-exec grep -q '<project-name>' {}/config \; \
-print
As you discovered, grep already has -l.
You can reduce the number of grep processes:
find <starting-dir> \
-maxdepth 4 -mindepth 4 \
-type f -name config \
-exec grep -l '<project-name>' {} +
Adding the -l flag to my output fixes the issue, for some reason I thought that would just print out "config" and not the whole path of that config file, but here we are.
find <starting-dir> -maxdepth 3 -mindepth 3 -type d -exec grep -l '<project-name>' {}/config \;
This will print out the full path of the config file of the project you search for.
Related
find /home/imk-prac/ \( -type f -size -13c -name '*\?plik\?*' \) -o\( -type d -name '\[Kolo1\]*' \)2> /dev/nul;
This command counts normal files which has less than 13 symbols and contains a sequence of symbols ?plik?.
I want to add looking for files which were modified less than 30 days and I wrote this command:
find /home/imk-prac/ -type f -mtime -30 -exec ls -l {} \; > /dev/null
I don't know how to combine this two commands in to one.
I wanted to add looking for files with specified quantity of symbols and I found this command:
grep -Po '(^|\s)\S{64}(\s|$)' file
But there is the same problem or even worse, because of grep command.
Thanks for your time and I hope you will help me to figure it out ;)
I have the following directories:
P922_101
P922_102
.
.
Each directory, for instance P922_101 has following subdirectories:
140311_AH8MHGADXX 140401_AH8CU4ADXX
Each subdirectory, for instance 140311_AH8MHGADXX has the following files:
1_140311_AH8MH_P922_101_1.fastq.gz 1_140311_AH8MH_P922_101_2.fastq.gz
2_140311_AH8MH_P922_101_1.fastq.gz 2_140311_AH8MH_P922_101_2.fastq.gz
And files in 140401_AH8CU4ADXX are:
1_140401_AH8CU_P922_101_1.fastq.gz 1_140401_AH8CU_P922_4001_2.fastq.gz
2_140401_AH8CU_P922_101_1.fastq.gz 2_140401_AH8CU_P922_4001_2.fastq.gz
I want to do 'cat' for the files in the subdirectories in the following way:
cat 1_140311_AH8MH_P922_101_1.fastq.gz 2_140311_AH8MH_P922_101_1.fastq.gz
1_140401_AH8CU_P922_101_1.fastq.gz 2_140401_AH8CU_P922_101_1.fastq.gz > P922_101_1.fastq.gz
which means that files ending with _1.fastq.gz should be concatenated into a single file and files ending with _2.fatsq.gz into another file.
It should be run for all files in subdirectories in all directories. Could someone give a linux solution to do this?
Since they're compressed, you should probably use gzip -dc (decompress and write to stdout) -
find /somePath -type f -name "*.fastq.gz" -exec gzip -dc {} \; | \
tee -a /someOutFolder/out.txt
You can use find for this:
find /top/path -mindepth 2 -type f -name "*_1.fastq.gz" -exec cat {} \; > one_file
find /top/path -mindepth 2 -type f -name "*_2.fastq.gz" -exec cat {} \; > another_file
This will look for all the files starting from /top/path and having a name matching the pattern _1.fastq.gz / _2.fastq.gz and cat them into the desired file. -mindepth 2 makes find look for files that are at least under the current directory; this way, files in /top/path won't be matched.
Note that you will probably need zcat instead of cat, for gz files.
As you keep adding details in comments, let's see what else we can do:
Say you have the list of directories in a file directories_list, each line containing one:
while read directory
do
find $directory -mindepth 2 -type f -name "*_1.fastq.gz" -exec cat {} \; > $directory/output
done < directories_list
In a Linux terminal, how to delete all files from a folder except one or two?
For example.
I have 100 image files in a directory and one .txt file.
I want to delete all files except that .txt file.
From within the directory, list the files, filter out all not containing 'file-to-keep', and remove all files left on the list.
ls | grep -v 'file-to-keep' | xargs rm
To avoid issues with spaces in filenames (remember to never use spaces in filenames), use find and -0 option.
find 'path' -maxdepth 1 -not -name 'file-to-keep' -print0 | xargs -0 rm
Or mixing both, use grep option -z to manage the -print0 names from find
In general, using an inverted pattern search with grep should do the job. As you didn't define any pattern, I'd just give you a general code example:
ls -1 | grep -v 'name_of_file_to_keep.txt' | xargs rm -f
The ls -1 lists one file per line, so that grep can search line by line. grep -v is the inverted flag. So any pattern matched will NOT be deleted.
For multiple files, you may use egrep:
ls -1 | grep -E -v 'not_file1.txt|not_file2.txt' | xargs rm -f
Update after question was updated:
I assume you are willing to delete all files except files in the current folder that do not end with .txt. So this should work too:
find . -maxdepth 1 -type f -not -name "*.txt" -exec rm -f {} \;
find supports a -delete option so you do not need to -exec. You can also pass multiple sets of -not -name somefile -not -name otherfile
user#host$ ls
1.txt 2.txt 3.txt 4.txt 5.txt 6.txt 7.txt 8.txt josh.pdf keepme
user#host$ find . -maxdepth 1 -type f -not -name keepme -not -name 8.txt -delete
user#host$ ls
8.txt keepme
Use the not modifier to remove file(s) or pattern(s) you don't want to delete, you can modify the 1 passed to -maxdepth to specify how many sub directories deep you want to delete files from
find . -maxdepth 1 -not -name "*.txt" -exec rm -f {} \;
You can also do:
find -maxdepth 1 \! -name "*.txt" -exec rm -f {} \;
In bash, you can use:
$ shopt -s extglob # Enable extended pattern matching features
$ rm !(*.txt) # Delete all files except .txt files
Having trouble listing the contents of a folder I'm not in, while excluding the actual folder name itself.
ex:
root#vps [~]# find ~/test -type d
/root/test
/root/test/test1
However I want it to only display /test1, as the example.
Thoughts?
There's nothing wrong with a simple
find ~/test -mindepth 1
Similarly, this will have the same effect:
find ~/test/*
as it matches everything contained within ~/test/ but not ~/test itself.
As an aside, you'll almost certainly find that find will complain about the -mindepth n option being after any other switches, as ordering is normally important but the -(min|max)depth n switches affect overall behaviour.
You can do that with -exec and basename:
find ~/test -type d -exec basename {} \;
Explanation:
The find ~/test -type d part finds all directories recursively under ~/test, as you already know.
The -exec basename {} \; part runs the basename command on {}, which is where all the results from the last step are substituted into.
Then you need -type f instead of -type d.
Or, if you want to display list of folders, excluding the parent -mindepth 1 (find ~/test -type d -mindepth 1).
And now that you edited it, I think what you want may be
find ~/test -type d -mindepth 1 |cut -d/ -f3-
But I think you need to be more specific ;-)
I just fixed it with sed
find $BASE -type d \( ! -iname "." \)|sed s/$BASE//g
Where $BASE is initial foldername.
This question already has answers here:
how to find files containing a string using egrep
(7 answers)
Closed 8 years ago.
I am trying to search All .PHP files or ALL .SH files for any reference that contains:
'into tbl_free_minutes_mar'
I have command line access to the server but the files may be scattered in different directories.
For all directories everywhere,
find / -type f \( -name '*.php' -o -name '*.sh' \) \
-exec fgrep 'into tbl_free_minutes_mar' {} \+
For fewer directories elsewhere, just give a list of paths instead of /. To just list the matching files, try fgrep -l. If your file names might not always match the wildcards in the -name conditions, maybe scan all files.
find / -type f \( -name \*.php -o -name \*.sh \) -exec grep 'into tbl_free_minutes_mar' {} /dev/null \;
Change find / ... to to something less all-encompassing if you know the general area that you want to look in, e.g. find /home ...
Provided /base/path is the path where you want to start looking this will get you a list of files:
find /base/path -type f -iregex '.*\.\(php\|sh\)$' -exec grep -l 'into tbl_free_minutes_mar' '{}' \;