How can I find the directories where a text is found in a specific file? E.g. I want to get all the directories in "/var/www/" that contain the text "foo-bundle" in the composer.json file. I have a command that already does it:
find ./ -maxdepth 2 -type f -print | grep -i 'composer.json' | xargs grep -i '"foo-bundle"'
However I want to make an sh script that gets all those directories and do things with them. Any idea?
find
Your current command is almost there, instead off using xargs with grep, lets:
Move the grep to an -exec
Use xargs to pass the result to dirname to show only the parent folder
find ./ -maxdepth 2 -type f -exec grep -l "foo-bundle" {} /dev/null \; | xargs dirname
If you only want to search for composer.json files, we can include the -iname option like so:
find ./ -maxdepth 2 -type f -iname '*composer.json' -exec grep -l "foo-bundle" {} /dev/null \; | xargs dirname
If the | xargs dirname doesn't give enough data, we can extend it so we can loop over the results of find using a while read like so:
find ./ -maxdepth 2 -type f -iname '*composer.json' -exec grep -l "foo-bundle" {} /dev/null \; | while read -r line ; do
parent="$(dirname ${line%%:*})"
echo "$parent"
done
grep
We can use grep to search for all files containing a specific text.
After looping over each line, we can
Remove behind the : to get the filepath
Use dirname to get the parent folder path
Consider this file setup, were /test/b/composer.json contains foo-bundle
➜ /tmp tree
.
├── test
│ ├── a
│ │ └── composer.json
│ └── b
│ └── composer.json
└── test.sh
When running the following test.sh:
#!/bin/bash
grep -rw '/tmp/test' --include '*composer.json' -e 'foo-bundle' | while read -r line ; do
parent="$(dirname ${line%:*})"
echo "$parent"
done
The result is as expected, the path to folder b:
/tmp/test/b
In order to find all files, containing a particular piece of text, you can use:
find ./ -maxdepth 2 -type f -exec grep -l "composer.json" {} /dev/null \;
The result is a list of filenames. Now all you need to do is to get a way to launch the command dirname on all of them. (I tried using a simple pipe, but that would have been too easy :-) )
Thanks to #0stone0 for leading the way. I finally got it with:
#!/bin/sh
find /var/www -maxdepth 2 -type f -print | grep -i 'composer.json' | xargs grep -i 'foo-bundle' | while read -r line ; do
parent="$(dirname ${line%%:*})"
echo "$parent"
done
Related
I am looking for a terminal linux command to give out the folder parent name that does not contain a certain file:
By now I use the following command:
find . -type d -exec test -e '{}'/recon-all.done \; -print| wc -l
Which gives me the amount of folders which contain then file.
The file recon-all.done would be in /subject/../../recon-all.done and I would need every single "subject" name which does not contain the recon-all.done file.
Loop through the directories, test for the existence of the file, and print the directory if the test fails.
for subject in */; do
if ! [ -e "${subject}scripts/recon-all.done" ]
then echo "$subject"
fi
done
Your command;
find . -type d -exec test -e '{}'/recon-all.done \; -print| wc -l
Almost does the job, we'll just need to
Remove | wc -l to show the directory path witch does not contain the recon-all file
Now, we can negate the -exec test by adding a ! like so:
find . -type d \! -exec test -e '{}'/recon-all.done \; -print
This way find will show each folder name if it does not contain the recon-all file
Note; Based on your comment on Barmar's answer, I've added a -maxdepth 1 to prevent deeper directorys from being checked.
Small example from my local machine:
$ /tmp/test$ tree
.
├── a
│ └── test.xt
├── b
├── c
│ └── test.xt
└── x
├── a
│ └── test.xt
└── b
6 directories, 3 files
$ /tmp/test$ find . -maxdepth 1 -type d \! -exec test -e '{}/test.xt' \; -print
.
./b
./x
$ /tmp/test$
I have these bad data
AWS-Console.pngcrop-AWS-Console.png
Alimofire.pngcrop-Alimofire.png
Amazon-ECR-.pngcrop-Amazon-ECR-.png
Amazon-ECS.pngcrop-Amazon-ECS.png
Amazon-RDS.pngcrop-Amazon-RDS.png
Angular.pngcrop-Angular.png
AngularJS.pngcrop-AngularJS.png
.... 1000 more
I'm trying to delete them
I've tried
ls public/assets/fe/img/skill/ | grep crop | rm -rf *crop*
ls public/assets/fe/img/skill/ | grep crop | rm -rf
rm -rf $(ls public/assets/fe/img/skill/ | grep crop)
None of them work ...
rm can handle the glob expressions that ls handles:
rm public/assets/fe/img/skill/*crop*
Use the find command instead
find . -name "*crop*" -type f -exec rm -i {} \;
-type f will specify to search file only and avoid directories
-exec requires the command input to end with \;, the {} being substitute by the result of the command
the -i will ask you to confirm ; remove it once sure what you do.
advice display the result beforehand with -print in place of -exec ...
find . -name "*crop*" -type f -print
More here where your question would find more accurate answers
The main problem in your commands is the missing path in the output of the ls command.
ls public/assets/fe/img/skill/ | grep crop will retur e.g. AWS-Console.pngcrop-AWS-Console.png which is passed to rm. But rm AWS-Console.pngcrop-AWS-Console.png fails because there is no such file in the current directory. It should be rm public/assets/fe/img/skill/AWS-Console.pngcrop-AWS-Console.png instead.
Adding -d to the ls command should do the trick:
ls -d public/assets/fe/img/skill/ | grep crop | rm -rf
rm -rf $(ls -d public/assets/fe/img/skill/ | grep crop)
As pointed out in other answers, other solutions exist, including:
rm public/assets/fe/img/skill/*crop*
find public/assets/fe/img/skill/ -name "*crop*" -type f -exec rm -i {} \;
If it's a really large number of files (apparently wasn't in your case), xargs can speed up the process. This applies for a lot of things you might want to read from a pipe.
find . -name "*crop*" -type f | xargs rm
The main advantage of using find here is that it's an easy way to ignore directories. If that's not an issue, let the OS handle all that.
printf "%s\n" public/assets/fe/img/skill/*crop* | xargs rm
If you need to be able to pick up files in subdirectories -
shopt -s globstar # double asterisks not include arbitrary preceding paths
printf "%s\n" public/assets/fe/img/skill/**crop* | xargs rm
You might want to look over the list first, though.
printf "%s\n" public/assets/fe/img/skill/*crop* >crop.lst
# check the list - vi, grep, whatever satisfies you.
xargs rm < crop.lst # fast-delete them in bulk
Here's what I have to do: Find all files which are in the directory src (or in its subdirectories) and have str in their name and copy them to dest preserving the subdirectory structure. For example I have the directory dir1 which contains foo.txt and the subdirectory subdir which also contains foo.txt. After running my script (with str=txt and dest=dir2) dir2 should countain foo.txt and subdir/foo.txt. So far I have come up with this code:
while read -r line; do
cp --parents $line $dest
done <<< "$(find $src -name "*$str*")"
which almost does the job except that it creates dir1 inside of dir2 and the desired files are inside dir2/dir1. I also tried doing it with the -exec option of find but didn't get better results.
IIUC, this can be done with find ... -exec. Let's say we have the following directory:
$ tree
.
└── src
├── dir1
│ └── yet_another_file_src
└── file_src
2 directories, 2 files
We can copy all files that contain *src* to /tmp/copy-here like this:
$ find . -type f -name "*src*" -exec sh -c 'echo mkdir -p /tmp/copy-here/$(dirname {})' \; -exec sh -c 'echo cp {} /tmp/copy-here/$(dirname {})' \;
mkdir -p /tmp/copy-here/./src
cp ./src/file_src /tmp/copy-here/./src
mkdir -p /tmp/copy-here/./src/dir1
cp ./src/dir1/yet_another_file_src /tmp/copy-here/./src/dir1
Notice that I used echo instead of really running this command -
read the output and make sure that this is what you want to
achieve. If you're sure that this would be what you want just remove
echo like this:
$ find . -type f -name "*src*" -exec sh -c 'mkdir -p /tmp/copy-here/$(dirname {})' \; -exec sh -c 'cp {} /tmp/copy-here/$(dirname {})' \;
$ tree /tmp/copy-here
/tmp/copy-here
└── src
├── dir1
│ └── yet_another_file_src
└── file_src
2 directories, 2 files
EDIT:
And of course, you can always use rsync:
$ rsync -avz --include "*/" --include="*src*" --exclude="*" "$PWD" /tmp/copy-here
I have several parent folders like GJ1, GJ2 etc. Each of these folders contain three images like GJ11_F.jpg, GJ11_P.jpg. I need to only display all the GJ11_F.jpg files including their respective parent directories.
find . -type f -name "*_F.jpg" | xargs cp -t ~/home/ubuntu/
but the above command will only copy the *_F.jpg files and not their respective parent directories GJ1.
Is xargs not the one im supposed to try?
I have also tried -
find . -name "*_F.jpg" -exec sh -c 'rsync -a "${0%/*}" ~/home/ubuntu/' {} \;
One easy way is to use tar which will deal with the directories automatically:
find . -type f -name "*_F.jpg" -print0 | tar c --null -T - | tar xC ~/home/ubuntu/
And here's a solution with a while loop:
find . -type f -name "*_F.jpg" -print0 |
while IFS= read -r -d '' file; do
mkdir -p ~/home/ubuntu/"$(dirname -- "$file")"
cp -ai -- "$file" ~/home/ubuntu/"$file"
done
In a Linux terminal, how to delete all files from a folder except one or two?
For example.
I have 100 image files in a directory and one .txt file.
I want to delete all files except that .txt file.
From within the directory, list the files, filter out all not containing 'file-to-keep', and remove all files left on the list.
ls | grep -v 'file-to-keep' | xargs rm
To avoid issues with spaces in filenames (remember to never use spaces in filenames), use find and -0 option.
find 'path' -maxdepth 1 -not -name 'file-to-keep' -print0 | xargs -0 rm
Or mixing both, use grep option -z to manage the -print0 names from find
In general, using an inverted pattern search with grep should do the job. As you didn't define any pattern, I'd just give you a general code example:
ls -1 | grep -v 'name_of_file_to_keep.txt' | xargs rm -f
The ls -1 lists one file per line, so that grep can search line by line. grep -v is the inverted flag. So any pattern matched will NOT be deleted.
For multiple files, you may use egrep:
ls -1 | grep -E -v 'not_file1.txt|not_file2.txt' | xargs rm -f
Update after question was updated:
I assume you are willing to delete all files except files in the current folder that do not end with .txt. So this should work too:
find . -maxdepth 1 -type f -not -name "*.txt" -exec rm -f {} \;
find supports a -delete option so you do not need to -exec. You can also pass multiple sets of -not -name somefile -not -name otherfile
user#host$ ls
1.txt 2.txt 3.txt 4.txt 5.txt 6.txt 7.txt 8.txt josh.pdf keepme
user#host$ find . -maxdepth 1 -type f -not -name keepme -not -name 8.txt -delete
user#host$ ls
8.txt keepme
Use the not modifier to remove file(s) or pattern(s) you don't want to delete, you can modify the 1 passed to -maxdepth to specify how many sub directories deep you want to delete files from
find . -maxdepth 1 -not -name "*.txt" -exec rm -f {} \;
You can also do:
find -maxdepth 1 \! -name "*.txt" -exec rm -f {} \;
In bash, you can use:
$ shopt -s extglob # Enable extended pattern matching features
$ rm !(*.txt) # Delete all files except .txt files