I just started bash and i am doing a search script that search for files and grep a pattern. Simple idea:
find $HOME -type f | grep $1
In the current script i match everything that contains $1 (files and directories). I only want to match the pattern in the filename, i don't want to match the directories in the path. I tried lots of advanced expressions with symbols like "/.*^$" etc.. to grep in a specific part but honestly for a new user is being a bit hard.
Cut tool is not an option because i want the path of the file.
EDIT:
Correct Example:
$ ./search test
/home/user/documents/test.txt
/home/user/downloads/blahtestblah.py
Incorrect example:
$ ./search test
/home/user/test/whatever.txt
In the incorrect example grep matched the keyword in the path, matching a directory.
^ matches the beginning of the string. Try:
find $HOME -type f | grep "/.*$1$"
You could also use the -regex switch to find instead of piping the output to grep:
find $HOME -type f -regex ".*$1$"
Related
I am trying to find a specific line in files that contains "Mutual_Values_23.0" in a directory that contains a lot of subdirectories. I know this line number is stored in a file which starts with "gnuout_mutual_....txt" (the ellipses part of the file name is the time stamp so that varies).
I wanted to know if there is a way to specify "grep" command to look into the subdirectories only for the files starting with "gnuout_mutual_....txt"
I have tried
grep -r "Mutual_Values_23.0" *
but that's taking a long time
You can use the following option of grep:
--include=GLOB
Search only files whose base name matches GLOB (using wildcard matching as described under --exclude).
And for the line number you should use the -n option.
From within the root of the folders you want to look into, you can use this final command:
grep -nr "Mutual_Values_23.0" --include="gnuout_mutual_*txt"
Use find to search all sub-directories for the "gnuout...txt` file with the search string "Mutual_Values_23.0"
find . -mindepth 1 -name gnuout_mutual_\*.txt -type f -exec grep "Mutual_Values_23.0" {} +
If you make use of bash, you can use the globstar option:
globstar
If set, the pattern ** used in a pathname expansion context will
match all files and zero or more directories and subdirectories.
If the pattern is followed by a /, only directories and
subdirectories match.
So you can use it like:
$ shopt -s globstar
$ grep "search_string" **/glob-pattern
or in the case of the OP:
$ shopt -s globstar
$ grep Mutual_Values_23.0 **/gnuout_mutual_*.txt
GNU grep has the --include GLOB option where GLOB can be used to specify the file name pattern that you need to match.
grep -rn --include 'gnuout_mutual_*txt' 'Mutual_Values_23.0' .
You could use find to search for files and pass results to grep.
find /directory_where_to_search/ -iname 'gnuout_mutual_*.txt' | xargs grep 'Mutual_Values_23.0' -sl
Use this command:
$ find . -name "*Mutual_Values_23.0*"
Note: Run this command in the directory where you want to search your set of files.
Hope it helps, cheers!
I want to display all directory names in directory /opt/BMC/patrol/ that begin with Patrol* or patrol*. Command
ls /opt/BMC/patrol/ | grep -i '^Patrol*'
produces
Patrol3
Patrol3.16
PatrolAgent_3181.sh
patrol_cfg.sh
It is correct but there are directories and files, instead of just directories.
Command ls -d /opt/BMC/patrol/*/ | grep -i '^Patrol*' produces nothing...
Command ls -d /opt/BMC/patrol/*/ | grep -i 'Patrol*' produces
/opt/BMC/patrol/BMCINSTALL/
/opt/BMC/patrol/bmc_products/
/opt/BMC/patrol/cert_gg/
/opt/BMC/patrol/common/
/opt/BMC/patrol/Install/
/opt/BMC/patrol/itools/
/opt/BMC/patrol/Patrol3/
/opt/BMC/patrol/Patrol3.16/
/opt/BMC/patrol/perform/
/opt/BMC/patrol/rtserver/
/opt/BMC/patrol/temp2/
/opt/BMC/patrol/test/
/opt/BMC/patrol/testftp/
/opt/BMC/patrol/Uninstall/
Does it search recursively? What is a command to find only directory names that begins with capital or small letters?
Try find:
find /opt/BMC/patrol -type d -iname 'patrol*'
-type d matches directories, and -iname is a case-insensitive match. The 'patrol*' has to be quoted '' because otherwise the shell will expand the * before find gets a chance.
find does search recursively by default (see Edit 2, below).
Edit ls is not optimized for this use case. ls -d prevents descending into directories, which is why you don't get any matches. As far as grep goes, ^ matches at the beginning of the line, not at the leading / before a directory's name. So grep -i '\/patrol' would be a way to find names beginning with Patrol or patrol, but you would still have to filter down to directories. find is designed to handle all these things.
Edit 2 For non-recursive, use -maxdepth:
find /opt/BMC/patrol -maxdepth 1 -type d -iname 'patrol*'
I made a test directory with the following contents, based on your question:
opt/BMC/patrol/
opt/BMC/patrol/BMCINSTALL/
opt/BMC/patrol/bmc_products/
opt/BMC/patrol/cert_gg/
opt/BMC/patrol/common/
opt/BMC/patrol/common/patrol.d/
opt/BMC/patrol/Install/
opt/BMC/patrol/itools/
opt/BMC/patrol/Patrol3/
opt/BMC/patrol/Patrol3.16/
opt/BMC/patrol/PatrolAgent_3181.sh
opt/BMC/patrol/patrol_cfg.sh
opt/BMC/patrol/perform/
opt/BMC/patrol/rtserver/
opt/BMC/patrol/temp2/
opt/BMC/patrol/test/
opt/BMC/patrol/testftp/
opt/BMC/patrol/Uninstall/
When I run the first command (without -maxdepth), I get:
opt/BMC/patrol
opt/BMC/patrol/common/patrol.d
opt/BMC/patrol/Patrol3
opt/BMC/patrol/Patrol3.16
When I run the second command (with -maxdepth), I get:
opt/BMC/patrol
opt/BMC/patrol/Patrol3
opt/BMC/patrol/Patrol3.16
and common/patrol.d is not present in the results.
How can I use this grep pattern to recursively search a directory? I need for both of these to be on the same line in the file the string. I keep getting the message back this is a directory. How can I make it search recursively all files with the extension .cfc?
"<cffunction" and "inject="
grep -insR "<cffunction" | grep "inject=" /c/mydirectory/
Use find and exec:
find your_dir -name "*.cfc" -type f -exec grep -insE 'inject=.*<cffunction|<cffunction.*inject=' /dev/null {} +
find finds your *.cfc files recursively and feeds into grep, picking only regular files (-type f)
inject=.*<cffunction|<cffunction.*inject= catches lines that have your patterns in either order
{} + ensures each invocation of grep gets up to ARG_MAX files
/dev/null argument to grep ensures that the output is prefixed with the name of file even when there is a single *.cfc file
You've got it backwards, you should pipe your file search to the second command, like:
grep -nisr "inject=" /c/mydirectory | grep "<cffunction"
edit: to exclude some directories and search only in *.cfc files, use:
grep -nisr --exclude-dir={/abs/dir/path,rel/dir/path} --include \*.cfc "inject=" /c/mydirectory | grep "<cffunction"
OS Linux Mint
Like the title says finally I would like to find and copy all images in a directory.
I found:
find all jpg (or JPG) files in a directory and copy them into the folder /home/joachim/neu2:
find . -iname \*.jpg -print0 | xargs -I{} -0 cp -v {} /home/joachim/neu2
and
find all image files in a direcotry:
find . -name '*' -exec file {} \; | grep -o -P '^.+: \w+ image'
My problem is first of all, I don't really understand the syntax. Could someone explain the code?
And secondly can someone connect the two codes for generating a code that does what I want ;)
Greetings and thanks in advance!
First, understand that the pipe "|" links commands piping the output of the first into the second as an argument. Your two shell codes both pipe output of the find command into other commands (grep and xargs). Let's look at those commands one after another:
First command: find
find is a program to "search for files in a directory hierarchy" (that is the explanation from find's man page). The syntax is (in this case)
find <search directory> <search pattern> <action>
In both cases the search directory is . (that is the current directory). Note that it does not just search the current directory but all its subdirectories as well (the directory hierarchy).
The search pattern accepts options -name (meaning it searches for files the name of which matches the pattern given as an argument to this option) or -iname (same as name but case insensitive) among others.
The action pattern may be -print0 (print the exact filename including its position in the given search directory, i.e. the relative or absolute path to the file) or -exec (execute the given command on the file(s), the command is to be ended with ";" and every instance of "{}" is replaced by the filename).
That is, the first shell code (first part, left of the pipe)
find . -iname \*.jpg -print0
searches all files with ending ".jpg" in the current directory hierarchy and prints their paths and names. The second one (first part)
find . -name '*' -exec file {} \;
finds all files in the current directory hierarchy and executes
file <filename>
on them. File is another command that determines and prints the file type (have a look at the man page for details, man file).
Second command: xargs
xargs is a command that "builds and exectues command lines from standard input" (man xargs), i.e. from the find output that is piped into xargs. The command that it builds and executes is in this case
cp -v {} /home/joachim/neu2"
Option -I{} defines the replacement string, i.e. every instance of {} in the command is to be replaced by the input it gets from file (that is, the filenames). Option -0 defines that input items are not terminated (seperated) by whitespace or newlines but only by a null character. This seems to be necessary when using and the standard way to deal with find output as xargs input.
The command that is built and executed is then of course the copy command with option -v (verbose) and it copies each of the filenames it gets from find to the directory.
Third command: grep
grep filters its input giving only those lines or strings that match a particular output pattern. Option -o tells grep to print only the matching string, not the entire line (see man grep), -P tells it to interpret the following pattern as a perl regexp pattern. In perl regex, ^ is the start of the line, .+ is any arbitrary string, this arbitrary should then be followed by a colon, a space, a number of alphanumeric characters (in perl regex denoted \w+) a space and the string "image". Essentially this grep command filters the file output to only output the filenames that are image files. (Read about perl regex's for instance here: http://www.comp.leeds.ac.uk/Perl/matching.html )
The command you actually wanted
Now what you want to do is (1) take the output of the second shell command (which lists the image files), (2) bring it into the appropriate form and (3) pipe it into the xargs command from the first shell command line (which then builds and executes the copy command you wanted). So this time we have a three (actually four) stage shell command with two pipes. Not a problem. We already have stages (1) and (3) (though in stage (3) we need to leave out the -0 option because the input is not find output any more; we need it to treat newlines as item seperators).
Stage (2) is still missing. I suggest using the cut command for this. cut changes strings py splitting them into different fields (seperated by a delimiter character in the original string) that can then be rearranged. I will choose ":" as the delimiter character (this ends the filename in the grep output, option -d':') and tell it to give us just the first field (option -f1, essentialls: print only the filename, not the part that comes after the ":"), i.e. stage (2) would then be
cut -d':' -f1
And the entire command you wanted will then be:
find . -name '*' -exec file {} \; | grep -o -P '^.+: \w+ image' | cut -d':' -f1 | xargs -I{} cp -v {} /home/joachim/neu2
Note that you can find all the man pages for instance here: http://www.linuxmanpages.com
I figured out a command only using awk that does the job as well:
find . -name '*' -exec file {} \; |
awk '{
if ($3=="image"){
print substr($1, 0, length($1)-1);
system("cp " substr($1, 0, length($1)-1) " /home/joachim/neu2" )
}
}'
the substr($1, 0, length($1)-1) is needed because in first column file returns name;
The above answer is really good. but it could take longer if it a huge directory.
here is a shorter version of it , if you already know your file extension
find . -name \*.jpg | cut -d':' -f1 | xargs -I{} cp --parents -v {} ~/testimage/
Here's another one which works like a charm.
It adds the EPOCH time to prevent overwriting files with the same name.
cd /media/myhome/'Local station'/
find . -path ./jpg -prune -o -type f -iname '*.jpg' -exec sh -c '
for file do
newname="${file##*/}"
newname="${newname%.jpg}"
mv -T -- "$file" "/media/myhome/Local station/jpg/$newname-$(date +%s).jpg"
done
' find-sh {} +
cd ~/
It's been designed by Kamil in this post here.
Find a specific type file from a directory:
find /home/user/find/data/ -name '*' -exec file {} \; | grep -o -P '^.+: \w+ image'
Copy specific type of file from one directory to another directory:
find /home/user/find/data/ -name '*' -exec file {} \; | grep -o -P '^.+: \w+ image' | cut -d':' -f1 | xargs -I{} cp -v {} /home/user/copy/data/
In Windows, I would have done a search for finding a word inside a folder. Similarly, I want to know if a specific word occurs inside a directory containing many sub-directories and files. My searches for grep syntax shows I must specify the filename, i.e. grep string filename.
Now, I do not know the filename, so what do I do?
A friend suggested to do grep -nr string, but I don't know what this means and I got no results with it (there is no response until I issue a Ctrl + C).
grep -nr 'yourString*' .
The dot at the end searches the current directory. Meaning for each parameter:
-n Show relative line number in the file
'yourString*' String for search, followed by a wildcard character
-r Recursively search subdirectories listed
. Directory for search (current directory)
grep -nr 'MobileAppSer*' . (Would find MobileAppServlet.java or MobileAppServlet.class or MobileAppServlet.txt; 'MobileAppASer*.*' is another way to do the same thing.)
To check more parameters use man grep command.
grep -nr string my_directory
Additional notes: this satisfies the syntax grep [options] string filename because in Unix-like systems, a directory is a kind of file (there is a term "regular file" to specifically refer to entities that are called just "files" in Windows).
grep -nr string reads the content to search from the standard input, that is why it just waits there for input from you, and stops doing so when you press ^C (it would stop on ^D as well, which is the key combination for end-of-file).
GREP: Global Regular Expression Print/Parser/Processor/Program.
You can use this to search the current directory.
You can specify -R for "recursive", which means the program searches in all subfolders, and their subfolders, and their subfolder's subfolders, etc.
grep -R "your word" .
-n will print the line number, where it matched in the file.
-i will search case-insensitive (capital/non-capital letters).
grep -inR "your regex pattern" .
There's also:
find directory_name -type f -print0 | xargs -0 grep -li word
but that might be a bit much for a beginner.
find is a general purpose directory walker/lister, -type f means "look for plain files rather than directories and named pipes and what have you", -print0 means "print them on the standard output using null characters as delimiters". The output from find is sent to xargs -0 and that grabs its standard input in chunks (to avoid command line length limitations) using null characters as a record separator (rather than the standard newline) and then applies grep -li word to each set of files. On the grep, -l means "list the files that match" and -i means "case insensitive"; you can usually combine single character options so you'll see -li more often than -l -i.
If you don't use -print0 and -0 then you'll run into problems with file names that contain spaces so using them is a good habit.
grep -nr search_string search_dir
will do a RECURSIVE (meaning the directory and all it's sub-directories) search for the search_string. (as correctly answered by usta).
The reason you were not getting any anwers with your friend's suggestion of:
grep -nr string
is because no directory was specified. If you are in the directory that you want to do the search in, you have to do the following:
grep -nr string .
It is important to include the '.' character, as this tells grep to search THIS directory.
Why not do a recursive search to find all instances in sub directories:
grep -r 'text' *
This works like a charm.
Similar to the answer posted by #eLRuLL, an easier way to specify a search that respects word boundaries is to use the -w option:
grep -wnr "yourString" .
Another option that I like to use:
find folder_name -type f -exec grep your_text {} \;
-type f returns you only files and not folders
-exec and {} runs the grep on the files that were found in the search (the exact syntax is "-exec command {}").
grep -r "yourstring" *
Will find "yourstring" in any files and folders
Now if you want to look for two different strings at the same time you can always use option E and add words for the search. example after the break
grep -rE "yourstring|yourotherstring|$" * will search for list locations where yourstring or yourotherstring matches
The answer you selected is fine, and it works, but it isn't the correct way to do it, because:
grep -nr yourString* .
This actually searches the string "yourStrin" and "g" 0 or many times.
So the proper way to do it is:
grep -nr \w*yourString\w* .
This command searches the string with any character before and after on the current folder.
grep -R "string" /directory/
-R follows also symlinks when -r does not.
The following sample looks recursively for your search string in the *.xml and *.js files located somewhere inside the folders path1, path2 and path3.
grep -r --include=*.xml --include=*.js "your search string" path1 path2 path3
So you can search in a subset of the files for many directories, just providing the paths at the end.
Run(terminal) the following command inside the directory. It will recursively check inside subdirectories too.
grep -r 'your string goes here' *
Don't use grep. Download Silver Searcher or ripgrep. They're both outstanding, and way faster than grep or ack with tons of options.