I need a way to tell if grep does find something, and ideally pass that return value to an if statement.
Let's say I have a tmp folder (current folder), in that there are several files and sub-folders. I want to search all files named abc for a pattern xyz. The search is assumed to be successful if it finds any occurrence of xyz (it does not matter how many times xyz is found). The search fails if no occurrence of xyz is found.
In bash, it can be done like this:
find . -name "abc" -exec grep "xyz" {} \;
That would show if xyz is found at all. But I'm not sure how pass the result (successful or not) back to an if statement.
Any help would be appreciated.
You can try
x=`find . -name abc | xargs grep xyz`
echo $x
That is, x contains your return value. It is blank when there is no match.
If you want to know that find finds some files abc and that at least one of them contains the string xyz, then you can probably use:
if find . -name 'abc' -type f -exec grep -q xyz {} +
then : All invocations of grep found at least one xyz and nothing else failed
else : One or more invocations of grep failed to find xyz or something else failed
fi
This relies on find returning an exit status for its own operations, and a non-zero exit status of any of the command(s) it executes. The + at the end groups as many file names as find thinks reasonable into a single command line. You need quite a lot of file name (a large number of fairly long names) to make find run grep multiple times. On a Mac running Mac OS X 10.10.4, I got to about 3,000 files, each with about 32 characters in the name, for an argument list of just over 100 KiB, without grep being run multiple times. OTOH, when I had just under 8000 files, I got two runs of grep, with around 130 KiB of argument list for each.
Someone briefly left a comment asking whether the exit status of find is guaranteed. The answer is 'yes' — though I had to modify my description of the exit status a bit. Under the description of -exec, POSIX specifies:
If any invocation [of the 'utility', such as grep in this question] returns a non-zero value as exit status, the find utility shall return a non-zero exit status.
And under the general 'Exit status' it says:
The following exit values shall be returned:
0 — All path operands were traversed successfully.
>0 — An error occurred.
Thus, find will report success as long as none of its own operations fails and as long as none of the grep commands it invokes fails. Zero files found but no failures will be reported as success. (Failures might be lack of permission to search a directory, for example.)
find returns the result of the -exec'd command as its result, just place the command in an if statement:
if [[ -n $(find . -name "abc" -exec grep "xyz" {} \;) ]]
then
# do something
fi
The grep command can be given a -q option that will "quiet" its output. Grep will return a success or a failure on the basis of whether it found anything in the file you pointed it at.
If your shell is capable of it, rather than trying to parse the output of a find command, you might want to try using a for loop instead. For example in bash:
shopt -s globstar
cd /some/directory
for file in **/abc; do
grep -q "xyz" "$file" && echo "Found something in $file"
done
Is this what you're looking for?
You should be able to do this with just grep with --include I believe.
Like this:
if grep -r --include=abc -q xyz; then
: Found at least one match
else
: No matching lines found
fi
Related
I have 2 files that I needed to grep in a separate file.
The two files are in this directory /var/list
TB.1234.txt
TB.135325.txt
I have to grep them in another file in another directory which is in /var/sup/. I used the command below:
for i in TB.*; do grep "$i" /var/sup/logs.txt; done
what I want to do is, if the result of the grep command contains the word "ERROR" the files which is found in /var/list will be moved to another directory /var/last.
for example I grep this file TB.1234.txt to /var/sup/logs.txt then the result is like this:
ERROR: TB.1234.txt
TB.1234.txt will be move to /var/last.
please help. I don't know how to construct the logic on how to move the files, I'm stuck in that I provided, I am also trying to use two greps in a for loop but I am encountering an error.
I am new in coding and really appreciates any help and suggestions. Thank you so much.
If you are asking how to move files which contain "ERROR", this should be extremely straightforward.
for file in TB.*; do
grep -q 'ERROR' "$file" &&
mv "$file" /var/last/
done
The notation this && that is a convenient shorthand for
if this; then
that
fi
The -q option to grep says to not print the matches, and quit as soon as you find one. Like all well-defined commands, grep sets its exit code to reflect whether it succeeded (the status is visible in $?, but usually you would not examine it directly; perhaps see also Why is testing ”$?” to see if a command succeeded or not, an anti-pattern?)
Your question is rather unclear, but if you want to find either of the matching files in a third file, perhaps something like
awk 'FNR==1 && (++n < ARGC-1) { a[n] = FILENAME; nextfile }
/ERROR/ { for(j=1; j<=n; ++j) if ($0 ~ a[j]) b[a[j]]++ }
END { for(f in b) print f }' TB*.txt /var/sup/logs.txt |
xargs -r mv -t /var/last/
This is somewhat inefficient in that it will read all the lines in the log file, and brittle in that it will only handle file names which do not contain newlines. (The latter restriction is probably unimportant here, as you are looking for file names which occur on the same line as the string "ERROR" in the first place.)
In some more detail, the Awk script collects the wildcard matches into the array a, then processes all lines in the last file, looking for ones with "ERROR" in them. On these lines, it checks if any of the file names in a are also found, and if so, also adds them to b. When all lines have been processed, print the entries in b, which are then piped to a simple shell command to move them.
xargs is a neat command to read some arguments from standard input, and run another command with those arguments added to its command line. The -r option says to not run the other command if there are no arguments.
(mv -t is a GNU extension; it's convenient, but not crucial to have here. If you need portable code, you could replace xargs with a simple while read -r loop.)
The FNR==1 condition requires that the input files are non-empty.
If the text file is small, or you expect a match near its beginning most of the time, perhaps just live with grepping it multiple times:
for file in TB.*; do
grep -Eq "ERROR.*$file|$file.*ERROR" /var/sup/logs.txt &&
mv "$file" /var/last/
done
Notice how we now need double quotes, not single, around the regular expression so that the variable $file gets substituted in the string.
grep has an -l switch, showing only the filename of the file which contains a pattern. It should not be too difficult to write something like (this is pseudocode, it won't work, it's just for giving you an idea):
if $(grep -l "ERROR" <directory> | wc -l) > 0
then foreach (f in $(grep -l "ERROR")
do cp f <destination>
end if
The wc -l is to check if there are any files which contain the word "ERROR". If not, nothing needs to be done.
Edit after Tripleee's comment:
My proposal can be simplified as:
if grep -lq "ERROR" TB.*;
then foreach (f in $(grep -l "ERROR")
do cp f <destination>
end if
Edit after Tripleee's second comment:
This is even shorter:
for f in $(grep -l "ERROR" TB.*);
do cp "$f" destination;
done
How do i print out all the files of the current directory that start with the letter "k" ?Also needs to count this files.
I tried some methods but i only got errors or wrong outputs. Really stuck on this as a newbie in bash.
Try this Shellcheck-clean pure POSIX shell code:
count=0
for file in k*; do
if [ -f "$file" ]; then
printf '%s\n' "$file"
count=$((count+1))
fi
done
printf 'count=%d\n' "$count"
It works correctly (just prints count=0) when run in a directory that contains nothing starting with 'k'.
It doesn't count directories or other non-files (e.g. fifos).
It counts symlinks to files, but not broken symlinks or symlinks to non-files.
It works with 'bash' and 'dash', and should work with any POSIX-compliant shell.
Here is a pure Bash solution.
files=(k*)
printf "%s\n" "${files[#]}"
echo "${#files[#]} files total"
The shell expands the wildcard k* into the array, thus populating it with a list of matching files. We then print out the array's elements, and their count.
The use of an array avoids the various problems with metacharacters in file names (see e.g. https://mywiki.wooledge.org/BashFAQ/020), though the syntax is slightly hard on the eyes.
As remarked by pjh, this will include any matching directories in the count, and fail in odd ways if there are no matches (unless you set nullglob to true). If avoiding directories is important, you basically have to get the directories into a separate array and exclude those.
To repeat what Dominique also said, avoid parsing ls output.
Demo of this and various other candidate solutions:
https://ideone.com/XxwTxB
To start with: never parse the output of the ls command, but use find instead.
As find basically goes through all subdirectories, you might need to limit that, using the -maxdepth switch, use value 1.
In order to count a number of results, you just count the number of lines in your output (in case your output is shown as one piece of output per line, which is the case of the find command). Counting a number of lines is done using the wc -l command.
So, this comes down to the following command:
find ./ -maxdepth 1 -type f -name "k*" | wc -l
Have fun!
This should work as well:
VAR="k"
COUNT=$(ls -p ${VAR}* | grep -v ":" | wc -w)
echo -e "Total number of files: ${COUNT}\n" 1>&2
echo -e "Files,that begin with ${VAR} are:\n$(ls -p ${VAR}* | grep -v ":" )" 1>&2
I have a bunch of code thats relatively new i.e. lots of bugs hiding and I have code as such:
if [ -d $DATA_ROOT/$name ], I've done research and understand that this means if directory exists but now I'm trying to print out those directories that exist to fix a problem.
Tried using
echo `First: $DATA_ROOT`
echo `Second: $name`
echo `Last: $DATA_ROOT/$name`
exit 1;
Got command not found for all, the code is meant to fix the bug I'm trying to by extracting all files but does not end up extracting all ending up with the data extraction failed error below, code:
num_files=`find $DATA_ROOT/$name -name '*' | wc -l`
if [ ! $num_files -eq $extract_file ] ; then
echo "Data extraction failed! Extracted $num_files instead of $extract_file"
exit 1;
I just want to extract all files correctly, how to do this please?
The back-ping you are using means "execute this as an command"
echo `First: $DATA_ROOT`
echo `Second: $name`
echo `Last: $DATA_ROOT/$name`
would try to execute a command called "First:" which does not exists.
Instead use double quotes as they allow for variable substitution, like this and does not try to execute it as a command
echo "First: $DATA_ROOT"
echo "Second: $name"
echo "Last: $DATA_ROOT/$name"
Also
find $DATA_ROOT/$name -name '*'
is probably not what you want, the -name '*' is the default so you don't need it. As others points out, find will return everything, including directories and special files if you have any of those. find "$DATA_ROOT/$name" -type f is what you want if you only want to list the files or find "$DATA_ROOT/$name" -type d if you only want to list directories. Also always use double quotes around your "$DATA_ROOT/$name" as it allows you to handle file names with spaces -- if you have a $name that contains a space you will fail otherwise.
find reports not only ordinary files, but also directories (including .).
Use find "$DATA_ROOT/$name" -type f.
You are using backticks and hence anything under backticks is treated as a command to execute and as a result you are getting command not found exception. You could use double quotes to avoid the error like below:
echo "First: $DATA_ROOT"
echo "Second: $name"
echo "Last: $DATA_ROOT/$name"
You could use find command to list down all directories like:
find $DATA_ROOT/$name -type d
Above command would list all the directories (with -type d option and use -type f to list all the files) within $DATA_ROOT/$name and then you can perform operations on those directories.
I'm trying to bring a piece of code that I had running on an AIX box over to Linux and I just can't get it to work.
[ -f $FolderPath/*/FileName.txt ] && echo 1 || echo 0
The above searchs the FolderPath and then the * is all its subdirectories looking for FileName.txt. If it is found (more than once), it returns 1, otherwise returns 0.
In Linux, I get the Too Many Arguments error, so I thought changing to [[ ]] would fix this, but it doesn't seem to handle the wildcard * in that.
Anyone any ideas?
Thanks
You could try the following command and then query the status code $?:
find $FolderPath -name 'FileName.txt' | grep -E '*'
This returns 1 when there are no files listed by the find command and 0 when there are.
Optionally, if you're only interested in hitting a specific level to avoid deep searches down the directory tree then you can use the -maxdepth n option.
I tried to update a file in subversion. I need to add a line in makefile, in order to build up the revised version of code. Here is the command I tried to find the place in make file.
find . -name "*ake*" -exec grep filename {} /dev/null \;
It works. But my questions are:
1. What is the "\;" for? If I change it, there will be error message.
2 The /dev/null didn't change the results. I know this is the device where dispose all the "garbage information". But I still don't quite understand it in this situation.
Thanks in advance!
The \; indicates the end of the command to be executed by find. The \ is required to stop the shell interpreting the ; itself. From man find:
-exec command ;
Execute command; true if 0 status is returned. All following
arguments to find are taken to be arguments to the command until an
argument consisting of ‘;’ is encountered.
The /dev/null is a clever trick that took me a while to figure out. If grep is passed more than one filename it prints the containing filename before each match. /dev/null acts as an empty file containing no matches, but makes grep think it is always passed more then one filename. A much clearer alternative suggested by richard would be to use grep's -H option:
-H, --with-filename
Print the filename for each match. This is the default when there
is more than one file to search.