Search for multiple patterns in multiple files - linux

Question

This is related to Function to search of multiple patterns using grep
I want to search multiple files with multiple patterns using command such as follows:
myscript *.txt pattern1 pattern2 pattern3
I tried implementing the codes in the previous question but they do not work with wildcards. For example, following does not work:
#!/bin/bash
ARGS=$#
if [ $ARGS -lt 2 ]
then
echo "You entered only $ARGS arguments- at least 2 are needed."
exit
fi
search() {
if [ $# -gt 0 ]
then
local pat=$1
shift
grep -i "$pat" | search "$#"
else
cat
fi
}
for VAR in $1
do
file=$VAR
shift
cat "$file" | search "$#"
done
How can I create a script which can search for multiple files (taking it from first argument) to search multiple patterns (from rest of arguments)?

Answer 1

Did you try to use find and sed?
find . -name *.txt -exec sed -n -e '/pattern1/p' -e '/pattern2/' '{}' ';'
The -n option will make sure sed does not print all the file, and the p command prints the matching lines. Finaly, find will get all the files you need.
EDIT:
If you want to put that in a script to generate the sed command, you can use this trick.
EDIT 2:
As #shellter said, it is usually better to use options, and as your script is written, *.txt will be expanded by bash. To avoid that, you'll need to quote the first argument.
As usual, there is several solutions to your problem:
Solution 1 (Using bash built-in):
#! /usr/bin/env bash
set -o nounset # Throw error if variable not set
set -o errexit # Exit if error is thrown
work_dir=$PWD # directory to search from
# Reading the command line
files_pattern=${1:-}; # Save first argument as files pattern.
shift 1; # Move $1 to next argument (and propagate such as $n gets $n+1)
echo "==> Files to search follow pattern: ${files_pattern}"
_len=$#; #save the number of arguments.
for (( i=0; i<$_len; i=$i+1 )); # Go through the search patterns.
do
search_patterns[$i]=$1; # store the next search pattern
shift 1; # move $1 to next patern.
echo "==> New search pattern #$i: ${search_patterns[$i]}"
done
while read -r file; # Go through all the matching files
do
echo "==> In file: ${file}"
while read -r line; # Go though all the lines in the file
do
for regex in "${search_patterns[#]}"; # iterate trough patterns
do
[[ "${line}" =~ $regex ]] && echo "${line}";
done
done < ${file}
done < <(find $work_dir -iname $files_pattern -print) # find all the files matching file_pattern
Solution 2 (using grep):
#! /usr/bin/env bash
set -o nounset # Throw error if variable not set
set -o errexit # Exit if error is thrown
work_dir=$PWD # directory to search from
# Reading the command line
files_pattern=${1:-}; # Save first argument as files pattern.
shift 1; # Move $1 to next argument (and propagate such as $n gets $n+1)
echo "==> Files to search follow pattern: ${files_pattern}"
while [ $# -gt 0 ]; # Go through the search patterns.
do
search_patterns+="$1"; # store the next search pattern
shift 1; # move $1 to next patern.
[ $# -gt 0 ] && search_patterns+="|" #Add or option
done
echo "==> Search patterns: ${search_patterns}"
cd ${work_dir} && egrep -iR '('"${search_patterns}"')' && cd -;
Solution 3 (Using sed):
#! /usr/bin/env bash
set -o nounset # Throw error if variable not set
set -o errexit # Exit if error is thrown
work_dir=$PWD # directory to search from
# Reading the command line
files_pattern=${1:-}; # Save first argument as files pattern.
shift 1; # Move $1 to next argument (and propagate such as $n gets $n+1)
echo "==> Files to search follow pattern: ${files_pattern}"
while [ $# -gt 0 ]; # Go through the search patterns.
do
search_patterns+="/$1/p;"; # store the next search pattern
shift 1; # move $1 to next patern.
[ $# -gt 0 ] && search_patterns+=" " #Add or option
done
echo "==> Search patterns: ${search_patterns}"
# Will print file names, and then matching lines
find "$work_dir" -iname "$files_pattern" -print -exec sed -n "${search_patterns}" '{}' ';'
I am sure there is plenty other ways to tweak or solve this problem, but this should get you started.
Good Luck!