Checking if a multi-line string exist in a file - string

I wanted to make a bash script to check if each line of multi-line string already exist in a file.
I had already wrote some code, but I'm not sure if it will work:
str="
this is
a multiple
line
string"
while read -r f; do
while read -r s: do
if [ $r == $s ]; then break; fi
done
done << some_file.txt
any ideas to get it working ?

Here's a trick if you can load the whole file twice to memory:
str="
this is
a multiple
line
string"
# Read the whole file in variable
content=$(<some_file.txt)
# Replace str to nothing in content
repr=${content/${str}}
# Check if it's the same.
if [[ "$content" != "$repr" ]]; then
# If it's not, means content contains str and it was removed.
echo "Contains"
fi

It can be a perl one-liner:
if perl -0777 -sne '/$text/ or exit 1' -- -text="$str" "$file"
then
echo Found
else
echo not found
fi
The -0777 option slurps the whole file into memory

This could be a solution in bash:
Reading the string lines into an array.
Then, for each string line in the array:
Checking the file contains a whole line equal to that string line.
Stop walking the array once a check fails.
#!/bin/bash
file="$1"
names="
this is
a multiple
line
string"
# Read string lines into an array
readarray -t <<< $names
# Walk array of lines
all_strings_found=true
for string in "${MAPFILE[#]}"; do
if ! grep -q "^$string$" $file; then
echo "'"$string"'" "not found"
all_strings_found=false
break
fi
done
if [ "$all_strings_found" == true ]; then
echo "---> All strings found"
else
echo "---> NOT all strings found"
fi

I like case statements for this.
$ cat script
str1="
this is
a multiple
line
string"
str2="$1"
f="$(<$0)"
case "$f" in
*"$str1"*) echo "String 1 exists" ;;
*) echo "String 1 not found" ;;
esac
case "$f" in
*"$str2"*) echo "String 2 exists" ;;
*) echo "String 2 not found" ;;
esac
$ ./script esac
String 1 exists
String 2 exists
$ ./script foo
String 1 exists
String 2 not found

Related

Shell script to to check if a line exist in a file

I have tried all the solutions available on stack overflow, but when I use if condition with with it always results true.
I need to find a line in the file and see if it doesn't exit then insert the line in that file, but it always results that the line already exists.
Here is my script
isInFile=$(grep -q '^export' /etc/bashrc)
if [[ $isInFile == 0 ]];
then
echo "line is not present";
echo "export PROMPT_COMMAND='RETRN_VAL=\$?;logger -p local6.debug \"\$(whoami) [\$\$]: \$(history 1 | sed \"s/^[ ]*[0-9]\+[ ]*//\" )\"'" >> /etc/bashrc;
source /etc/bashrc;
else
echo "line is in the file";
fi
It always says that
line is in the file
The if statement branches based on the exit status of the command it's given. [[ is just one command you can use, it's not mandatory syntax. At an interactive prompt, enter help if
Do this:
if grep -q '^export' /etc/bashrc
then
# exit status of grep is zero: the pattern DOES MATCH the file
echo "line is in the file";
else
# exit status of grep is non-zero: the pattern DOES NOT MATCH the file
echo "line is not present";
echo "export PROMPT_COMMAND='RETRN_VAL=\$?;logger -p local6.debug \"\$(whoami) [\$\$]: \$(history 1 | sed \"s/^[ ]*[0-9]\+[ ]*//\" )\"'" >> /etc/bashrc;
source /etc/bashrc;
fi
I see 2 issues in your code:
if [[ $isInFile == 0 ]]; --If condition should not terminate with ;. Remove that.
The expression you are checking is always an empty string. Try echo $isInFile. What you are checking is output of the command, not its return value. Instead, you should remove -q from your grep expression and check if the output is empty or not.
Following code should work:
isInFile=$(grep '^export' /etc/bashrc)
if [ -z "$isInFile" ]
then
echo "line is not present";
echo "export PROMPT_COMMAND='RETRN_VAL=\$?;logger -p local6.debug \"\$(whoami) [\$\$]: \$(history 1 | sed \"s/^[ ]*[0-9]\+[ ]*//\" )\"'" >> /etc/bashrc;
source /etc/bashrc;
else
echo "line is in the file";
fi
-z check for emptiness of variable.

LINUX script bash [duplicate]

I want to check if a file contains a specific string or not in bash. I used this script, but it doesn't work:
if [[ 'grep 'SomeString' $File' ]];then
# Some Actions
fi
What's wrong in my code?
if grep -q SomeString "$File"; then
Some Actions # SomeString was found
fi
You don't need [[ ]] here. Just run the command directly. Add -q option when you don't need the string displayed when it was found.
The grep command returns 0 or 1 in the exit code depending on
the result of search. 0 if something was found; 1 otherwise.
$ echo hello | grep hi ; echo $?
1
$ echo hello | grep he ; echo $?
hello
0
$ echo hello | grep -q he ; echo $?
0
You can specify commands as an condition of if. If the command returns 0 in its exitcode that means that the condition is true; otherwise false.
$ if /bin/true; then echo that is true; fi
that is true
$ if /bin/false; then echo that is true; fi
$
As you can see you run here the programs directly. No additional [] or [[]].
In case if you want to check whether file does not contain a specific string, you can do it as follows.
if ! grep -q SomeString "$File"; then
Some Actions # SomeString was not found
fi
In addition to other answers, which told you how to do what you wanted, I try to explain what was wrong (which is what you wanted.
In Bash, if is to be followed with a command. If the exit code of this command is equal to 0, then the then part is executed, else the else part if any is executed.
You can do that with any command as explained in other answers: if /bin/true; then ...; fi
[[ is an internal bash command dedicated to some tests, like file existence, variable comparisons. Similarly [ is an external command (it is located typically in /usr/bin/[) that performs roughly the same tests but needs ] as a final argument, which is why ] must be padded with a space on the left, which is not the case with ]].
Here you needn't [[ nor [.
Another thing is the way you quote things. In bash, there is only one case where pairs of quotes do nest, it is "$(command "argument")". But in 'grep 'SomeString' $File' you have only one word, because 'grep ' is a quoted unit, which is concatenated with SomeString and then again concatenated with ' $File'. The variable $File is not even replaced with its value because of the use of single quotes. The proper way to do that is grep 'SomeString' "$File".
Shortest (correct) version:
grep -q "something" file; [ $? -eq 0 ] && echo "yes" || echo "no"
can be also written as
grep -q "something" file; test $? -eq 0 && echo "yes" || echo "no"
but you dont need to explicitly test it in this case, so the same with:
grep -q "something" file && echo "yes" || echo "no"
##To check for a particular string in a file
cd PATH_TO_YOUR_DIRECTORY #Changing directory to your working directory
File=YOUR_FILENAME
if grep -q STRING_YOU_ARE_CHECKING_FOR "$File"; ##note the space after the string you are searching for
then
echo "Hooray!!It's available"
else
echo "Oops!!Not available"
fi
grep -q [PATTERN] [FILE] && echo $?
The exit status is 0 (true) if the pattern was found; otherwise blankstring.
if grep -q [string] [filename]
then
[whatever action]
fi
Example
if grep -q 'my cat is in a tree' /tmp/cat.txt
then
mkdir cat
fi
In case you want to checkif the string matches the whole line and if it is a fixed string, You can do it this way
grep -Fxq [String] [filePath]
example
searchString="Hello World"
file="./test.log"
if grep -Fxq "$searchString" $file
then
echo "String found in $file"
else
echo "String not found in $file"
fi
From the man file:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by newlines, any of
which is to be matched.
(-F is specified by POSIX.)
-x, --line-regexp
Select only those matches that exactly match the whole line. (-x is specified by
POSIX.)
-q, --quiet, --silent
Quiet; do not write anything to standard output. Exit immediately with zero
status if any match is
found, even if an error was detected. Also see the -s or --no-messages
option. (-q is specified by
POSIX.)
Try this:
if [[ $(grep "SomeString" $File) ]] ; then
echo "Found"
else
echo "Not Found"
fi
I done this, seems to work fine
if grep $SearchTerm $FileToSearch; then
echo "$SearchTerm found OK"
else
echo "$SearchTerm not found"
fi
grep -q "something" file
[[ !? -eq 0 ]] && echo "yes" || echo "no"

Need to verify every file passed as an argument on the command line exists using a shell script

I am looking to create a shell script that reads command line arguments, then concatenates the contents of those files and print it to stdout. I need to verify the files passed to the command line exist.
I have written some code so far, but the script only works if only one command line argument is passed. If passing more than one argument, the error checking I have tried does not work.
#!/bin/bash
if [ $# -eq 0 ]; then
echo -e "Usage: concat FILE ... \nDescription: Concatenates FILE(s)
to standard output separating them with divider -----."
exit 1
fi
for var in "$#"
do
if [[ ! -e $# ]]; then
echo "One or more files does not exist"
exit 1
fi
done
for var in "$#"
do
if [ -f $var ]; then
cat $var
echo "-----"
exit 0
fi
done
I need to fix the error checking on this so that every command line argument is checked to be an existing file. If a file does not exist, the error must be printed to stderr and nothing should be printed to stdout.
You have a bug in line 11:
if [[ ! -e $# ]]; then
You do need to check for a given file here using $var like that:
if [[ ! -e "$var" ]]; then
And you exit prematurely in line 23 - you will always print only a
single file. And remember to always quote your variable because
otherwise your script would not run correctly on files that have a whitespaces in the name, for example:
$ echo a line > 'a b'
$ cat 'a b'
a line
$ ./concat.sh 'a b'
cat: a: No such file or directory
cat: b: No such file or directory
-----.
You said:
if a file does not exist, the error must be printed to stderr and
nothing should be printed to stdout.
You aren't printing anything to stderr at the moment, if you want to
you should do:
echo ... >&2
And you should use printf instead of echo as it's more portable
even though you're using Bash.
All in all, your script could look like this:
#!/bin/bash
if [ $# -eq 0 ]; then
printf "Usage: concat FILE ... \nDescription: Concatenates FILE(s) to standard output separating them with divider -----.\n" >&2
exit 1
fi
for var in "$#"
do
if [[ ! -e "$var" ]]; then
printf "One or more files does not exist\n" >&2
exit 1
fi
done
for var in "$#"
do
if [ -f "$var" ]; then
cat "$var"
printf -- "-----\n"
fi
done
exit 0

syntax checking for config files

I have a requirement to check the syntax of some config files.
The format of the config is as below:
[sect1]
sect1file1
sect1file2
[sect1_ends]
[sect2]
sect2file1
sect2file2
[sect2_ends]
My requirement is to check the for start of sect1 which is inside square brackets [sect1], then check that the files sect1file1 and sect1file2 exist, then check for the end of sect1 by reading sect1_ends inside square braces [sect1_ends]. Then repeat the same for sect2, and so on.
There is already a set of section names which are permitted. My objective is to check whether the section names are in the list, and whether the syntax is without any error.
I tried using
perl -lne 'print $1 while (/^\[(.*?)\]$/g)' <config filename>
but I'm not sure how to check and go through the file.
I am happy to see you have tried. Try again with this prototype:
while read -r line; do
if [ ${#line} -eq 0 ]; then
continue # ignore empty lines
fi
if [[ "${line}" = \[*\] ]]; then
echo "Line with [...]"
if [ -n "${inSection}" ]; then
if [ "${line}" = "${inSection/]/_ends]}" ]; then
echo "End of section"
unset inSection
else
echo "Invalid endtag ${line} while processing ${inSection}"
exit 1
fi
else
echo "Start of new section ${line}"
inSection="${line}"
fi
else
if [ -f "${line}" ]; then
echo "OK file ${line}"
else
echo "NOK file ${line}"
fi
fi
done < inputfile
Your solution looks good, but -n reads lines from standard input (STDIN). You need to feed your config file into STDIN to pass it to your script:
perl -lne 'print $1 while (/^\[(.*?)\]$/g)' <config.ini
Alternate option would be using -p.

Bash scripting: why is the last line missing from this file append?

I'm writing a bash script to read a set of files line by line and perform some edits. To begin with, I'm simply trying to move the files to backup locations and write them out as-is, to test the script is working. However, it is failing to copy the last line of each file. Here is the snippet:
while IFS= read -r line
do
echo "Line is ***$line***"
echo "$line" >> $POM
done < $POM.backup
I obviously want to preserve whitespace when I copy the files, which is why I have set the IFS to null. I can see from the output that the last line of each file is being read, but it never appears in the output.
I've also tried an alternative variation, which does print the last line, but adds a newline to it:
while IFS= read -r line || [ -n "$line" ]
do
echo "Line is ***$line***"
echo "$line" >> $POM
done < $POM.backup
What is the best way to do this do this read-write operation, to write the files exactly as they are, with the correct whitespace and no newlines added?
The command that is adding the line feed (LF) is not the read command, but the echo command. read does not return the line with the delimiter still attached to it; rather, it strips the delimiter off (that is, it strips it off if it was present in the line, IOW, if it just read a complete line).
So, to solve the problem, you have to use echo -n to avoid adding back the delimiter, but only when you have an incomplete line.
Secondly, I've found that when providing read with a NAME (in your case line), it trims leading and trailing whitespace, which I don't think you want. But this can be solved by not providing a NAME at all, and using the default return variable REPLY, which will preserve all whitespace.
So, this should work:
#!/bin/bash
inFile=in;
outFile=out;
rm -f "$outFile";
rc=0;
while [[ $rc -eq 0 ]]; do
read -r;
rc=$?;
if [[ $rc -eq 0 ]]; then ## complete line
echo "complete=\"$REPLY\"";
echo "$REPLY" >>"$outFile";
elif [[ -n "$REPLY" ]]; then ## incomplete line
echo "incomplete=\"$REPLY\"";
echo -n "$REPLY" >>"$outFile";
fi;
done <"$inFile";
exit 0;
Edit: Wow! Three excellent suggestions from Charles Duffy, here's an updated script:
#!/bin/bash
inFile=in;
outFile=out;
while { read -r; rc=$?; [[ $rc -eq 0 || -n "$REPLY" ]]; }; do
if [[ $rc -eq 0 ]]; then ## complete line
echo "complete=\"$REPLY\"";
printf '%s\n' "$REPLY" >&3;
else ## incomplete line
echo "incomplete=\"$REPLY\"";
printf '%s' "$REPLY" >&3;
fi;
done <"$inFile" 3>"$outFile";
exit 0;
After review i wonder if :
{
line=
while IFS= read -r line
do
echo "$line"
line=
done
echo -n "$line"
} <$INFILE >$OUTFILE
is juts not enough...
Here my initial proposal :
#!/bin/bash
INFILE=$1
if [[ -z $INFILE ]]
then
echo "[ERROR] missing input file" >&2
exit 2
fi
OUTFILE=$INFILE.processed
# a way to know if last line is complete or not :
lastline=$(tail -n 1 "$INFILE" | wc -l)
if [[ $lastline == 0 ]]
then
echo "[WARNING] last line is incomplete -" >&2
fi
# we add a newline ANYWAY if it was complete, end of file will be seen as ... empty.
echo | cat $INFILE - | {
first=1
while IFS= read -r line
do
if [[ $first == 1 ]]
then
echo "First Line is ***$line***" >&2
first=0
else
echo "Next Line is ***$line***" >&2
echo
fi
echo -n "$line"
done
} > $OUTFILE
if diff $OUTFILE $INFILE
then
echo "[OK]"
exit 0
else
echo "[KO] processed file differs from input"
exit 1
fi
Idea is to always add a newline at the end of file and to print newlines only BETWEEN lines that are read.
This should work for quite all text files given they are not containing 0 byte ie \0 character, in which case 0 char byte will be lost.
Initial test can be used to decided whether an incomplete text file is acceptable or not.
Add a new line if line is not a line. Like this:
while IFS= read -r line
do
echo "Line is ***$line***";
printf '%s' "$line" >&3;
if [[ ${line: -1} != '\n' ]]
then
printf '\n' >&3;
fi
done < $POM.backup 3>$POM

Resources