#!/bin/bash
#if there are no args supplied exit with 1
if [ "$#" -eq 0 ]; then
echo "Unfortunately you have not passed any parameter"
exit 1
fi
#loop over each argument
for arg in "$#"
do
if [ -f arg ]; then
echo "$arg is a file."
#iterates over the files stated in arguments and reads them $
cat $arg | while read line;
do
#should access only first line of the file
if [ head -n 1 "$arg" ]; then
process line
echo "Script has ran successfully!"
exit 0
#should access only last line of the file
elif [ tail -n 1 "$arg" ]; then
process line
echo "Script has ran successfully!"
exit 0
#if it accesses any other line of the file
else
echo "We only process the first and the last line of the file."
fi
done
else
exit 2
fi
done
#function to process the passed string and decode it in base64
process() {
string_to_decode = "$1"
echo "$string_to_decode = " | base64 --decode
}
Basically what I want this script to do is to loop over the arguments passed to the script and then if it's a file then call the function that decodes in base64 but just on the first and the last line of the chosen file. Unfortunately when I run it even with calling a right file it does nothing. I think it might be encountering problems with the if [ head -n 1 "$arg" ]; then part of the code. Any ideas?
EDIT: So I understood that I am actually just extracting first line over and over again without really comparing it to anything. So I tried changing the if conditional of the code to this:
first_line = $(head -n 1 "$arg")
last_line = $(tail -n 1 "$arg")
if [ first_line == line ]; then
process line
echo "Script has ran successfully!"
exit 0
#should access only last line of the file
elif [ last_line == line ]; then
process line
echo "Script has ran successfully!"
exit 0
My goal is to iterate through files for example one is looking like this:
MTAxLmdvdi51awo=
MTBkb3duaW5nc3RyZWV0Lmdvdi51awo=
MXZhbGUuZ292LnVrCg==
And to decode the first and the last line of each file.
To decode the first and last line of each file given to your script, use this:
#! /bin/bash
for file in "$#"; do
[ -f "$file" ] || exit 2
head -n1 "$file" | base64 --decode
tail -n2 "$file" | base64 --decode
done
Yea, as the others already said the true goal of the script isn't really clear. That said, i imagine every variation of what you may have wanted to do would be covered by something like:
#!/bin/bash
process() {
encoded="$1";
decoded="$( echo "${encoded}" | base64 --decode )";
echo " Value ${encoded} was decoded into ${decoded}";
}
(( $# )) || {
echo "Unfortunately you have not passed any parameter";
exit 1;
};
while (( $# )) ; do
arg="$1"; shift;
if [[ -f "${arg}" ]] ; then
echo "${arg} is a file.";
else
exit 2;
fi;
content_of_first_line="$( head -n 1 "${arg}" )";
echo "Content of first line: ${content_of_first_line}";
process "${content_of_first_line}";
content_of_last_line="$( tail -n 1 "${arg}" )";
echo "Content of last line: ${content_of_last_line}";
process "${content_of_last_line}";
line=""; linenumber=0;
while IFS="" read -r line; do
(( linenumber++ ));
echo "Iterating over all lines. Line ${linenumber}: ${line}";
process "${line}";
done < "${arg}";
done;
some additions you may find useful:
If the script is invoked with multiple filenames, lets say 4 different filenames, and the second file does not exist (but the others do),
do you really want the script to: process the first file, then notice that the second file doesnt exist, and exit at that point ? without processing the (potentially valid) third and fourth file ?
replacing the line:
exit 2;
with
continue;
would make it skip any invalid filenames, and still process valid ones that come after.
Also, within your process function, directly after the line:
decoded="$( echo "${encoded}" | base64 --decode )";
you could check if the decoding was successful before echoing whatever the resulting garbage may be if the line wasnt valid base64.
if [[ "$?" -eq 0 ]] ; then
echo " Value ${encoded} was decoded into ${decoded}";
else
echo " Garbage.";
fi;
--
To answer your followup question about the IFS/read-construct, it is a mixture of a few components:
read -r line
reads a single line from the input (-r tells it not to do any funky backslash escaping magic).
while ... ; do ... done ;
This while loop surrounds the read statement, so that we keep repeating the process of reading one line, until we run out.
< "${arg}";
This feeds the content of filename $arg into the entire block of code as input (so this becomes the source that the read statement reads from)
IFS=""
This tells the read statement to use an empty value instead of the real build-in IFS value (the internal field separator). Its generally a good idea to do this for every read statement, unless you have a usecase that requires splitting the line into multiple fields.
If instead of
IFS="" read -r line
you were to use
IFS=":" read -r username _ uid gid _ homedir shell
and read from /etc/passwd which has lines such as:
root:x:0:0:root:/root:/bin/bash
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
then that IFS value would allow it to load those values into the right variables (in other words, it would split on ":")
The default value for IFS is inherited from your shell, and it usually contains the space and the TAB character and maybe some other stuff. When you only read into one single variable ($line, in your case). IFS isn't applied but when you ever change a read statement and add another variable, word splitting starts taking effect and the lack of a local IFS= value will make the exact same script behave very different in different situations. As such it tends to be a good habbit to control it at all times.
The same goes for quoting your variables like "$arg" or "${arg}" , instead of $arg . It doesn't matter when ARG="hello"; but once the value starts containing spaces suddenly all sorts of things can act different; suprises are never a good thing.
i stumbled in a confusing way of conditionally returning value based on variable. I would like to check if process is successful then echo "process success", but if it's failed, i want to check specific error message then return the error message,
ERRMSG="$(cd /nonexist 2>&1)"
if [ $? -ne 0 ]
then
if [ -z "$ERRMSG|grep -o 'No such file or directory'|head -1" ]
then
echo "empty" >> $FQLOGNAME
else
echo $ERRMSG|grep -o 'No such file or directory'|head -1 >> $FQLOGNAME
fi
else
echo "success" >> $FQLOGNAME
fi
Please advice,
Thanks
You don't need to use grep to check if a string contains a substring. The built-in pattern matching in Bash is sufficient. This code should do something close to what you want:
if ERRMSG=$(cd /nonexist 2>&1) ; then
echo 'process success'
elif [[ $ERRMSG == *'No such file or directory'* ]] ; then
echo 'No such file or directory'
else
echo 'empty'
fi >> "$FQLOGNAME"
See the Conditional Constructs section of the Bash Reference Manual for details of the pattern matching capabilities of [[...]].
I've retained the ERRMSG and FQLOGNAME variables, but note that it's best to avoid ALL_UPPERCASE variable names. There is a danger that they will clash with environment variables or Bash builtin variables. See Correct Bash and shell script variable capitalization.
To find error messages defined by a pattern in multi-line error messages, and only print the first one, you can use regular expression matching (=~) in [[...]]. To provide a concrete example, this code assumes that error messages consist of 'ERROR' followed by one or more spaces followed by a decimal number:
# Example function for testing
function dostuff
{
printf 'Output line A\n'
printf 'Encountered ERROR 29\n' >&2
printf 'Output line B\n'
printf 'Encountered ERROR 105\n' >&2
printf 'Output line C\n'
return 1
}
# Regular expression matching an error string
readonly error_rx='ERROR +[0-9]+'
if ERRMSG=$(dostuff 2>&1) ; then
echo 'process success'
elif [[ $ERRMSG =~ $error_rx ]] ; then
printf '%s\n' "${BASH_REMATCH[0]}"
else
echo 'empty'
fi >> "$FQLOGNAME"
It appends 'ERROR 29' to the log file.
For more information about Bash's built-in regular expression matching see mklement0's answer to "How do I use a regex in a shell script?".
Make it simpler and easier:
if ! ERRMSG=$(cd /nonexist 2>&1); then
if <<<"$ERRMSG" grep -q 'No such file or directory'; then
# if the error string contains the message 'No such file or directory'
echo "empty" >> "$FQLOGNAME"
else
printf "Unhandled cd error: %s" "$ERRMSG" >> "$FQLOGNAME"
fi
else
echo "process success" >> "$FQLOGNAME"
fi
if statements checks for the return status of a COMMAND. [ or test is just a command, which return a status. The return status of assignment is the same as command status. What I mean, is that out=$(cmd); if [ "$?" -eq 0 ]; then is the same as if out=$(cmd); then.
Using HERE-strings is a bit better than echo "$string". Echo is not that much portable, better get used to printf "%s" "$string" which is a portable way. However HERE-strings puts additional EOF at the end of the stream, which sometimes breaks while read loops, but for most cases works ok.
Don't if [ -z "$(echo smth | grep ..)" ]; then. You can just check grep return status, just if echo smth | grep ...; then or with HERE-strings if <<<"smth" grep -q ...; then or if grep -q ... file; then. The -q option which has --quiet or --silent alternatives makes grep produce no output.
The quoting is not needed when assigning a variable from a single command substitution. tmp="$(...)" is just the same as tmp=$(...).
I have fully working code that obtains a list of file names from the '$testDir', removes the final 3 characters from each, runs a different script using the adjusted names, and displays a list of the edited names that had errors when used in the seperate script, until the letter 'q' is entered by the user.
Here is the code:
#!/bin/bash
# Declarations
declare -a testNames
declare -a errorNames
declare -a errorDescription
declare resultDir='../Results'
declare outputCheck=''
declare userEntry=''
declare -i userSelect
# Obtain list of files in $resultDir and remove the last 3 chars from each file name
for test in `ls $resultDir`; do
testNames+=("${test::-3}")
done
# Run 'checkFile.sh' script for each adjusted file name and add name and result to apporopriate arrays if 'checkFile.sh' script fails
for f in "${testNames[#]}"; do
printf '%s' "$f: "
outputCheck=$(./scripts/checkFile.sh -v "${f}" check)
if [[ $outputCheck != "[0;32mNo errors found.[0m" ]];
then
errorNames+=("$f")
errorDescription+=("$outputCheck")
echo -e '\e[31mError(s) found\e[0m'
else
printf '%s\n' "$outputCheck"
fi
done
#Prompts user to save errors, if any are present
if [ "${#errorNames[#]}" != 0 ];
then
until [[ $userEntry = "q" ]]; do
echo "The following tests had errors:"
for(( i=1; i<=${#errorNames[#]}; i++ )); do
echo -e $i: "\e[31m${errorNames[i-1]}\e[0m"
done
echo "Enter the corresponding number in the list to save the errors found or enter 'q' to quit"
read -r userEntry
numInput=$userEntry
if [ $numInput -gt 0 -a $numInput -le ${#errorNames[#]} ];
then
mkdir -p ./Errors
echo -e "File" "\e[96m${errorNames[$userEntry-1]}""_Error_Info.txt\e[0m created in the 'Errors' folder which contains details of the error(s)"
echo "${errorDescription[$userEntry-1]}" > "./Errors/""${errorNames[$userEntry-1]}""_Error_Info.txt"
fi
done
echo 'Successfully Quit'
exit $?
fi
echo 'No errors found'
exit $?
As someone who is new to Bash and programming concepts, I would really appreciate any suggested improvements to this code. In particular, the script needs to run quickly as the 'checkFile.sh' script takes a long time to run itself. I have a feeling the way I have written the code is not concise either.
I'm writing my first Bash script, I have some experience with C and C# so I think the logic of the program is correct, it's just the syntax is so complicated because apparently there are many different ways to write the same thing!
Here is the script, it simply checks if the argument (string) is contained in a certain file. If so it stores each line of the file in an array and writes an item of the array in a file. I'm sure there must be easier ways to achieve that but I want to do some practice with bash loops
#!/bin/bash
NOME=$1
c=0
#IF NAME IS FOUND IN THE PHONEBOOK THEN STORE EACH LINE OF THE FILE INTO ARRAY
#ONCE THE ARRAY IS DONE GET THE INDEX OF MATCHING NAME AND RETURN ARRAY[INDEX+1]
if grep "$NOME" /root/phonebook.txt ; then
echo "CREATING ARRAY"
while read line
do
myArray[$c]=$line # store line
c=$(expr $c + 1) # increase counter by 1
done < /root/phonebook.txt
else
echo "Name not found"
fi
c=0
for i in myArray;
do
if myArray[$i]="$NOME" ; then
echo ${myArray[i+1]} >> /root/numbertocall.txt
fi
done
This code returns the only the second item of myArray (myArray[2]) or the second line of the file, why?
The first part (where you build the array) looks ok, but the second part has a couple of serious errors:
for i in myArray; -- this executes the loop once, with $i set to "myArray". In this case, you want $i to iterate over the indexes of myArray, so you need to use
for i in "${!myArray[#]}"
or
for ((i=0; i<${#a[#]}; i++))
(although I generally prefer the first, since it'll work with noncontiguous and associative arrays).
Also, you don't need the ; unless do is on the same line (in shell, ; is mostly equivalent to a line break so having a semicolon at the end of a line is redundant).
if myArray[$i]="$NOME" ; then -- the if statement takes a command, and will therefore treat myArray[$i]="$NOME" as an assignment command, which is not at all what you wanted. In order to compare strings, you could use the test command or its synonym [
if [ "${myArray[i]}" = "$NOME" ]; then
or a bash conditional expression
if [[ "${myArray[i]}" = "$NOME" ]]; then
The two are very similar, but the conditional expression has much cleaner syntax (e.g. in a test command, > redirects output, while \> is a string comparison; in [[ ]] a plain > is a comparison).
In either case, you need to use an appropriate $ expression for myArray, or it'll be interpreted as a literal. On the other hand, you don't need a $ before the i in "${myArray[i]}" because it's in a numeric expression context and therefore will be expanded automatically.
Finally, note that the spaces between elements are absolutely required -- in shell, spaces are very important delimiters, not just there for readability like they usually are in c.
1.-This is what you wrote with small adjustments
#!/bin/bash
NOME=$1
#IF NAME IS FOUND IN THE PHONE-BOOK **THEN** READ THE PHONE BOOK LINES INTO AN ARRAY VARIABLE
#ONCE THE ARRAY IS COMPLETED, GET THE INDEX OF MATCHING LINE AND RETURN ARRAY[INDEX+1]
c=0
if grep "$NOME" /root/phonebook.txt ; then
echo "CREATING ARRAY...."
IFS= while read -r line #IFS= in case you want to preserve leading and trailing spaces
do
myArray[c]=$line # put line in the array
c=$((c+1)) # increase counter by 1
done < /root/phonebook.txt
for i in ${!myArray[#]}; do
if myArray[i]="$NOME" ; then
echo ${myArray[i+1]} >> /root/numbertocall.txt
fi
done
else
echo "Name not found"
fi
2.-But you can also read the array and stop looping like this:
#!/bin/bash
NOME=$1
c=0
if grep "$NOME" /root/phonebook.txt ; then
echo "CREATING ARRAY...."
readarray myArray < /root/phonebook.txt
for i in ${!myArray[#]}; do
if myArray[i]="$NOME" ; then
echo ${myArray[i+1]} >> /root/numbertocall.txt
break # stop looping
fi
done
else
echo "Name not found"
fi
exit 0
3.- The following improves things. Supposing a)$NAME matches the whole line that contains it and b)there's always one line after a $NOME found, this will work; if not (if $NOME can be the last line in the phone-book), then you need to do small adjustments.
!/bin/bash
PHONEBOOK="/root/phonebook.txt"
NUMBERTOCALL="/root/numbertocall.txt"
NOME="$1"
myline=""
myline=$(grep -A1 "$NOME" "$PHONEBOOK" | sed '1d')
if [ -z "$myline" ]; then
echo "Name not found :-("
else
echo -n "$NOME FOUND.... "
echo "$myline" >> "$NUMBERTOCALL"
echo " .... AND SAVED! :-)"
fi
exit 0
I have a file that contains directory names:
my_list.txt :
/tmp
/var/tmp
I'd like to check in Bash before I'll add a directory name if that name already exists in the file.
grep -Fxq "$FILENAME" my_list.txt
The exit status is 0 (true) if the name was found, 1 (false) if not, so:
if grep -Fxq "$FILENAME" my_list.txt
then
# code if found
else
# code if not found
fi
Explanation
Here are the relevant sections of the man page for grep:
grep [options] PATTERN [FILE...]
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched.
-x, --line-regexp
Select only those matches that exactly match the whole line.
-q, --quiet, --silent
Quiet; do not write anything to standard output. Exit immediately with zero status if any match is found, even if an error was detected. Also see the -s or --no-messages option.
Error handling
As rightfully pointed out in the comments, the above approach silently treats error cases as if the string was found. If you want to handle errors in a different way, you'll have to omit the -q option, and detect errors based on the exit status:
Normally, the exit status is 0 if selected lines are found and 1 otherwise. But the exit status is 2 if an error occurred, unless the -q or --quiet or --silent option is used and a selected line is found. Note, however, that POSIX only mandates, for programs such as grep, cmp, and diff, that the exit status in case of error be greater than 1; it is therefore advisable, for the sake of portability, to use logic that tests for this general condition instead of strict equality with 2.
To suppress the normal output from grep, you can redirect it to /dev/null. Note that standard error remains undirected, so any error messages that grep might print will end up on the console as you'd probably want.
To handle the three cases, we can use a case statement:
case `grep -Fx "$FILENAME" "$LIST" >/dev/null; echo $?` in
0)
# code if found
;;
1)
# code if not found
;;
*)
# code if an error occurred
;;
esac
Regarding the following solution:
grep -Fxq "$FILENAME" my_list.txt
In case you are wondering (as I did) what -Fxq means in plain English:
F: Affects how PATTERN is interpreted (fixed string instead of a regex)
x: Match whole line
q: Shhhhh... minimal printing
From the man file:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched.
(-F is specified by POSIX.)
-x, --line-regexp
Select only those matches that exactly match the whole line. (-x is specified by POSIX.)
-q, --quiet, --silent
Quiet; do not write anything to standard output. Exit immediately with zero status if any match is
found, even if an error was detected. Also see the -s or --no-messages option. (-q is specified by
POSIX.)
Three methods in my mind:
1) Short test for a name in a path (I'm not sure this might be your case)
ls -a "path" | grep "name"
2) Short test for a string in a file
grep -R "string" "filepath"
3) Longer bash script using regex:
#!/bin/bash
declare file="content.txt"
declare regex="\s+string\s+"
declare file_content=$( cat "${file}" )
if [[ " $file_content " =~ $regex ]] # please note the space before and after the file content
then
echo "found"
else
echo "not found"
fi
exit
This should be quicker if you have to test multiple string on a file content using a loop for example changing the regex at any cicle.
Easiest and simplest way would be:
isInFile=$(cat file.txt | grep -c "string")
if [ $isInFile -eq 0 ]; then
#string not contained in file
else
#string is in file at least once
fi
grep -c will return the count of how many times the string occurs in the file.
Simpler way:
if grep "$filename" my_list.txt > /dev/null
then
... found
else
... not found
fi
Tip: send to /dev/null if you want command's exit status, but not outputs.
Here's a fast way to search and evaluate a string or partial string:
if grep -R "my-search-string" /my/file.ext
then
# string exists
else
# string not found
fi
You can also test first, if the command returns any results by running only:
grep -R "my-search-string" /my/file.ext
grep -E "(string)" /path/to/file || echo "no match found"
-E option makes grep use regular expressions
If I understood your question correctly, this should do what you need.
you can specifiy the directory you would like to add through $check variable
if the directory is already in the list, the output is "dir already listed"
if the directory is not yet in the list, it is appended to my_list.txt
In one line: check="/tmp/newdirectory"; [[ -n $(grep "^$check\$" my_list.txt) ]] && echo "dir already listed" || echo "$check" >> my_list.txt
The #Thomas's solution didn't work for me for some reason but I had longer string with special characters and whitespaces so I just changed the parameters like this:
if grep -Fxq 'string you want to find' "/path/to/file"; then
echo "Found"
else
echo "Not found"
fi
Hope it helps someone
If you just want to check the existence of one line, you do not need to create a file. E.g.,
if grep -xq "LINE_TO_BE_MATCHED" FILE_TO_LOOK_IN ; then
# code for if it exists
else
# code for if it does not exist
fi
My version using fgrep
FOUND=`fgrep -c "FOUND" $VALIDATION_FILE`
if [ $FOUND -eq 0 ]; then
echo "Not able to find"
else
echo "able to find"
fi
I was looking for a way to do this in the terminal and filter lines in the normal "grep behaviour". Have your strings in a file strings.txt:
string1
string2
...
Then you can build a regular expression like (string1|string2|...) and use it for filtering:
cmd1 | grep -P "($(cat strings.txt | tr '\n' '|' | head -c -1))" | cmd2
Edit: Above only works if you don't use any regex characters, if escaping is required, it could be done like:
cat strings.txt | python3 -c "import re, sys; [sys.stdout.write(re.escape(line[:-1]) + '\n') for line in sys.stdin]" | ...
A grep-less solution, works for me:
MY_LIST=$( cat /path/to/my_list.txt )
if [[ "${MY_LIST}" == *"${NEW_DIRECTORY_NAME}"* ]]; then
echo "It's there!"
else
echo "its not there"
fi
based on:
https://stackoverflow.com/a/229606/3306354
grep -Fxq "String to be found" | ls -a
grep will helps you to check content
ls will list all the Files
Slightly similar to other answers but does not fork cat and entries can contain spaces
contains() {
[[ " ${list[#]} " =~ " ${1} " ]] && echo 'contains' || echo 'does not contain'
}
IFS=$'\r\n' list=($(<my_list.txt))
so, for a my_list.txt like
/tmp
/var/tmp
/Users/usr/dir with spaces
these tests
contains '/tmp'
contains '/bin'
contains '/var/tmp'
contains '/Users/usr/dir with spaces'
contains 'dir with spaces'
return
exists
does not exist
exists
exists
does not exist
if grep -q "$Filename$" my_list.txt
then
echo "exist"
else
echo "not exist"
fi