I have a requirement to check the syntax of some config files.
The format of the config is as below:
[sect1]
sect1file1
sect1file2
[sect1_ends]
[sect2]
sect2file1
sect2file2
[sect2_ends]
My requirement is to check the for start of sect1 which is inside square brackets [sect1], then check that the files sect1file1 and sect1file2 exist, then check for the end of sect1 by reading sect1_ends inside square braces [sect1_ends]. Then repeat the same for sect2, and so on.
There is already a set of section names which are permitted. My objective is to check whether the section names are in the list, and whether the syntax is without any error.
I tried using
perl -lne 'print $1 while (/^\[(.*?)\]$/g)' <config filename>
but I'm not sure how to check and go through the file.
I am happy to see you have tried. Try again with this prototype:
while read -r line; do
if [ ${#line} -eq 0 ]; then
continue # ignore empty lines
fi
if [[ "${line}" = \[*\] ]]; then
echo "Line with [...]"
if [ -n "${inSection}" ]; then
if [ "${line}" = "${inSection/]/_ends]}" ]; then
echo "End of section"
unset inSection
else
echo "Invalid endtag ${line} while processing ${inSection}"
exit 1
fi
else
echo "Start of new section ${line}"
inSection="${line}"
fi
else
if [ -f "${line}" ]; then
echo "OK file ${line}"
else
echo "NOK file ${line}"
fi
fi
done < inputfile
Your solution looks good, but -n reads lines from standard input (STDIN). You need to feed your config file into STDIN to pass it to your script:
perl -lne 'print $1 while (/^\[(.*?)\]$/g)' <config.ini
Alternate option would be using -p.
Related
I wanted to make a bash script to check if each line of multi-line string already exist in a file.
I had already wrote some code, but I'm not sure if it will work:
str="
this is
a multiple
line
string"
while read -r f; do
while read -r s: do
if [ $r == $s ]; then break; fi
done
done << some_file.txt
any ideas to get it working ?
Here's a trick if you can load the whole file twice to memory:
str="
this is
a multiple
line
string"
# Read the whole file in variable
content=$(<some_file.txt)
# Replace str to nothing in content
repr=${content/${str}}
# Check if it's the same.
if [[ "$content" != "$repr" ]]; then
# If it's not, means content contains str and it was removed.
echo "Contains"
fi
It can be a perl one-liner:
if perl -0777 -sne '/$text/ or exit 1' -- -text="$str" "$file"
then
echo Found
else
echo not found
fi
The -0777 option slurps the whole file into memory
This could be a solution in bash:
Reading the string lines into an array.
Then, for each string line in the array:
Checking the file contains a whole line equal to that string line.
Stop walking the array once a check fails.
#!/bin/bash
file="$1"
names="
this is
a multiple
line
string"
# Read string lines into an array
readarray -t <<< $names
# Walk array of lines
all_strings_found=true
for string in "${MAPFILE[#]}"; do
if ! grep -q "^$string$" $file; then
echo "'"$string"'" "not found"
all_strings_found=false
break
fi
done
if [ "$all_strings_found" == true ]; then
echo "---> All strings found"
else
echo "---> NOT all strings found"
fi
I like case statements for this.
$ cat script
str1="
this is
a multiple
line
string"
str2="$1"
f="$(<$0)"
case "$f" in
*"$str1"*) echo "String 1 exists" ;;
*) echo "String 1 not found" ;;
esac
case "$f" in
*"$str2"*) echo "String 2 exists" ;;
*) echo "String 2 not found" ;;
esac
$ ./script esac
String 1 exists
String 2 exists
$ ./script foo
String 1 exists
String 2 not found
I'm writing a bash script to read a set of files line by line and perform some edits. To begin with, I'm simply trying to move the files to backup locations and write them out as-is, to test the script is working. However, it is failing to copy the last line of each file. Here is the snippet:
while IFS= read -r line
do
echo "Line is ***$line***"
echo "$line" >> $POM
done < $POM.backup
I obviously want to preserve whitespace when I copy the files, which is why I have set the IFS to null. I can see from the output that the last line of each file is being read, but it never appears in the output.
I've also tried an alternative variation, which does print the last line, but adds a newline to it:
while IFS= read -r line || [ -n "$line" ]
do
echo "Line is ***$line***"
echo "$line" >> $POM
done < $POM.backup
What is the best way to do this do this read-write operation, to write the files exactly as they are, with the correct whitespace and no newlines added?
The command that is adding the line feed (LF) is not the read command, but the echo command. read does not return the line with the delimiter still attached to it; rather, it strips the delimiter off (that is, it strips it off if it was present in the line, IOW, if it just read a complete line).
So, to solve the problem, you have to use echo -n to avoid adding back the delimiter, but only when you have an incomplete line.
Secondly, I've found that when providing read with a NAME (in your case line), it trims leading and trailing whitespace, which I don't think you want. But this can be solved by not providing a NAME at all, and using the default return variable REPLY, which will preserve all whitespace.
So, this should work:
#!/bin/bash
inFile=in;
outFile=out;
rm -f "$outFile";
rc=0;
while [[ $rc -eq 0 ]]; do
read -r;
rc=$?;
if [[ $rc -eq 0 ]]; then ## complete line
echo "complete=\"$REPLY\"";
echo "$REPLY" >>"$outFile";
elif [[ -n "$REPLY" ]]; then ## incomplete line
echo "incomplete=\"$REPLY\"";
echo -n "$REPLY" >>"$outFile";
fi;
done <"$inFile";
exit 0;
Edit: Wow! Three excellent suggestions from Charles Duffy, here's an updated script:
#!/bin/bash
inFile=in;
outFile=out;
while { read -r; rc=$?; [[ $rc -eq 0 || -n "$REPLY" ]]; }; do
if [[ $rc -eq 0 ]]; then ## complete line
echo "complete=\"$REPLY\"";
printf '%s\n' "$REPLY" >&3;
else ## incomplete line
echo "incomplete=\"$REPLY\"";
printf '%s' "$REPLY" >&3;
fi;
done <"$inFile" 3>"$outFile";
exit 0;
After review i wonder if :
{
line=
while IFS= read -r line
do
echo "$line"
line=
done
echo -n "$line"
} <$INFILE >$OUTFILE
is juts not enough...
Here my initial proposal :
#!/bin/bash
INFILE=$1
if [[ -z $INFILE ]]
then
echo "[ERROR] missing input file" >&2
exit 2
fi
OUTFILE=$INFILE.processed
# a way to know if last line is complete or not :
lastline=$(tail -n 1 "$INFILE" | wc -l)
if [[ $lastline == 0 ]]
then
echo "[WARNING] last line is incomplete -" >&2
fi
# we add a newline ANYWAY if it was complete, end of file will be seen as ... empty.
echo | cat $INFILE - | {
first=1
while IFS= read -r line
do
if [[ $first == 1 ]]
then
echo "First Line is ***$line***" >&2
first=0
else
echo "Next Line is ***$line***" >&2
echo
fi
echo -n "$line"
done
} > $OUTFILE
if diff $OUTFILE $INFILE
then
echo "[OK]"
exit 0
else
echo "[KO] processed file differs from input"
exit 1
fi
Idea is to always add a newline at the end of file and to print newlines only BETWEEN lines that are read.
This should work for quite all text files given they are not containing 0 byte ie \0 character, in which case 0 char byte will be lost.
Initial test can be used to decided whether an incomplete text file is acceptable or not.
Add a new line if line is not a line. Like this:
while IFS= read -r line
do
echo "Line is ***$line***";
printf '%s' "$line" >&3;
if [[ ${line: -1} != '\n' ]]
then
printf '\n' >&3;
fi
done < $POM.backup 3>$POM
I need some help to write a script for the following scenario.
The requirement is, based on the number of configuration files(*.cfg) inside a given directory, I need load all the configuration file names with out the file extension into an array. If there is only one configuration file in the directory, then array will be assigned the value "" (not the name of the only available configuration file)
I am trying to do this using logical operators. This is what i have tried so far.
[`ls *.cfg |wc -l`] || code_to_initialize_array;
My problem here is that, how do I integrate the case where i have only one configuration file.
Short code:
#!/bin/bash
array=(*.cfg)
array=("${array[#]%.cfg}")
[ ${#array[#]} -eq 1 ] && array=""
#!/bin/bash
config=(*.cfg) #glob instead ls usage
num=${#config[#]}
case $num in
0)
echo "No config file"
;;
1)
echo "Only one config file"
;;
*)
code_to_initialize_array
;;
esac
You can have this example script for your requirement. It's detailed and variable names are long but you could have your own customizations. Using readarray is safer than A=($(...)) since it doesn't depend on IFS and is not subject to pathname expansion.
#!/bin/bash
DIR=/path/to/somewhere
readarray -t FILES < <(compgen -G "${DIR%/}/*.cfg") ## Store matches to array.
FILES_COUNT=${#FILES[#]} ## Match count.
FILES_NAMES=("${FILES[#]##*/}") ## No directory parts.
FILES_NAMES_WITHOUT_CFG=("${FILES_NAMES[#]%.cfg}") ## No .cfg extension.
if [[ FILES_COUNT -gt 0 ]]; then
printf "File: %s\n" "${FILES[#]}"
printf "Name: %s\n" "${FILES_NAMES[#]}"
printf "Name (no .cfg): %s\n" "${FILES_NAMES_WITHOUT_CFG[#]}"
printf "Total: %d\n" "$FILES_COUNT"
fi
Note that each entry has the same index number. So ${FILES[1]} is ${FILES_NAMES[1]} and also ${FILES_NAMES_WITHOUT_CFG[1]}. Entries begin with index 0.
You can also have other details through this:
if [[ FILES_COUNT -gt 0 ]]; then
for I in "${!FILES[#]}"; do
printf "File: %s\n" "${FILES[I]}"
printf "Name: %s\n" "${FILES_NAMES[I]}"
printf "Name (no .sh): %s\n" "${FILES_NAMES_WITHOUT_CFG[I]}"
printf "Index number: $I\n\n"
done
printf "Total: %d\n" "$FILES_COUNT"
fi
I've always liked abusing a for loop for a situation like this.
for x in *.cfg; do
[[ -f $x ]] && code_to_initialize_array
break
The explicit break means the loop iterates only once, no matter how many .cfg files you have. If you have none, *.cfg will be treated literally, so the [[ -f $x ]] checks if the "first" cfg file actually exists before trying to run code_to_initialize_array.
I'm reading a string from file by a shell script.
it goes like this:
count = 0
while read LINE
do
count++
if [ "$LINE" == "NONE" ]
then
echo "state is NONE"
else
if [ "$LINE" == "PLAYING" ]
then
echo "state is PLAYING"
fi
fi
done<$FILENAME
this is what I read from the file, and how I handle it, now I want to do something else if the file not found, is there anyway to do that?
for example:
if[ file not found]
then
do something
fi
if [ -f path_to_file ]
then
echo "file was found"
else
echo "file was not found"
fi
You'd better start you script with a condition like:
if [ ! -f /your/file ]; then
echo "file not found"
else
...
proceed with your `while`, etc.
...
fi
Since you are trying to read the file, then maybe you should test to see if the file exists, and if you have read access:
if [[ ! -f $FILENAME ]] || [[ ! -r $FILENAME ]]
then
# do stuff
fi
The -f tests to see if $FILENAME is a regular file, -r tests to see if you (the current user) has read access.
This works just fine
if [[ -e img.png ]]
then
echo "exist"
else
echo "doesn't exist"
fi
but what if I know that there might be imgage with name img but I do not know if the file is .jpg , .gif , .jpeg , .tff and so on.
I do not care what is the extension I just want to know if there is a file with name 'img'
How can I do this ?
You can do:
files=$(ls img.* 2> /dev/null | wc -l)
if [ "$files" != "0" ]
then
echo "exist"
else
echo "doesn't exist"
fi
You can use the following scripts
files=`ls img.* 2>/dev/null`
if [ "$files" -a ${#files[#]} ]; then
echo "exist"
else
echo "doesn't exist"
fi
In this snippet, you use ls img.* to list all the files in current working directory whose name match the pattern img.*.
The result is stored into an array named files.
Then check size of the array to determine whether required files exist.
See this for how to get length of the array.
Something like this should do the job:
if [[ $(ls img.*) ]]; then
echo "file exist";
else
echo "file does not exist";
fi
I recommend to have a look at bash's pattern matching:
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_04_03.html
Without any external command:
$ for i in img.*
> do
> [ -f $i ] && echo exist || echo not exist
> break
> done
Check if any file is present. If present print exist , else not exist, and break immediately. The "-f" check is needed because if no files are present, still the loop runs once with i as "img.*" itself.
shopt -s nullglob
files=( img.* )
if (( ${#files[#]} == 0 )); then
echo "there are no 'img' files"
fi
If you don't use nullglob then, if there are no such files, the array will have 1 element, the literal string "img.*".