I have a text file.
I want to get lines starting with specific format.
I just want to get lines that have x/x/x format. x is a number.
But this regex is not working. It is always giving no match :
while read line
do
regex="\d+\/\d+\/\d+"
if [[ ${line} =~ ${regex} ]]; then
echo ${line}
else
echo "no match : ${line}"
fi
done <${textFileName}
File is :
Don't use bash if you can use a better tool:
grep -E '^[[:digit:]]+/[[:digit:]]+/[[:digit:]]+' "${textFileName}"
But if you have to use bash:
while IFS= read -r line
do
if [[ "$line}" =~ ^[[:digit:]]+/[[:digit:]]+/[[:digit:]]+ ]]; then
echo -- "$line"
else
echo "no match: $line"
fi
done < "$textFileName"
\d is not valid regex(3).
Related
I have tried all the solutions available on stack overflow, but when I use if condition with with it always results true.
I need to find a line in the file and see if it doesn't exit then insert the line in that file, but it always results that the line already exists.
Here is my script
isInFile=$(grep -q '^export' /etc/bashrc)
if [[ $isInFile == 0 ]];
then
echo "line is not present";
echo "export PROMPT_COMMAND='RETRN_VAL=\$?;logger -p local6.debug \"\$(whoami) [\$\$]: \$(history 1 | sed \"s/^[ ]*[0-9]\+[ ]*//\" )\"'" >> /etc/bashrc;
source /etc/bashrc;
else
echo "line is in the file";
fi
It always says that
line is in the file
The if statement branches based on the exit status of the command it's given. [[ is just one command you can use, it's not mandatory syntax. At an interactive prompt, enter help if
Do this:
if grep -q '^export' /etc/bashrc
then
# exit status of grep is zero: the pattern DOES MATCH the file
echo "line is in the file";
else
# exit status of grep is non-zero: the pattern DOES NOT MATCH the file
echo "line is not present";
echo "export PROMPT_COMMAND='RETRN_VAL=\$?;logger -p local6.debug \"\$(whoami) [\$\$]: \$(history 1 | sed \"s/^[ ]*[0-9]\+[ ]*//\" )\"'" >> /etc/bashrc;
source /etc/bashrc;
fi
I see 2 issues in your code:
if [[ $isInFile == 0 ]]; --If condition should not terminate with ;. Remove that.
The expression you are checking is always an empty string. Try echo $isInFile. What you are checking is output of the command, not its return value. Instead, you should remove -q from your grep expression and check if the output is empty or not.
Following code should work:
isInFile=$(grep '^export' /etc/bashrc)
if [ -z "$isInFile" ]
then
echo "line is not present";
echo "export PROMPT_COMMAND='RETRN_VAL=\$?;logger -p local6.debug \"\$(whoami) [\$\$]: \$(history 1 | sed \"s/^[ ]*[0-9]\+[ ]*//\" )\"'" >> /etc/bashrc;
source /etc/bashrc;
else
echo "line is in the file";
fi
-z check for emptiness of variable.
I am new to bash scripting, I have a file and i want to check each line but it is not working.
My bach script code
declare -a arr
for (( i=0; i<${len}; i++ ))
do
if [[ ${arr[$i]} =~ ^[0-9]+$ ]]
then
echo ${arr[$i]}" numbr"
else
echo "No match"
fi
done
Why only last line is match ? is there any space or line break issue ? suggest me some solution
Your file is saved in "DOS" format, with \r\n line endings.
To convert to "unix" format:
dos2unix file.txt
# or, if that's not installed
sed -i 's/\r$//' file.txt
tangentially, I find the POSIX character class can describe what you want to match:
[[ ${arr[$i]} =~ ^[[:digit:]]+$ ]] && echo "${arr[i]} numbr"
[[ ${arr[$i]} =~ ^[[:alpha:]]+$ ]] && echo "${arr[i]} ltr only"
[[ ${arr[$i]} =~ ^[[:alnum:]]+$ ]] && echo "${arr[i]} both"
Your issue is MS-DOS line endings of $'\r\n' instead of $'\n' only.
You can remove the offending $'\n' on-the-fly like this:
mapfile -t arr < <(tr -d '\r' <file.txt)
Other than that, I suggest you check you script with https://shellcheck.net/ as it has some issues
I would like to check in bash if line starts with digit then put those lines in a separate file. I tried ^[[0-9]] but it doesn't work.
Here is the code which I tried:
myFailedFile=/tmp/1.txt
myTestsFile:
177711 TESTING ...yahoo.tests.calendar.resources.scheduler.1
854756 TESTING ...yahoo.tests.calendar.resources.scheduler.2
* 2102637 DONE ...yahoo.tests.mail.contacts.add.3
while read line
do
if ( [[ ${line} = "^[[0-9]]"* ]] && [[ ${line} = *".tests."* ]] ); then
echo -e "${line}\r" >> ${myFailedFile}
fi
done <"${myTestsFile}"
Expected output of myFailedFile:
177711 TESTING ...yahoo.tests.calendar.resources.scheduler.1
854756 TESTING ...yahoo.tests.calendar.resources.scheduler.2
The correct operator to use regex in Bash script is =~. Moreoever you don't need to double [ in range of characters. Try this:
while read line
do
if ( [[ ${line} =~ ^[0-9] ]] && [[ ${line} = *".tests."* ]] ); then
echo -e "${line}\r" >> ${myFailedFile}
fi
done <"${myTestsFile}"
Edit:
But you don't need a Bash loop for that job. You can do it with a sed one-liner:
sed '/^[0-9].*\.tests\./!d' "${myTestsFile}" > myFailedFile
Explanations(from right to left):
!d: do not delete
/^[0-9].*\.tests\./: all lines that start with one or more digits and that contain .tests. string
Without using regex you can use glob as this:
[[ $line = [0-9]* ]] && [[ $line = *".tests."* ]]
[0-9]* matches a string start start with digits
*".tests."* matches a string that contains .tests.
you can use
if [[ ${line} =~ ^[0-9] && ${line} =~ ".tests." ]] ; then
or
if [[ ${line} == [0-9]* && ${line} == *".tests."* ]] ; then
I'm writing a bash script to read a set of files line by line and perform some edits. To begin with, I'm simply trying to move the files to backup locations and write them out as-is, to test the script is working. However, it is failing to copy the last line of each file. Here is the snippet:
while IFS= read -r line
do
echo "Line is ***$line***"
echo "$line" >> $POM
done < $POM.backup
I obviously want to preserve whitespace when I copy the files, which is why I have set the IFS to null. I can see from the output that the last line of each file is being read, but it never appears in the output.
I've also tried an alternative variation, which does print the last line, but adds a newline to it:
while IFS= read -r line || [ -n "$line" ]
do
echo "Line is ***$line***"
echo "$line" >> $POM
done < $POM.backup
What is the best way to do this do this read-write operation, to write the files exactly as they are, with the correct whitespace and no newlines added?
The command that is adding the line feed (LF) is not the read command, but the echo command. read does not return the line with the delimiter still attached to it; rather, it strips the delimiter off (that is, it strips it off if it was present in the line, IOW, if it just read a complete line).
So, to solve the problem, you have to use echo -n to avoid adding back the delimiter, but only when you have an incomplete line.
Secondly, I've found that when providing read with a NAME (in your case line), it trims leading and trailing whitespace, which I don't think you want. But this can be solved by not providing a NAME at all, and using the default return variable REPLY, which will preserve all whitespace.
So, this should work:
#!/bin/bash
inFile=in;
outFile=out;
rm -f "$outFile";
rc=0;
while [[ $rc -eq 0 ]]; do
read -r;
rc=$?;
if [[ $rc -eq 0 ]]; then ## complete line
echo "complete=\"$REPLY\"";
echo "$REPLY" >>"$outFile";
elif [[ -n "$REPLY" ]]; then ## incomplete line
echo "incomplete=\"$REPLY\"";
echo -n "$REPLY" >>"$outFile";
fi;
done <"$inFile";
exit 0;
Edit: Wow! Three excellent suggestions from Charles Duffy, here's an updated script:
#!/bin/bash
inFile=in;
outFile=out;
while { read -r; rc=$?; [[ $rc -eq 0 || -n "$REPLY" ]]; }; do
if [[ $rc -eq 0 ]]; then ## complete line
echo "complete=\"$REPLY\"";
printf '%s\n' "$REPLY" >&3;
else ## incomplete line
echo "incomplete=\"$REPLY\"";
printf '%s' "$REPLY" >&3;
fi;
done <"$inFile" 3>"$outFile";
exit 0;
After review i wonder if :
{
line=
while IFS= read -r line
do
echo "$line"
line=
done
echo -n "$line"
} <$INFILE >$OUTFILE
is juts not enough...
Here my initial proposal :
#!/bin/bash
INFILE=$1
if [[ -z $INFILE ]]
then
echo "[ERROR] missing input file" >&2
exit 2
fi
OUTFILE=$INFILE.processed
# a way to know if last line is complete or not :
lastline=$(tail -n 1 "$INFILE" | wc -l)
if [[ $lastline == 0 ]]
then
echo "[WARNING] last line is incomplete -" >&2
fi
# we add a newline ANYWAY if it was complete, end of file will be seen as ... empty.
echo | cat $INFILE - | {
first=1
while IFS= read -r line
do
if [[ $first == 1 ]]
then
echo "First Line is ***$line***" >&2
first=0
else
echo "Next Line is ***$line***" >&2
echo
fi
echo -n "$line"
done
} > $OUTFILE
if diff $OUTFILE $INFILE
then
echo "[OK]"
exit 0
else
echo "[KO] processed file differs from input"
exit 1
fi
Idea is to always add a newline at the end of file and to print newlines only BETWEEN lines that are read.
This should work for quite all text files given they are not containing 0 byte ie \0 character, in which case 0 char byte will be lost.
Initial test can be used to decided whether an incomplete text file is acceptable or not.
Add a new line if line is not a line. Like this:
while IFS= read -r line
do
echo "Line is ***$line***";
printf '%s' "$line" >&3;
if [[ ${line: -1} != '\n' ]]
then
printf '\n' >&3;
fi
done < $POM.backup 3>$POM
I need to find strings matching some regexp pattern and represent the search result as array for iterating through it with loop ), do I need to use sed ? In general I want to replace some strings but analyse them before replacing.
Using sed and diff:
sed -i.bak 's/this/that/' input
diff input input.bak
GNU sed will create a backup file before substitutions, and diff will show you those changes. However, if you are not using GNU sed:
mv input input.bak
sed 's/this/that/' input.bak > input
diff input input.bak
Another method using grep:
pattern="/X"
subst=that
while IFS='' read -r line; do
if [[ $line = *"$pattern"* ]]; then
echo "changing line: $line" 1>&2
echo "${line//$pattern/$subst}"
else
echo "$line"
fi
done < input > output
The best way to do this would be to use grep to get the lines, and populate an array with the result using newline as the internal field separator:
#!/bin/bash
# get just the desired lines
results=$(grep "mypattern" mysourcefile.txt)
# change the internal field separator to be a newline
IFS=$'/n'
# populate an array from the result lines
lines=($results)
# return the third result
echo "${lines[2]}"
You could build a loop to iterate through the results of the array, but a more traditional and simple solution would just be to use bash's iteration:
for line in $lines; do
echo "$line"
done
FYI: Here is a similar concept I created for fun. I thought it would be good to show how to loop a file and such with this. This is a script where I look at a Linux sudoers file check that it contains one of the valid words in my valid_words array list. Of course it ignores the comment "#" and blank "" lines with sed. In this example, we would probably want to just print the Invalid lines only but this script prints both.
#!/bin/bash
# -- Inspect a sudoer file, look for valid and invalid lines.
file="${1}"
declare -a valid_words=( _Alias = Defaults includedir )
actual_lines=$(cat "${file}" | wc -l)
functional_lines=$(cat "${file}" | sed '/^\s*#/d;/^\s*$/d' | wc -l)
while read line ;do
# -- set the line to nothing "" if it has a comment or is empty line.
line="$(echo "${line}" | sed '/^\s*#/d;/^\s*$/d')"
# -- if not set to nothing "", check if the line is valid from our list of valid words.
if ! [[ -z "$line" ]] ;then
unset found
for each in "${valid_words[#]}" ;do
found="$(echo "$line" | egrep -i "$each")"
[[ -z "$found" ]] || break;
done
[[ -z "$found" ]] && { echo "Invalid=$line"; sleep 3; } || echo "Valid=$found"
fi
done < "${file}"
echo "actual lines: $actual_lines funtional lines: $functional_lines"