BASH =~ contains substring: "+" or a regex character behavior - linux

Curious to know why the following is not working for character: +
Characters "\", "(" and "*" makes sense i.e. * will expand to folders/files in current directory (during command line shell expansion) and similarly \ and ( will expect closing character to work but my understanding was "+" should have worked like "-" did.
PS: I know putting double quotes i.e. "${o}" in the IF statement, will work for all characters in my test case below. Using \${o} in IF statement with or without double quote will fail all the checks.
$ for o in - + \` ~ \~ , _ = / \\ ! # \# $ \$ % ^ \& \* \( \); do a="a${o}b${o}c";if [[ $a =~ ${o} ]]; then echo "${o} exists in $a and =~ works"; else echo -e "\ncharacter ${o} doesn't work with =~\n"; fi; done
- exists in a-b-c and =~ works
character + doesn't work with =~
` exists in a`b`c and =~ works
/home/ubuntu exists in a/home/ubuntub/home/ubuntuc and =~ works
~ exists in a~b~c and =~ works
, exists in a,b,c and =~ works
_ exists in a_b_c and =~ works
= exists in a=b=c and =~ works
/ exists in a/b/c and =~ works
character \ doesn't work with =~
! exists in a!b!c and =~ works
# exists in a#b#c and =~ works
# exists in a#b#c and =~ works
$ exists in a$b$c and =~ works
$ exists in a$b$c and =~ works
% exists in a%b%c and =~ works
^ exists in a^b^c and =~ works
& exists in a&b&c and =~ works
character * doesn't work with =~
character ( doesn't work with =~
) exists in a)b)c and =~ works

The fundamental misunderstanding behind this question is that =~ is a substring-search operator. It is not.
The right-hand side of =~ is evaluated as a POSIX ERE expression. =~ is thus a regex-matching operator, which happens to be frequently used for searches when the right-hand side is quoted to make its contents literal (or when this string is known to match only itself when interpreted as an ERE).
+, in regex, means "1-or-more of the preceding token" -- just as * means "0-or-more of the preceding token".
Thus, either [[ $foo =~ + ]] or [[ $foo =~ * ]] makes no sense, because these are checking for zero-or-more of a preceding token that doesn't exist at all.
Similarly, ( and ) have meaning in ERE as the beginning and end of a match group, so when they're given bare (unescaped/unquoted), they result in an invalid regex.
If you quote the expansion, by contrast, all characters contained will be treated as literal, rather than being treated as regular expression metacharacters, thus resulting in the presumably-intended behavior.
If you want to check whether a literal character is contained in a string, either quote it -- [[ $foo =~ "$o" ]] -- or use a glob-style pattern: [[ $foo = *"$o"* ]]

Related

Validate string for file characters including space

I have the following code:
if ! [[ $1 =~ ^[0-9a-zA-Z._-]+$ ]]; then
echo "argument contains characters not valid for name file"
fi
All I want is to validate if the string has valid characters for valid file name (I know I should add test for beginning of file, and length afterwards).
PROBLEM:
As such, it does not validate strings with spaces in it.
So I need to include space in the regex, but nothing of the following works:
[[ $1 =~ ^[0-9a-zA-Z ._-]+$ ]] >> syntax error
[[ $1 =~ ^[0-9a-zA-Z\t._-]+$ ]] >> still do not pass spaces
[[ $1 =~ ^[0-9a-zA-Z\s._-]+$ ]] >> still do not pass spaces
[[ $1 =~ "^[0-9a-zA-Z ._-]+$" ]] >> syntax error
I'm not sure what more to try.
So far, I come up with a quick and dirty thing:
myNewVar="${1// /}"
and do the tests with that, but that's far from elegant ...
You could use the [:blank:] character class:
re='^[[:alnum:][:blank:]._-]+$'
if ! [[ $1 =~ $re ]]; then
Notice that I've move the regex into a separate variable1, and also introduced the [:alnum:] character class.
Instead of regular expressions, you could use parameter expansion to remove allowed characters and see if anything is left:
if [[ -n ${1//[[:alnum:][:blank:]._-]} ]]; then
echo "illegal character found"
fi
1Mostly for portability reasons, but also to avoid quoting surprises (like the unquoted blank in your last example), see the BashGuide (section "Regular Expressions").

How to extract the text after a hyphen in bash

I have a string: dev/2.0 or dev/2.0-tymlez. How can I extract the string after the last - hyphen in bash? If there is no -, then the variable should be empty else tymlez and I want to store the result in $STRING. After that I would like to check the variable with:
if [ -z "$STRING" ]
then
echo "\$STRING is empty"
else
echo "\$STRING is NOT empty"
fi
Is that possible?
I recommend against calling your variable STRING. All-uppercase variables are used by the system (e.g. HOME) or the shell itself (e.g. PWD, RANDOM).
That said, you could do something like
string='dev/2.0-tymlez'
case "$string" in
*-*) string="${string##*-}";;
*) string='';;
esac
It's a bit clunky: It first checks whether there are any - at all, and if so, it removes the longest prefix matching *-; otherwise it just sets string to empty (because *- wouldn't have matched anything then).
You could use the =~ operator:
string="dev/2.0-tymlez"
[[ $string =~ -([^-]+)$ ]]; string=${BASH_REMATCH[1]}
BASH_REMATCH is a special array where the matches from [[ ... =~ ... ]] are assigned to.
You can use sed:
for string in "dev/2.0" "dev/2.0-1-2-3" "dev/2.0-tymlez"; do
string=$(sed 's/[^-]*[-]*//' <<< "${string}")
echo "string=[${string}]"
done
Result
string=[]
string=[1-2-3]
string=[tymlez]

What does "=~" do in the bash shell of linux

As the title said what does the "=~" do in the bash shell script running on Linux? I googled online and found that "==" checks equality and "!=" checks inequality. How about "=~"? I guess it may be some regular expression matching?
The =~ does a bash regular expression match
Example
$ [[ 45 =~ [0-9]+ ]] && echo "45 contains digits"
45 contains digits
$ [[ "hello" =~ [0-9]+ ]] && echo "hello doesnt contains digits"
$ [[ "hello" =~ [a-z]+ ]] && echo "hello contains alphabets"
hello contains alphabets
Yes, it's regular expression matching. It's in the bash manual:
An additional binary operator, =~, is available, with the same precedence as == and !=. When it is used, the string to the right of the operator is considered an extended regular expression and matched accordingly (as in regex3)).

Linux input pattern matching [duplicate]

String:
name#gmail.com
Checking for:
#
.com
My code
if [[ $word =~ "#" ]]
then
if [[ $word =~ ".com" || $word =~ ".ca" ]]
My problem
name#.com
The above example gets passed, which is not what I want. How do I check for characters (1 or more) between "#" and ".com"?
You can use a very very basic regex:
[[ $var =~ ^[a-z]+#[a-z]+\.[a-z]+$ ]]
It looks for a string being exactly like this:
at least one a-z char
#
at least one a-z char
.
at least one a-z char
It can get as complicated as you want, see for example Email check regular expression with bash script.
See in action
$ var="a#b.com"
$ [[ $var =~ ^[a-z]+#[a-z]+\.[a-z]+$ ]] && echo "kind of valid email"
kind of valid email
$ var="a#.com"
$ [[ $var =~ ^[a-z]+#[a-z]+\.[a-z]+$ ]] && echo "kind of valid email"
$
why not go for other tools like perl:
> echo "x#gmail.com" | perl -lne 'print $1 if(/#(.*?)\.com/)'
gmail
The glob pattern would be: [[ $word == ?*#?*.#(com|ca) ]]
? matches any single character and * matches zero or more characters
#(p1|p2|p3|...) is an extended globbing pattern that matches one of the given patterns. This requires:
shopt -s extglob
testing:
$ for word in #.com #a.ca a#.com a#b.ca a#b.org; do
echo -ne "$word\t"
[[ $word == ?*#?*.#(com|ca) ]] && echo matches || echo does not match
done
#.com does not match
#a.ca does not match
a#.com does not match
a#b.ca matches
a#b.org does not match

Linux command to do wild card matching

Is there any bash command to do something similar to:
if [[ $string =~ $pattern ]]
but that it works with simple wild cards (?,*) and not complex regular expressions ??
More info:
I have a config file (a sort of .ini-like file) where each line is composed of a wild card pattern and some other data.
For any given input string that my script receives, I have to find the first line in the config file where the wild card pattern matches the input string and then return the rest of the data in that line.
It's simple. I just need a way to match a string against wild card patterns and not RegExps since the patterns may contain dots, brackets, dashes, etc. and I don't want those to be interpreted as special characters.
The [ -z ${string/$pattern} ] trick has some pretty serious problems: if string is blank, it'll match all possible patterns; if it contains spaces, the test command will parse it as part of an expression (try string="x -o 1 -eq 1" for amusement). bash's [[ expressions do glob-style wildcard matching natively with the == operator, so there's no need for all these elaborate (and trouble-prone) tricks. Just use:
if [[ $string == $pattern ]]
There's several ways of doing this.
In bash >= 3, you have regex matching like you describe, e.g.
$ foo=foobar
$ if [[ $foo =~ f.ob.r ]]; then echo "ok"; fi
ok
Note that this syntax uses regex patterns, so it uses . instead of ? to match a single character.
If what you want to do is just test that the string contains a substring, there's more classic ways of doing that, e.g.
# ${foo/b?r/} replaces "b?r" with the empty string in $foo
# So we're testing if $foo does not contain "b?r" one time
$ if [[ ${foo/b?r/} = $foo ]]; then echo "ok"; fi
You can also test if a string begins or ends with an expression this way:
# ${foo%b?r} removes "bar" in the end of $foo
# So we're testing if $foo does not end with "b?r"
$ if [[ ${foo%b?r} = $foo ]]; then echo "ok"; fi
# ${foo#b?r} removes "b?r" in the beginning of $foo
# So we're testing if $foo does not begin with "b?r"
$ if [[ ${foo#b?r} = $foo ]]; then echo "ok"; fi
ok
See the Parameter Expansion paragraph of man bash for more info on these syntaxes. Using ## or %% instead of # and % respectively will achieve a longest matching instead of a simple matching.
Another very classic way of dealing with wildcards is to use case:
case $foo in
*bar)
echo "Foo matches *bar"
;;
bar?)
echo "Foo matches bar?"
;;
*)
echo "Foo didn't match any known rule"
;;
esac
John T's answer was deleted, but I actually think he was on the right track. Here it is:
Another portable method which will work in most versions of bash is
to echo your string then pipe to grep. If no match is found, it will
evaluate to false as the result will be blank. If something is returned,
it will evaluate to true.
[john#awesome]$string="Hello World"
[john#awesome]$if [[ `echo $string | grep Hello` ]];then echo "match";fi
match
What John didn't consider is the wildcard requested by the answer. For that, use egrep, a.k.a. grep -E, and use the regex wildcard .*. Here, . is the wildcard, and * is a multiplier meaning "any number of these". So, John's example becomes:
$ string="Hello World"
$ if [[ `echo $string | egrep "Hel.*"` ]]; then echo "match"; fi
The . wildcard notation is fairly standard regex, so it should work with any command that speaks regex's.
It does get nasty if you need to escape the special characters, so this may be sub-optimal:
$ if [[ `echo $string | egrep "\.\-\$.*"` ]]; then echo "match"; fi

Resources