Bash split an array, add a variable and concatenate it back together - linux

I've been trying to figure this out, unfortunately I can't. I am trying to create a function that finds the ';' character, puts four spaces before it and then and puts the code back together in a neat sentence. I've been cracking at this for a bit, and can't figure out a couple of things. I can't get the output to display what I want it to. I've tried finding the index of the ';' character and it seems I'm going about it the wrong way. The other mistake that I seem to be making is that I'm trying to split in a array in a for loop, and then split the individual words in the array by letter but I can't figure out how to do that either. If someone can give me a pointer this would be greatly appreciated. This is in bash version 4.3.48
#!commentPlacer()
{
arg=($1) #argument
len=${#arg[#]} #length of the argument
comment=; #character to look for in second loop
commaIndex=(${arg[#]#;}) #the attempted index look up
commentSpace=" ;" #the variable being concatenated into the array
for(( count1=0; count1 <= ${#arg[#]}; count1++ )) #search the argument looking for comment space
do if [[ ${arg[count1]} != commentSpace ]] #if no commentSpace variable then
then for (( count2=0; count2 < ${#arg[count1]} ; count2++ )) #loop through again
do if [[ ${arg[count2]} != comment ]] #if no comment
then A=(${arg[#]:0:commaIndex})
A+=(commentSpace)
A+=(${arg[#]commaIndex:-1}) #concatenate array
echo "$A"
fi
done
fi
done
}

If I understand what you want correctly, it's basically to put 4 spaces in front of each ";" in the argument, and print the result. This is actually simple to do in bash with a string substitution:
commentPlacer() {
echo "${1//;/ ;}"
}
The expansion here has the format ${variable//pattern/replacement}, and it gives the contents of the variable, with each occurrence of pattern replaced by replacement. Note that with only a single / before the pattern, it would replace only the first occurrence.
Now, I'm not sure I understand how your script is supposed to work, but I see several things that clearly aren't doing what you expect them to do. Here's a quick summary of the problems I see:
arg=($1) #argument
This doesn't create an array of characters from the first argument. var=(...) treats the thing in ( ) as a list of words, not characters. Since $1 isn't in double-quotes, it'll be split into words based on whitespace (generally spaces, tabs, and linefeeds), and then any of those words that contain wildcards will be expanded to a list of matching filenames. I'm pretty sure this isn't at all what you want (in fact, it's almost never what you want, so variable references should almost always be double-quoted to prevent it). Creating a character array in bash isn't easy, and in general isn't something you want to do. You can access individual characters in a string variable with ${var:index:1}, where index is the character you want (counting from 0).
commaIndex=(${arg[#]#;}) #the attempted index look up
This doesn't do a lookup. The substitution ${var#pattern} gives the value of var with pattern removed from the front (if it matches). If there are multiple possible matches, it uses the shortest one. The variant ${var##pattern} uses the longest possible match. With ${array[#]#pattern}, it'll try to remove the pattern from each element -- and since it's not in double-quotes, the result of that gets word-split and wildcard-expanded as usual. I'm pretty sure this isn't at all what you want.
if [[ ${arg[count1]} != commentSpace ]] #if no commentSpace variable then
Here (and in a number of other places), you're using a variable without $ in front; this doesn't use the variable at all, it just treats "commentSpace" as a static string. Also, in several places it's important to have double-quotes around it, e.g. to keep the spaces in $commentSpace from vanishing due to word splitting. There are some places where it's safe to leave the double-quotes off, but in general it's too hard to keep track of them, so just use double-quotes everywhere.
General suggestions: don't try to write c (or java or whatever) programs in bash; it works too differently, and you have to think differently. Use shellcheck.net to spot common problems (like non-double-quoted variable references). Finally, you can see what bash is doing by putting set -x before a section that doesn't do what you expect; that'll make bash print each line as it executes it, showing the equivalent of what it's executing.

Make a little function using pattern substitution on stdin:
semicolon4s() { while read x; do echo "${x//;/ ;}"; done; }
semicolon4s <<< 'foo;bar;baz'
Output:
foo ;bar ;baz

Related

Shell script (bash) to match a string variable with multiple values

I am trying write a script to compare one string variable to a list of values, i.e. if the variable matches (exact) to one of the values, then some action needs to be done.
The script is trying to match Unix pathnames, i.e. if the user enters / , /usr, /var etc, then to give an error, so that we do not get accidental corruption using the script. The list of values may change in future due to the application requirements. So I cannot have huge "if" statement to check this.
What I intend to do is that in case if the user enters, any of the forbidden path to give an error but sub-paths which are not forbidden should be allowed, i.e. /var should be rejected but /var/opt/app should be accepted.
I cannot use regex as partial match will not work
I am not sure of using a where loop and an if statement, is there any alternative?
thanks
I like to use associative arrays for this.
declare -A nonoList=(
[/foo/bar]=1
["/some/other/path with spaces"]=1
[/and/so/on]=1
# as many as you need
)
This can be kept in a file and sourced, if you want to separate it out.
Then in your script, just do a lookup.
if [[ -n "${nonoList[$yourString]}" ]] # -n checks for nonzero length
This also prevents you from creating a big file ad grep'ing over it redundantly, though that also works.
As an alternative, if you KNOW there will not be embedded newlines in any of those filenames (it's a valid character, but messy for programming) then you can do this:
$: cat foo
/foo/bar
/some/other/path with spaces
/and/so/on
Just a normal file with one file-path per line. Now,
chkSet=$'\n'"$(<foo)"$'\n' # single var, newlines before & after each
Then in your processing, assuming f=/foo/bar or whatever file you're checking,
if [[ "$chkSet" =~ $'\n'"$f"$'\n' ]] # check for a hit
This won't give you accidental hits on /some/other/path when the actual filename is /some/other/path with spaces because the pattern explicitly checks for a newline character before and after the filename. That's why we explicitly assure they exist at the front and end of the file. We assume they are in between, so make sure your file doesn't have any spaces (or any other characters, like quotes) that aren't part of the filenames.
If you KNOW there will also be no embedded whitespace in your filenames, it's a lot easier.
mapfile -t nopes < foo
if [[ " ${nopes[*]} " =~ " $yourString " ]]; then echo found; else echo no; fi
Note that " ${nopes[*]} " embeds spaces (technically it uses the first character of $IFS, but that's a space by default) into a single flattened string. Again, literal spaces before and behind key and list prevent start/end mismatches.
Paul,
Your alternative work around worked like a charm. I don't have any directories which need embedded space in them. So as long as my script can recognize that there are certain directories to avoid, it does its job.
Thanks

How to compare and recursively modify strings in Bash

I need to write a bash code performing some tasks I am going to explain.
The input: two uppercase strings of same length, no matter
their length is. Es:
CYVFGDDAS --> string1 , unchangeable reference string
CRFDGVEAT --> string2 , modifiable string
I am trying to write Bash code that is able to compare the characters with same index recursively starting from the first position:
-- beginnig of the cycle --
if the characters are the same skip any action and go to the
the next position,
while
if the characters are not the same the character of string1
replaces the character of string2 at that position
the new string2 is saved in a file
a substituion code is also written in the same file (I will
explain this below)
the old string2 is replaced by the new string2 in such a way
its changes are retained
start anothe cycle from the beginning
------
Repeat the cycle until the last character is processed.
So, for the example above, the code should start checking from the
first position where two C characters are placed. They match so no
action is taken and both strings are left unchanged.
Going to he second position Y should replace R in the second string,
the modified string should be saved and written in a text file togheter with the substitution code YA2V (Y is the replacing character of string1, A is a costant character that must be present in all substitutions codes, 2 is the positional index where the substitution occurred, and V is the replaced character of string2).
I am proficient in Python which has a large number of modules for string manipulation but because the code should be added to a pre-existing Bash program I need to get this done in Bash environment (builtin commands, awk, sed etc, does not matter). Looks to me that Bash does not have an extended arsenal of tools like Python, so I am first of all wondering if this project is feasible or not.
However, what I tried so far is to convert the strings in blank
separated fields by inserting spaces between the characters in such
a way awk can deal better with them as fields but I did not go very
far with this.
Sorry for the lengthy explanation. Any help is greatly appreciated.
No recursion is needed, just iterate over the strings. You can use parameter expansion with a for loop:
#!/bin/bash
s1=CYVFGDDAS
s2=CRFDGVEAT
for ((i=0; i<${#s1} ; ++i)) ; do
if [[ ${s1:i:1} != ${s2:i:1} ]] ; then
printf '%s\n' "${s1:0:i+1}${s2:i+1}"
printf '%s\n' "${s1:i:1}A$((i+1))${s2:i:1}"
fi
done
${s1:i:1} means extract the substring of $s1 from position $i of length 1. If the length is omitted, it extracts as much as it can.
It just outputs the strings, redirect them to files as you need.
CYFDGVEAT
YA2R
CYVDGVEAT
VA3F
CYVFGVEAT
FA4D
CYVFGDEAT
DA6V
CYVFGDDAT
DA7E
CYVFGDDAS
SA9T

how to concatenate two variable values with different rows in linux?

I have two variable or files with strings like
a="p|q"
b="x y z"
b variable has three characters separate by space.
my question is how to generate output as using a shell command in Linux with separator as "|"
p|q|x
p|q|y
p|q|z
Not quite sure if I understand your question correctly. The following for-loop will give you the output you want:
for i in $b; do echo "$a|$i"; done
You can do it without a loop, but it's not recommended in general:
printf "$a|%s\n" $b
$a is expanded before the command is run, so your format string ends up as 'p|q|%s\n'
$b is also expanded and since it is unquoted, each word becomes a separate argument
So the command you run is:
printf 'p|q|%s\n' x y z
which produces the output you wanted.
Note that this is not ideal because:
In general, putting variables into a printf format string can lead to unexpected output (e.g. if the variable contains characters like %s).
Deliberately using unquoted variables reads like a typo to a lot of people, so you probably need to add a comment to explain that you're being clever.

Why this bash function prints only first word of whole string?

I'm trying to create function that will print message bound to variable in certain color. The message variable is passed as argument of this function. The problem is that I'm getting only text up to first space (only first word of message). My script looks like this:
#!/usr/bash
lbGREEN='\e[1;92m'
NC='\e[0m'
normalMessage="Everything fine"
echo_message() {
echo -e ${lbGREEN}$1${NC}
}
echo_message $normalMessage
My output is:
Everything
As Inian pointed out in comments, your problem is unquoted variable expansion
echo_message $normalMessage
becomes
echo_message Everything fine
once the variable expands, meaning that each word of your input string is getting read in as a separate argument. When this happens $1=Everything and $2=fine.
This is fixed by double-quoting your variable, which allows expansion, but will mean the result of the expansion will still be read as one argument.
echo_message "$normalMessage"
becomes
echo_message "Everything fine"
Like this $1=Everything fine
In the future, I recommend using https://www.shellcheck.net/, or the CLI version of shellcheck, it will highlight all kinds of common bash gotcha's, included unquoted expansion.
For me, I had to change the header for "#!/bin/bash", but apparently that is not the problem for you.
In your echo you are printing only the first word with the $1, if you change it to $2 you will print the second word (parameter) and so on.
You can pass the name inside quotes or print all the parameters with $#
Solution 1 (with $#):
lbGREEN='\e[1;92m'
NC='\e[0m'
normalMessage="Everything fine"
echo_message() {
echo -e ${lbGREEN}$#${NC}
}
echo_message $normalMessage
Solution 2 (with quotes):
lbGREEN='\e[1;92m'
NC='\e[0m'
normalMessage="Everything fine"
echo_message() {
echo -e ${lbGREEN}$1${NC}
}
echo_message "$normalMessage"
You should get a look to https://stackoverflow.com/a/6212408/1428602
IMHO, $1 only return the 1st word, so you have to use a loop or try with $*
You've got the quoting wrong.
If you want to simulate the behaviour of echo, your function should accept multiple parameters, and print them all. Currently it's only evaluating the first parameter, so I suggest using $* instead. You also need to enclose the argument in double quotes to protect any special characters:
echo_message() {
echo -e "${lbGREEN}$*${NC}"
}
The special variable $* expands to all the arguments, separated by spaces (or more accurately, the first character of $IFS, which is usually a space character). Note that you almost always want "$#" instead of "$*", and this is one of the rare occasions where the latter is also correct, though with slightly different semantics if IFS is set to a non-standard value.
Now the function supports multiple arguments, and prints them all in green, separated by spaces. However, I would recommend that you also quote the argument when calling the function:
echo_message "$normalMessage"
While spaces in $normalMessage will now be treated correctly, other special characters like ! will still require the quotes.

Expanding known env vers in a file and leaving others

I'm sure there's a really easy way of doing this. I'm trying to take a file which contains some environment variables and expand it so that those which are known are expanded to their values whereas those which are not are left alone.
For example, if my file contained the following:
${I_EXIST}
${I_ALSO_EXIST}
${I_DONT_EXIST}
this would be expanded to:
existValue
alsoExistValue
${I_DONT_EXIST}
I ideally want to do this as simply as possible so I don't want a complex substitution using sed, awk or perl. I'm thinking of something similar to a "Here" file, but apart from the fact that I can't get the syntax right, it also blanks out anything which does not expand. E.g:
cat <<EOF
> ${I_EXIST}
> ${I_ALSO_EXIST}
> ${I_DONT_EXIST}
EOF
existValue
alsoExistValue
(i.e. the last value expands to nothing)
Update
Should really have made clear that I was thinking about potentially more than one substitution per line. One way I did find to do this, if we're not fussed about the variables appearing in the file as ${MYVAR} but maybe MYVAR will do:
m4 $( env | sed 's/\([A-Za-z0-9]*\)=\([\/A-Za-z_0-9:|%*. -#]*\)/-D\1=\2' ) myfile
This uses the M4 preprocessor to substitute all the pairs in your environment. A couple of caviats here:
Sorry about the reg exp stuff. It looks pretty nasty and I'm sure there are nicer ways of expressing this. I found problems if my env vars had spaces in them or any unusual characters that weren't in the set.
Of course this is a blunt substitution tool (which I was trying to avoid) so variable might get substituted when you didn't want it to happen.
#!/bin/bash
while read a;
do
n=$(eval echo $a)
if [[ "$n" == "" ]]
then
echo $a
else
echo $n
fi
done < input
Using this as input
${HOME}
${nonexistent}
Gives
/home/myuser
${nonexistent}
Easy to read? Maybe not. It is short and works though :-)
while read r; do
echo $(eval echo ${r%\}}:-'$r'\})
done < input
Magic used:
http://www.gnu.org/software/bash/manual/bashref.html#Shell-Parameter-Expansion
Edit: Further explanation, I hope it makes some sense.
We use two techniques; from the above docs:
${parameter:−word} If parameter is
unset or null, the expansion of word
is substituted. Otherwise, the value
of parameter is substituted.
And
${parameter%word}
The word is expanded to produce a
pattern just as in filename expansion.
If the pattern matches a trailing
portion of the expanded value of
parameter, then the result of the
expansion is the value of parameter
with the shortest matching pattern
(the ‘%’ case) or the longest matching
pattern (the ‘%%’ case) deleted. ...
We use the fact that the input is just what we can use in the shell, we have ${FOOBAR} but need ${FOOBAR:-'${FOOBAR}'} (Single quotes to avoid expansion).
# echo ${doesntexist:-Hello}
Hello
# doesexist=World
# echo ${doesexist:-Will not be printed}
World
So what we need to inject is :-'${FOOBAR}'
To achieve this we trim the } at the end, add the string, then put another } back afterwards.
# echo $r
${FOOBAR}
# echo ${r%\}}
${FOOBAR
The final \} isn't really necessary, since it's got no beginning in this case, but it's better to be explicit and escape it. (Much like you would escape echo \* even if echo * without any matching files gives you a literal *).
Edit2: This of course doesn't take into account that you wanted to support multiple variables in a single row; or any rows with other stuff in them.
while read name; do echo "$name = " $(eval echo $name); done < file_with_vars.txt
will echo all variables what know.
e.g.
in my file called vv
${PATH}
${HAVENOT}
${LOCALE}
will print
${PATH} = /usr/local/narwhal/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:~/bin
${HAVENOT} =
${LOCALE} = UTF-8
modify the output format as you wish :)

Resources