I have a shell script that consist of two files, one bash-file (main.sh) and one file holding all my config-variables(vars.config).
vars.config
domains=("something.com" "else.something.com")
something_com_key="key-to-something"
else_something_com_key="key-to-something else"
In my code i want to loop through the domains array and get the key for the domain.
#!/usr/bin/env sh
source ./vars.config
key="_key"
for i in ${domains[#]};
do
base="$(echo $i | tr . _)" # this swaps out . to _ to match the vars
let farmid=$base$key
echo $farmid
done
So when i run it i get an error message
./main.sh: line 13: let: key-to-something: syntax error: operand
expected (error token is "key-to-something")
So it actually swaps it out, but i cant save it to a variable.
You can expand a variable to the value of its value using ${!var_name}, for example in your code you can do:
key="_key"
for i in ${domains[#]};
do
base="$(echo $i | tr . _)" # this swaps out . to _ to match the vars
farmid=$base$key
farmvalue=${!farmid}
echo $farmvalue
done
Related
NOTE BEFORE READING: The following question is described very precisely and that is the reason for the length of a question. If you want to understand the problem, it's better to read the entire thing. Many thanks for all the answers!
I am working on a bash script (.sh file) which will check certain values in every file of a directory. Bash script will be executed in a pre-commit (pre-commit is not a part of the question).
There is a directory that contains multiple .c files in multiple subdirectories. I want to check a part of two lines which are NOT in every .c file but only in some of them. The structure of a file that contains the useful information is as following:
/*
## SYMBOL = some_symbol1
## A2L_TYPE = PARAMETER
.
.
.
#! DEFAULT = some_value1
## END
*/
some_symbol1 = some_value1
/*
## SYMBOL = some_symbol2
## A2L_TYPE = PARAMETER
.
.
.
#! DEFAULT = some_value2
## END
*/
some_symbol2 = some_value2
This kind of structure is automatically generated by another script.
I want to check if some_value1 (in comment) is equal to some_value1 (in variable).
There are hundreds of these variable in each .c file (not necessarily in each .c file).
The main functionality of a script should be:
Check some_value1 in comment and variable and throw an error if they are not the same. Script has to go through EVERY .c file in a directory (bash is in root) and ALL subdirectories to find previously mentioned structure.
Value of variable can be something as 0.06F, where in comment, there is 0.06 (compare only the numbers)
Value of variable can also be an array: { 0.0F, 0.45F, 0.3F } where in the comment, there is [ 0.0, 0.45, 0.3 ] (without F and difference in braces)
To summarize:
I want to build a check script that compares some_value1 (in comment) and some_value1 (in variable) and throw an error if they don't match
Useful information is not in EVERY .c file but only in some of them (don't know which)
Values after #! DEFAULT is a comment where the value of variable is a number (maybe this is not that important?)
between A2L_TYPE and DEFAULT, there can be desired number of unimportant stuff. (still a comment)
What I tried so far is for loop through every .c file and a nested for loop to read every line in each .c file. What I wanted to implement was a grep command inside for loop to check each line if there is a #! DEFAULT pattern and save it to the variable.
Latest code that I tried:
!/bin/bash
shopt -s globstar
for d in */**/*.c
do
while IFS="" read -r p || [ -n "$p" ]
do
grep -P "#! DEFAULT" $d
done < $d
done
This is currently not working because it gives an error that certain grep targets are directories
If any has any questions, I will try to explain it better.
# search for files with extension ".c"
# execute awk on any matches, using '= ' as field separator
find . -type f -name '*.c' -exec awk -F'=[[:space:]]*' '
# check if first three lines match template
( NR==1 && /^\/\*/ ) ||
( NR==2 && /^## SYMBOL = / ) ||
( NR==3 && /^## A2L_TYPE = PARAMETER/ ) { ok++ }
# template mismatch - skip this file
( NR==4 && ok!=3 ) {
printf "%s : ignored\n", FILENAME
nextfile
}
# store first occurrence of some_value1
# note line number where second occurrence expected
/^#! DEFAULT =/ { v[1]=v1=$2; n=NR+3 }
# test second occurrence
NR==n {
v[2]=v2=$2;
# prune everything except numbers and array delimiters
for (s in v) gsub(/[^0-9.,]/,"",v[s]);
# output result
# match exactly or only number list
printf "%s #(%d,%d) : ", FILENAME,n-3,n
if (v1==v2 || v[1]==v[2])
printf "match (%s)==(%s)\n", v1,v2
else
printf "mismatch (%s)!=(%s)\n", v1,v2
# no need to check rest of this file
# elide to check multiple values per file
nextfile
}
' {} +
I know I can do this to reflect just last 2 directories in the PS1 value.
PS1=${PWD#"${PWD%/*/*}/"}#
but lets say we have a directory name that's really messy and will reduce my working space , like
T-Mob/2021-07-23--07-48-49_xperia-build-20191119010027#
OR
2021-07-23--07-48-49_nokia-build-20191119010027/T-Mob#
those are the last 2 directories before the prompt
I want to set a condition if directory length of either of the last 2 directories is more than a threshold e.g. 10 chars , shorten the name with 1st 3 and last 3 chars of the directory (s) whose length exceeds 10
e.g.
2021-07-23--07-48-49_xperia-build-20191119010027 &
2021-07-23--07-48-49_nokia-build-20191119010027
both gt 10 will be shortened to 202*027 & PS1 will be respectively
T-Mob/202*027/# for T-Mob/2021-07-23--07-48-49_xperia-build-20191119010027# and
202*027/T-Mob# for 2021-07-23--07-48-49_nokia-build-20191119010027/T-Mob#
A quick 1 Liner to get this done ?
I cant post this in comments so Updating here. Ref to Joaquins Answer ( thx J)
PS1=''`echo ${PWD#"${PWD%/*/*}/"} | awk -v RS='/' 'length() <=10{printf $0"/"}; length()>10{printf "%s*%s/", substr($0,1,3), substr($0,length()-2,3)};'| tr -d "\n"; echo "#"`''
see below o/p's
/root/my-applications/bin # it shortened as expected
my-*ons/bin/#cd - # going back to prev.
/root
my-*ons/bin/# #value of prompt is the same but I am in /root
A one-liner is basically always the wrong choice. Write code to be robust, readable and maintainable (and, for something that's called frequently or in a tight loop, to be efficient) -- not to be terse.
Assuming availability of bash 4.3 or newer:
# Given a string, a separator, and a max length, shorten any segments that are
# longer than the max length.
shortenLongSegments() {
local -n destVar=$1; shift # arg1: where do we write our result?
local maxLength=$1; shift # arg2: what's the maximum length?
local IFS=$1; shift # arg3: what character do we split into segments on?
read -r -a allSegments <<<"$1"; shift # arg4: break into an array
for segmentIdx in "${!allSegments[#]}"; do # iterate over array indices
segment=${allSegments[$segmentIdx]} # look up value for index
if (( ${#segment} > maxLength )); then # value over maxLength chars?
segment="${segment:0:3}*${segment:${#segment}-3:3}" # build a short version
allSegments[$segmentIdx]=$segment # store shortened version in array
fi
done
printf -v destVar '%s\n' "${allSegments[*]}" # build result string from array
}
# function to call from PROMPT_COMMAND to actually build a new PS1
buildNewPs1() {
# declare our locals to avoid polluting global namespace
local shorterPath
# but to cache where we last ran, we need a global; be explicit.
declare -g buildNewPs1_lastDir
# do nothing if the directory hasn't changed
[[ $PWD = "$buildNewPs1_lastDir" ]] && return 0
shortenLongSegments shorterPath 10 / "$PWD"
PS1="${shorterPath}\$"
# update the global tracking where we last ran this code
buildNewPs1_lastDir=$PWD
}
PROMPT_COMMAND=buildNewPs1 # call buildNewPs1 before rendering the prompt
Note that printf -v destVar %s "valueToStore" is used to write to variables in-place, to avoid the performance overhead of var=$(someFunction). Similarly, we're using the bash 4.3 feature namevars -- accessible with local -n or declare -n -- to allow destination variable names to be parameterized without the security risk of eval.
If you really want to make this logic only apply to the last two directory names (though I don't see why that would be better than applying it to all of them), you can do that easily enough:
buildNewPs1() {
local pathPrefix pathFinalSegments
pathPrefix=${PWD%/*/*} # everything but the last 2 segments
pathSuffix=${PWD#"$pathPrefix"} # only the last 2 segments
# shorten the last 2 segments, store in a separate variable
shortenLongSegments pathSuffixShortened 10 / "$pathSuffix"
# combine the unshortened prefix with the shortened suffix
PS1="${pathPrefix}${pathSuffixShortened}\$"
}
...adding the performance optimization that only rebuilds PS1 when the directory changed to this version is left as an exercise to the reader.
Probably not the best solution, but a quick solution using awk:
PS1=`echo ${PWD#"${PWD%/*/*}/"} | awk -v RS='/' 'length()<=10{printf $0"/"}; length()>10{printf "%s*%s/", substr($0,1,3), substr($0,length()-2,3)};'| tr -d "\n"; echo "#"`
I got this results with your examples:
T-Mob/202*027/#
202*027/T-Mob/#
I'm a few weeks into bash scripting and I haven't advanced enough yet to get my head wrapped around this problem. Any help would be appreciated!
I have a "script.conf" file that contains the following:
key1=value1
key2=${HOME}/Folder
key3=( "k3v1" "k3 v2" "k3v3")
key4=( "k4v1"
"k4 v2"
"k4v3"
)
key5=value5
#key6="Do Not Include Me"
In a bash script, I want to read the contents of this script.conf file into an array. I've learned how to handle the scenarios for keys 1, 2, 3, and 5, but the key4 scenario throws a wrench into it with it spanning across multiple lines.
I've been exploring the use of sed -n '/=\s*[(]/,/[)]/{/' which does capture key4 and its value, but I can't figure out how to mix this so that the other keys are also captured in the matches. The range syntax is also new to me, so I haven't figured out how to separate the key/value. I feel like there is an easy regex that would accomplish what I want... in plain-text: "find and group the pattern ^(.*)= (for the key), then group everything after the '=' char until another ^(.*)= match is found, rinse and repeat". I guess if I do this, I need to change the while read line to not handle the key/value separation for me (I'll be looking into this while I'm waiting for a response). BTW, I think a solution where the value of key4 is flattened (new lines removed) would be acceptable; I know for key3 I have to store the value as a string and then convert it to an array later when I want to iterate over it since an array element apparently can't contain a list.
Am I on the right path with sed or is this a job for awk or some other tool? (I haven't ventured into awk yet). Is there an easier approach that I'm missing because I'm too deep into the forest (like changing the while read line in the LoadConfigFile function)?
Here is the code that I have so far in script.sh for processing and capturing the other pairs into the $config array:
__AppDir=$(dirname $0)
__AppName=${__ScriptName%.*}
typeset -A config #init config array
config=( #Setting Default Config values
[key1]="defaultValue1"
[key2]="${HOME}/defaultFolder"
[QuietMode]=0
[Verbose]=0 #Ex. Usage: [[ "${config[Verbose]}" -gt 0 ]] && echo ">>>Debug print"
)
function LoadConfigFile() {
local cfgFile="${1}"
shopt -s extglob #Needed to remove trailing spaces
if [ -f ${cfgFile} ]; then
while IFS='=' read -r key value; do
if [[ "${key:0:1}" == "#" ]]; then
#echo "Skipping Comment line: ${key}"
elif [ "${key:-EMPTY}" != "EMPTY" ]; then
value="${value%%\#*}" # Delete in-line, right comments
value="${value%%*( )}" # Delete trailing spaces
value="${value%%( )*}" # Delete leading spaces
#value="${value%\"*}" # Delete opening string quotes
#value="${value#\"*}" # Delete closing string quotes
#Manipulate any variables included in the value so that they can be expanded correctly
# - value must be stored in the format: "${var1}". `backticks`, "$var2", and "doubleQuotes" are left as is
value="${value//\"/\\\"}" # Escape double quotes for eval
value="${value//\`/\\\`}" # Escape backticks for eval
value="${value//\$/\\\$}" # Escape ALL '$' for eval
value="${value//\\\${/\${}" # Undo the protection of '$' if it was followed by a '{'
value=$(eval "printf '%s\n' \"${value}\"")
config[${key}]=${value} #Store the value into the config array at the specified key
echo " >>>DBG: Key = ${key}, Value = ${value}"
#else
# echo "Skipped Empty Key"
fi
done < "${cfgFile}"
fi
}
CONFIG_FILE=${__AppDir}/${__AppName}.conf
echo "Config File # ${CONFIG_FILE}"
LoadConfigFile ${CONFIG_FILE}
#Print elements of $config
echo "Script Config Values:"
echo "----------------------------"
for key in "${!config[#]}"; do #The '!' char gets an array of the keys, without it, we would get an array of the values
printf " %-20s = %s\n" "${key}" "${config[${key}]}"
done
echo "------ End Script Config ------"
#To convert to an array...
declare -a valAsArray=${config[RequiredAppPackages]} #Convert the value from a string to an array
echo "Count = ${#valAsArray[#]}"
for itemCfg in "${valAsArray[#]}"; do
echo " item = ${itemCfg}"
done
As I mentioned before, I'm just starting to learn bash and Linux scripting in general, so if you see that I'm doing some taboo things in other areas of my code too, please feel free to provide feedback in the comments... I don't want to start bad habits early on :-).
*If it matters, the OS is Ubuntu 14.04.
EDIT:
As requested, after reading the script.conf file, I would like for the elements in $config[#] to be equivalent to the following:
typeset -A config #init config array
config=(
[key1]="value1"
[key2]="${HOME}/Folder"
[key3]="( \"k3v1\" \"k3 v2\" \"k3v3\" )"
[key4]="( \"k4v1\" \"k4 v2\" \"k4v3\" )"
[key5]="value5"
)
I want to be able to convert the values of elements 'key4' and 'key3' into an array and iterated over them the same way in the following code:
declare -a keyValAsArray=${config[keyN]} #Convert the value from a string to an array
echo "Count = ${#keyValAsArray[#]}"
for item in "${keyValAsArray[#]}"; do
echo " item = ${item}"
done
I don't think it matters if \n is preserved for key4's value or not... that depends on if declare has a problem with it.
A shell is an environment from which to call tools with a language to sequence those calls. It is NOT a tool to manipulate text. The standard UNIX tool to manipulate text is awk. Trying to manipulate text in shell IS a bad habit, see why-is-using-a-shell-loop-to-process-text-considered-bad-practice for SOME of the reasons why
You still didn't post the expected result of populating the config array so I'm not sure but I think this is what you wanted:
$ cat tst.sh
declare -A config="( $(awk '
{ gsub(/^[[:space:]]+|([[:space:]]+|#.*)$/,"") }
!NF { next }
/^[^="]+=/ {
name = gensub(/=.*/,"",1)
value = gensub(/^[^=]+=/,"",1)
n2v[name] = value
next
}
{ n2v[name] = n2v[name] OFS $0 }
END {
for (name in n2v) {
value = gensub(/"/,"\\\\&","g",n2v[name])
printf "[%s]=\"%s\"\n", name, value
}
}
' script.conf
) )"
declare -p config
$ ./tst.sh
declare -A config='([key5]="value5" [key4]="( \"k4v1\" \"k4 v2\" \"k4v3\" )" [key3]="( \"k3v1\" \"k3 v2\" \"k3v3\")" [key2]="/home/Ed/Folder" [key1]="value1" )'
The above uses GNU awk for gensub(), with other awks you'd use [g]sub() instead.
I wrote simple script as follow
#!/bin/bash
auth_type=""
SM_Read-only="Yes"
SM_write-only="No"
echo -e ${SM_Read-only}
echo -e ${SM_Write-only}
if [ "${SM_Read-only}" == "Yes" ] && [ "${SM_Write-only}" == "Yes" ]
then
auth_type="Read Write"
else
auth_type="Read"
fi
echo -e $auth_type
And when i execute it i got following output with errors.
./script.bash: line 5: SM_Read-only=Yes: command not found
./script.bash: line 6: SM_write-only=No: command not found
only
only
Read
Any one know correct way to declare the variable with "-" (dash)?
EDIT:
have getting response from c code and evaluate the variables for example
RESP=`getValue SM_ Read-only ,Write-only 2>${ERR_DEV}`
RC=$?
eval "$RESP"
from above scripts code my c binary getValue know that script want Read-only and Write-only and return value to script.So during eval $RESP in cause error and in my script i access variable by
echo -e ${SM_Read-only}
echo -e ${SM_Write-only}
which also cause error.
Rename the variable name as follows:
SM_Read_only="Yes"
SM_write_only="No"
Please, don't use - minus sign in variable names in bash, please refer to the answer, on how to set the proper variable name in bash.
However if you generate the code, based on others output, you can simply process their output with sed:
RESP=$(getValue SM_ Read-rule,Write-rule 2>${ERR_DEV}|sed "s/-/_/g")
RC=$?
eval "$RESP"
- is not allowed in shell variable names. Only letters, numbers, and underscore, and the first character must be a letter or underscore.
I think you cant have a dash in your variables names, only letters, digits and "_"
Try:
SM_Read_only
Or
SM_ReadOnly
I have two arrays that I want to loop in. I construct those properly and before going into for loop, I do echo them to be sure everything is ok with arrays.
But when I run the script, it outputs an error:
l<=: syntax error: operand expected (error token is "<="
I consulted the mighty Google and I understood it suffers from the lack of the second variable, but I mentioned earlier I do echo the values and everything seems to be OK. Here is the snippet..
#!/bin/bash
k=0
#this loop is just for being sure array is loaded
while [[ $k -le ${#hitEnd[#]} ]]
do
echo "hitEnd is: ${hitEnd[k]} and hitStart is: ${hitStart[k]}"
# here outputs the values correct
k=$((k+1))
done
k=0
for ((l=${hitStart[k]};l<=${hitEnd[k]};l++)) ; do //this is error line..
let array[l]++
k=$((k+1))
done
The variables in the for loop are echoed correctly but for loop won't work.. where am I wrong?
#
as gniourf_gniourf answered:
"... At some point, k will reach the value ${#hitEnd[#]}, and this is
exactly when hitEnd[k] is not defined and expands to an empty string!
Bang!"
meaning error output is displayed not at the beginning of the loop, but when k has a greater value than array's indices, pointing an index that array does not include...
That's because at some point ${hitEnd[k]} expands to nothing (it is undefined). I get the same error with ((l<=)). You should write your for loop as:
k=0
for ((l=${hitStart[0]};k<${#hitEnd[#]} && l<=${hitEnd[k]};l++)); do
so as to always have an index k that corresponds to a defined field in the array ${hitEnd[#]}.
Also, instead of
k=$((k+1))
you can just write
((++k))
Done!
Your script revised using better modern bash practice:
#!/bin/bash
k=0
#this loop is just for being sure array is loaded
while ((k<=${#hitEnd[#]})); do
echo "hitEnd is: ${hitEnd[k]} and hitStart is: ${hitStart[k]}"
# here outputs the values correct
((++k))
done
k=0
for ((l=hitStart[0];k<${#hitEnd[#]} && l<=hitEnd[k];++l)); do
((++array[l]))
((++k))
done
Now, I'm not too sure the for loop does exactly what you want it to... Don't you mean this instead?
#!/bin/bash
# define arrays hitStart[#] and hitEnd[#]...
# define array array[#]
#this loop is just for being sure array is loaded
for ((k=0;k<${#hitEnd[#]};++k)); do
echo "hitEnd is: ${hitEnd[k]} and hitStart is: ${hitStart[k]}"
# here outputs the values correct
((++k))
done
for ((k=0;k<${#hitEnd[#]};++k)); do
for ((l=hitStart[k];l<=hitEnd[k];++l)); do
((++array[l]))
done
done
A bit bandaid-y, but you rewrite your for-loop into a while loop:
l="${hitStart[k]}"
while [[ "$l" -le "${hitEnd[k]}" ]]; do
let array[l]++
k=$((k+1))
l=$((l+1))
done