I'm trying to read from a file, that has multiple lines, each with 3 informations I want to assign to the variables and work with.
I figured out, how to simply display them each on the terminal, but can't figure out how to actually assign them to variables.
while read i
do
for j in $i
do
echo $j
done
done < ./test.txt
test.txt:
1 2 3
a b c
So I want to read the line in the outer loop, then assign the 3 variables and then work with them, before going to the next line.
I'm guessing I have to read the values of the lines without an inside loop, but I can't figure it out right now.
Hope someone can point me in the right direction.
I think all you're looking for is to read multiple variables per line: the read command can assign words to variables by itself.
while read -r first second third; do
do_stuff_with "$first"
do_stuff_with "$second"
do_stuff_with "$third"
done < ./test.txt
The below assumes that your desired result is the set of assignments a=1, b=2, and c=3, taking the values from the first line and the keys from the second.
The easy way to do this is to read your keys and values into two separate arrays. Then you can iterate only once, referring to the items at each position within those arrays.
#!/usr/bin/env bash
case $BASH_VERSION in
''|[123].*) echo "ERROR: This script requires bash 4.0 or newer" >&2; exit 1;;
esac
input_file=${1:-test.txt}
# create an associative array in which to store your variables read from a file
declare -A vars=( )
{
read -r -a vals # read first line into array "vals"
read -r -a keys # read second line into array "keys"
for idx in "${!keys[#]}"; do # iterate over array indexes (starting at 0)
key=${keys[$idx]} # extract key at that index
val=${vals[$idx]} # extract value at that index
vars[$key]=$val # assign the value to the key inside the associative array
done
} < "$input_file"
# print for debugging
declare -p vars >&2
echo "Value of variable a is ${vars[a]}"
See:
BashFAQ #6 - How can I use variable variables (indirect variables, pointers, references) or associative arrays?
The bash-hackers page on the read builtin, documenting use of -a to read words into an array.
Related
For example, I have a string /something an-arg=some-value another-arg=another-value.
What would be the most straightforward way to extract an-arg's value to a variable and another-arg's value to another variable?
To better exemplify, this is what I need to happen:
STRING="/something an-arg=some-value another-arg=another-value"
AN_ARG=... # <-- do some magic here to extract an-arg's value
ANOTHER_ARG=... # <-- do some magic here to extract another-arg's value
echo $AN_ARG # should print `some-value`
echo $ANOTHER_ARG # should print `another-value`
So I was looking for a simple/straightforward way to do this, I tried:
ARG_NAME="an-arg="
AN_ARG=${STRING#*$ARG_NAME}
But the problem with this solution is that it will print everything that comes after an-arg, including the second argument's name and its value, eg some-value another-arg=another-value.
Letting data set arbitrary variables incurs substantial security risks. You should either prefix your generated variables (with a prefix having at least one lower-case character to keep the generated variables in the namespace POSIX reserves for application use), or put them in an associative array; the first example below does the latter.
Generating An Associative Array
As you can see at https://ideone.com/cKcMSM --
#!/usr/bin/env bash
# ^^^^- specifically, bash 4.0 or newer; NOT /bin/sh
declare -A vars=( )
re='^([^=]* )?([[:alpha:]_-][[:alnum:]_-]+)=([^[:space:]]+)( (.*))?$'
string="/something an-arg=some-value another-arg=another-value third-arg=three"
while [[ $string =~ $re ]]; do : "${BASH_REMATCH[#]}"
string=${BASH_REMATCH[5]}
vars[${BASH_REMATCH[2]}]=${BASH_REMATCH[3]}
done
declare -p vars # print the variables we extracted
...correctly emits:
declare -A vars=([another-arg]="another-value" [an-arg]="some-value" [third-arg]="three" )
...so you can refer to ${vars[an-arg]}, ${vars[another-arg]} or ${vars[third-arg]}.
This avoids faults in the original proposal whereby a string could set variables with meanings to the system -- changing PATH, LD_PRELOAD, or other security-sensitive values.
Generating Prefixed Names
To do it the other way might look like:
while [[ $string =~ $re ]]; do : "${BASH_REMATCH[#]}"
string=${BASH_REMATCH[5]}
declare -n _newVar="var_${BASH_REMATCH[2]//-/_}" || continue
_newVar=${BASH_REMATCH[3]}
unset -n _newVar
declare -p "var_${BASH_REMATCH[2]//-/_}"
done
...which work as you can see at https://ideone.com/zUBpsC, creating three separate variables with a var_ prefix on the name of each:
declare -- var_an_arg="some-value"
declare -- var_another_arg="another-value"
declare -- var_third_arg="three"
Assumptions:
OP understands all the issues outlined by Charles Duffy but still wants standalone variables
all variables names to be uppercased
hyphens (-) converted to underscores (_)
neither variable names nor the associated values contain embedded white space
One bash idea using namerefs:
unset newarg AN_ARG ANOTHER_ARG 2>/dev/null
STRING="/something an-arg=some-value another-arg=another-value"
read -ra list <<< "${STRING}" # read into an array; each space-delimited item is a new entry in the array
#typeset -p list # uncomment to display contents of the list[] array
regex='[^[:space:]]+=[^[:space:]]+' # search pattern: <var>=<value>, no embedded spaces in <var> nor <value>
for item in "${list[#]}" # loop through items in list[] array
do
if [[ "${item}" =~ $regex ]] # if we have a pattern match (<var>=<val>) then ...
then
IFS="=" read -r ndx val <<< "${BASH_REMATCH[0]}" # split on '=' and read into variables ndx and val
declare -nu newarg="${ndx//-/_}" # convert '-' to '_' and assign uppercased ndx to nameref 'newarg'
newarg="${val}" # assign val to newarg
fi
done
This generates:
$ typeset -p AN_ARG ANOTHER_ARG
declare -- AN_ARG="some-value"
declare -- ANOTHER_ARG="another-value"
NOTE:
once the for loop processing has completed, accessing the new variables will require some foreknowledge of the new variables' names
using an associative array to manage the list of new variables makes post for loop accessing quite a bit easier (eg, the new variable names are simply the indices of the associative array)
What I have is an array with some variables. I can iterate to get the values of those vars but what I need is actually their names (values will be used elsewhere).
Going with var[i] won't work cause I will have different names. I guess I could workaround this by creating another array with the names - something similar to this:
Getting variable values from variable names listed in array in Bash
But I'm wondering if there is a better way to do this.
var1=$'1'
var2=$'2'
var3=$'3'
Array=( $var1 $var2 $var3)
for ((i=0; i<${#Array[#]}; i++))
do
echo ${Array[i]}
done
Is:
>1
>2
>3
Should be:
>var1
>var2
>var3
It sounds like you want an associative array.
# to set values over time
declare -A Array=( ) || { echo "ERROR: Need bash 4.0 or newer" >&2; exit 1; }
Array[var1]=1
Array[var2]=2
Array[var3]=3
This can also be assigned at once:
# or as just one assignment
declare -A Array=( [var1]=1 [var2]=2 [var3]=3 )
Either way, one can iterate over the keys with "${!Array[#]}", and retrieve the value for a key with ${Array[key]}:
for var in "${!Array[#]}"; do
val="${Array[$var]}"
echo "$var -> $val"
done
...will, after either of the assignments up top, properly emit:
var1 -> 1
var2 -> 2
var3 -> 3
What about this solution?
#!/bin/bash
var1=$'1'
var2=$'2'
var3=$'3'
Array=( var1 var2 var3 )
for var in "${Array[#]}"; do
echo "$var = ${!var}"
done
The idea just consists in putting your variable names in the array, then relying on the indirection feature of Bash.
But as pointed out by #CharlesDuffy, the use of associative arrays sounds better adapted to the OP's use case.
Also, this related article may be worth reading: How can I use variable variables… or associative arrays?
I'm a few weeks into bash scripting and I haven't advanced enough yet to get my head wrapped around this problem. Any help would be appreciated!
I have a "script.conf" file that contains the following:
key1=value1
key2=${HOME}/Folder
key3=( "k3v1" "k3 v2" "k3v3")
key4=( "k4v1"
"k4 v2"
"k4v3"
)
key5=value5
#key6="Do Not Include Me"
In a bash script, I want to read the contents of this script.conf file into an array. I've learned how to handle the scenarios for keys 1, 2, 3, and 5, but the key4 scenario throws a wrench into it with it spanning across multiple lines.
I've been exploring the use of sed -n '/=\s*[(]/,/[)]/{/' which does capture key4 and its value, but I can't figure out how to mix this so that the other keys are also captured in the matches. The range syntax is also new to me, so I haven't figured out how to separate the key/value. I feel like there is an easy regex that would accomplish what I want... in plain-text: "find and group the pattern ^(.*)= (for the key), then group everything after the '=' char until another ^(.*)= match is found, rinse and repeat". I guess if I do this, I need to change the while read line to not handle the key/value separation for me (I'll be looking into this while I'm waiting for a response). BTW, I think a solution where the value of key4 is flattened (new lines removed) would be acceptable; I know for key3 I have to store the value as a string and then convert it to an array later when I want to iterate over it since an array element apparently can't contain a list.
Am I on the right path with sed or is this a job for awk or some other tool? (I haven't ventured into awk yet). Is there an easier approach that I'm missing because I'm too deep into the forest (like changing the while read line in the LoadConfigFile function)?
Here is the code that I have so far in script.sh for processing and capturing the other pairs into the $config array:
__AppDir=$(dirname $0)
__AppName=${__ScriptName%.*}
typeset -A config #init config array
config=( #Setting Default Config values
[key1]="defaultValue1"
[key2]="${HOME}/defaultFolder"
[QuietMode]=0
[Verbose]=0 #Ex. Usage: [[ "${config[Verbose]}" -gt 0 ]] && echo ">>>Debug print"
)
function LoadConfigFile() {
local cfgFile="${1}"
shopt -s extglob #Needed to remove trailing spaces
if [ -f ${cfgFile} ]; then
while IFS='=' read -r key value; do
if [[ "${key:0:1}" == "#" ]]; then
#echo "Skipping Comment line: ${key}"
elif [ "${key:-EMPTY}" != "EMPTY" ]; then
value="${value%%\#*}" # Delete in-line, right comments
value="${value%%*( )}" # Delete trailing spaces
value="${value%%( )*}" # Delete leading spaces
#value="${value%\"*}" # Delete opening string quotes
#value="${value#\"*}" # Delete closing string quotes
#Manipulate any variables included in the value so that they can be expanded correctly
# - value must be stored in the format: "${var1}". `backticks`, "$var2", and "doubleQuotes" are left as is
value="${value//\"/\\\"}" # Escape double quotes for eval
value="${value//\`/\\\`}" # Escape backticks for eval
value="${value//\$/\\\$}" # Escape ALL '$' for eval
value="${value//\\\${/\${}" # Undo the protection of '$' if it was followed by a '{'
value=$(eval "printf '%s\n' \"${value}\"")
config[${key}]=${value} #Store the value into the config array at the specified key
echo " >>>DBG: Key = ${key}, Value = ${value}"
#else
# echo "Skipped Empty Key"
fi
done < "${cfgFile}"
fi
}
CONFIG_FILE=${__AppDir}/${__AppName}.conf
echo "Config File # ${CONFIG_FILE}"
LoadConfigFile ${CONFIG_FILE}
#Print elements of $config
echo "Script Config Values:"
echo "----------------------------"
for key in "${!config[#]}"; do #The '!' char gets an array of the keys, without it, we would get an array of the values
printf " %-20s = %s\n" "${key}" "${config[${key}]}"
done
echo "------ End Script Config ------"
#To convert to an array...
declare -a valAsArray=${config[RequiredAppPackages]} #Convert the value from a string to an array
echo "Count = ${#valAsArray[#]}"
for itemCfg in "${valAsArray[#]}"; do
echo " item = ${itemCfg}"
done
As I mentioned before, I'm just starting to learn bash and Linux scripting in general, so if you see that I'm doing some taboo things in other areas of my code too, please feel free to provide feedback in the comments... I don't want to start bad habits early on :-).
*If it matters, the OS is Ubuntu 14.04.
EDIT:
As requested, after reading the script.conf file, I would like for the elements in $config[#] to be equivalent to the following:
typeset -A config #init config array
config=(
[key1]="value1"
[key2]="${HOME}/Folder"
[key3]="( \"k3v1\" \"k3 v2\" \"k3v3\" )"
[key4]="( \"k4v1\" \"k4 v2\" \"k4v3\" )"
[key5]="value5"
)
I want to be able to convert the values of elements 'key4' and 'key3' into an array and iterated over them the same way in the following code:
declare -a keyValAsArray=${config[keyN]} #Convert the value from a string to an array
echo "Count = ${#keyValAsArray[#]}"
for item in "${keyValAsArray[#]}"; do
echo " item = ${item}"
done
I don't think it matters if \n is preserved for key4's value or not... that depends on if declare has a problem with it.
A shell is an environment from which to call tools with a language to sequence those calls. It is NOT a tool to manipulate text. The standard UNIX tool to manipulate text is awk. Trying to manipulate text in shell IS a bad habit, see why-is-using-a-shell-loop-to-process-text-considered-bad-practice for SOME of the reasons why
You still didn't post the expected result of populating the config array so I'm not sure but I think this is what you wanted:
$ cat tst.sh
declare -A config="( $(awk '
{ gsub(/^[[:space:]]+|([[:space:]]+|#.*)$/,"") }
!NF { next }
/^[^="]+=/ {
name = gensub(/=.*/,"",1)
value = gensub(/^[^=]+=/,"",1)
n2v[name] = value
next
}
{ n2v[name] = n2v[name] OFS $0 }
END {
for (name in n2v) {
value = gensub(/"/,"\\\\&","g",n2v[name])
printf "[%s]=\"%s\"\n", name, value
}
}
' script.conf
) )"
declare -p config
$ ./tst.sh
declare -A config='([key5]="value5" [key4]="( \"k4v1\" \"k4 v2\" \"k4v3\" )" [key3]="( \"k3v1\" \"k3 v2\" \"k3v3\")" [key2]="/home/Ed/Folder" [key1]="value1" )'
The above uses GNU awk for gensub(), with other awks you'd use [g]sub() instead.
Content of the file:
Class_one 23
Class_two 17
Class-three 22
..
How to read the file and split all lines into a two-dimensional array? like java. Like:
arr[0][0] = Class_one arr[0][1] = 23
arr[1][0] = Class_two arr[1][1] = 17
thanks.
GNU bash has no two-dimensional array. A workaround is an associative array.
#!/bin/bash
declare -A arr # declare an associative array
declare -i c=0
# read from stdin (from file)
while read -r label number; do
arr[$c,0]="$label"; arr[$c,1]="$number"
c=c+1
done < file
# print array arr
for ((i=0;i<${#arr[#]}/2;i++)); do
echo "${arr[$i,0]} ${arr[$i,1]}"
done
See: help declare and man bash
#Cyrus's approach involves associative arrays which is noticeably only in bash 4.0 and up. Below is what works for bash sub-4.0. Note that nowadays Mac's are still shipped w/ bash 3.x.
#!/bin/bash
l=0; while read -a a$l; do
let l++;
done < ${data_file_name}
## now everything is stored in the 2D array ${a};
## $(($l+1)) is #rows, and ${#a0[#]} is #cols;
## elements can be accessed in the form of "ai[j]";
## e.g., a0[0] is the element at (0,0);
## but to access "ai[j]" using var ${i} and ${j}
## as indexes can be a just little tricky
echo "#rows: $((l+1))"
echo "#cols: ${#a0[#]}"$'\n'
echo "element at (0, 0): ${a0[0]}"
## the following shows how to access an element at (i,j)
i=1; j=1
tmp_a="a${i}[${j}]"; echo "element at ($i, $j): ${!tmp_a}"$'\n'
## the following shows how to iterate through the 2D array
echo "all elements printed from top left to bottom right:"
for i in `eval echo {0..$l}`; do
for j in `eval echo {0.."$((${#a0[#]}-1))"}`; do
tmp_a="a${i}[${j}]"; echo ${!tmp_a}
done
done
I have two arrays that I want to loop in. I construct those properly and before going into for loop, I do echo them to be sure everything is ok with arrays.
But when I run the script, it outputs an error:
l<=: syntax error: operand expected (error token is "<="
I consulted the mighty Google and I understood it suffers from the lack of the second variable, but I mentioned earlier I do echo the values and everything seems to be OK. Here is the snippet..
#!/bin/bash
k=0
#this loop is just for being sure array is loaded
while [[ $k -le ${#hitEnd[#]} ]]
do
echo "hitEnd is: ${hitEnd[k]} and hitStart is: ${hitStart[k]}"
# here outputs the values correct
k=$((k+1))
done
k=0
for ((l=${hitStart[k]};l<=${hitEnd[k]};l++)) ; do //this is error line..
let array[l]++
k=$((k+1))
done
The variables in the for loop are echoed correctly but for loop won't work.. where am I wrong?
#
as gniourf_gniourf answered:
"... At some point, k will reach the value ${#hitEnd[#]}, and this is
exactly when hitEnd[k] is not defined and expands to an empty string!
Bang!"
meaning error output is displayed not at the beginning of the loop, but when k has a greater value than array's indices, pointing an index that array does not include...
That's because at some point ${hitEnd[k]} expands to nothing (it is undefined). I get the same error with ((l<=)). You should write your for loop as:
k=0
for ((l=${hitStart[0]};k<${#hitEnd[#]} && l<=${hitEnd[k]};l++)); do
so as to always have an index k that corresponds to a defined field in the array ${hitEnd[#]}.
Also, instead of
k=$((k+1))
you can just write
((++k))
Done!
Your script revised using better modern bash practice:
#!/bin/bash
k=0
#this loop is just for being sure array is loaded
while ((k<=${#hitEnd[#]})); do
echo "hitEnd is: ${hitEnd[k]} and hitStart is: ${hitStart[k]}"
# here outputs the values correct
((++k))
done
k=0
for ((l=hitStart[0];k<${#hitEnd[#]} && l<=hitEnd[k];++l)); do
((++array[l]))
((++k))
done
Now, I'm not too sure the for loop does exactly what you want it to... Don't you mean this instead?
#!/bin/bash
# define arrays hitStart[#] and hitEnd[#]...
# define array array[#]
#this loop is just for being sure array is loaded
for ((k=0;k<${#hitEnd[#]};++k)); do
echo "hitEnd is: ${hitEnd[k]} and hitStart is: ${hitStart[k]}"
# here outputs the values correct
((++k))
done
for ((k=0;k<${#hitEnd[#]};++k)); do
for ((l=hitStart[k];l<=hitEnd[k];++l)); do
((++array[l]))
done
done
A bit bandaid-y, but you rewrite your for-loop into a while loop:
l="${hitStart[k]}"
while [[ "$l" -le "${hitEnd[k]}" ]]; do
let array[l]++
k=$((k+1))
l=$((l+1))
done