Shell : create and assign variables inside for loop - linux

i ve this shell script ; it's a loop which set in the variable "a" each time result :
declare -a names=("one" "two" "three" "four")
for item in "${names[#]}";
do
a="$(cat <<-EOF
{
"NAME": "${item}_ABC",
"CHANGED": "${item}_CHANGING",
"VERSION": "${item}_GC",
}
EOF
)"
done
echo $a
My Purpose is how to change "a" by a dynamic variable name which be $item_MYPREFIX
(concatination :$item + _MYPREFIX )
So that my code would be generic , something like this :
for item in "${names[#]}";
do
$item_MYPREFIX="$(cat <<-EOF
{
"NAME": "${item}_ABC",
"CHANGED": "${item}_CHANGING",
"VERSION": "${item}_GC",
}
EOF
)"
done
and i would be able to display each variable : echo $one_MYPREFIX , echo $two_MYPREFIX ....
Of course it's not alerady working
Suggestions , to crrect it ?

Use an associative array.
declare -A foo
for item in "${names[#]}";
do
foo[$item]="{
\"NAME\": \"${item}_ABC\",
\"CHANGED\": \"${item}_CHANGING\",
\"VERSION\": \"${item}_GC\"
}"
done

Try it like this
#!/bin/bash
for item in "${names[#]}"; do
varname=${item}_MYPREFIX
declare $varname="
{
\"NAME\": \"${item}_ABC\",
\"CHANGED\": \"${item}_CHANGING\",
\"VERSION\": \"${item}_GC\",
}
"
echo "${!varname}"
done
But better use arrays for that.
#!/bin/bash
declare -A array
names=("one" "two" "three" "four")
for item in "${names[#]}"; do
indexname=${item}_MYPREFIX
array[$indexname]="
{
\"NAME\": \"${item}_ABC\",
\"CHANGED\": \"${item}_CHANGING\",
\"VERSION\": \"${item}_GC\",
}
"
echo "${array[$indexname]}"
done

This is really not best practice, but you can (in bash) do:
$ cat a.sh
#!/bin/bash
declare -a names=("one" "two" "three" "four")
for item in "${names[#]}"; do
eval "read -d '' ${item}_MYPREFIX" << EOF
{
"NAME": "${item}_ABC",
"CHANGED": "${item}_CHANGING",
"VERSION": "${item}_GC",
}
EOF
done
for item in "${names[#]}"; do
k="${item}_MYPREFIX"
echo "$k = ${!k}"
done
$ ./a.sh
one_MYPREFIX = {
"NAME": "one_ABC",
"CHANGED": "one_CHANGING",
"VERSION": "one_GC",
}
two_MYPREFIX = {
"NAME": "two_ABC",
"CHANGED": "two_CHANGING",
"VERSION": "two_GC",
}
three_MYPREFIX = {
"NAME": "three_ABC",
"CHANGED": "three_CHANGING",
"VERSION": "three_GC",
}
four_MYPREFIX = {
"NAME": "four_ABC",
"CHANGED": "four_CHANGING",
"VERSION": "four_GC",
}
I believe the only bashism there (besides the existence of arrays, but several shells have arrays) is the usage of ${!...} indirection, but that's just for output and is not really necessary. However, since you're using a shell that supports arrays, you might as well not do this at all. Instead of creating a variable named "two_MYPREFIX", you ought to create an array and store that value in either index 2, or use an associative array and store in with index "two". That would be much cleaner than using eval.

The heredocs is not needed, try this.
#!/usr/bin/env bash
declare -a names=("one" "two" "three" "four")
declare -a prefix=(foo bar baz more)
for item in "${!names[#]}"; do
array+=("${prefix[$item]} = {
"NAME": "${names[$item]}_ABC",
"CHANGED": "${names[$item]}_CHANGING",
"VERSION": "${names[$item]}_GC",
}"
)
done
printf '%s\n' "${array[#]}"
Output
foo = {
NAME: one_ABC,
CHANGED: one_CHANGING,
VERSION: one_GC,
}
bar = {
NAME: two_ABC,
CHANGED: two_CHANGING,
VERSION: two_GC,
}
baz = {
NAME: three_ABC,
CHANGED: three_CHANGING,
VERSION: three_GC,
}
more = {
NAME: four_ABC,
CHANGED: four_CHANGING,
VERSION: four_GC,
}

Bash solution without eval, without associative array and using a template:
#!/usr/bin/env bash
declare -a names=("one" "two" "three" "four")
read -r -d '' template <<'EOF'
{
"NAME": "%q_ABC",
"CHANGED": "%q_CHANGING",
"VERSION": "%q_GC"
}
EOF
for item in "${names[#]}"; do
# shellcheck disable=SC2059 # format with a template variable
printf -v "${item}_MYPREFIX" "$template" "$item" "$item" "$item"
done
# Dump variables for debug
IFS=$'\n' read -r -d '' -a k < <(printf '%s_MYPREFIX\n' "${names[#]}")
typeset -p "${k[#]}"
Output:
declare -- one_MYPREFIX="{
\"NAME\": \"one_ABC\",
\"CHANGED\": \"one_CHANGING\",
\"VERSION\": \"one_GC\"
}"
declare -- two_MYPREFIX="{
\"NAME\": \"two_ABC\",
\"CHANGED\": \"two_CHANGING\",
\"VERSION\": \"two_GC\"
}"
declare -- three_MYPREFIX="{
\"NAME\": \"three_ABC\",
\"CHANGED\": \"three_CHANGING\",
\"VERSION\": \"three_GC\"
}"
declare -- four_MYPREFIX="{
\"NAME\": \"four_ABC\",
\"CHANGED\": \"four_CHANGING\",
\"VERSION\": \"four_GC\"
}"

Related

Accessing JQ Variables Using Arguments

I am trying to get values base on key of this JSON:
{
"streams": {
"vs-first": {
"version": "2.33.0",
"branch": "ewewew",
"hash": "ewewewewe",
"widgets": []
},
"vs-second": {
"version": "1.58.0",
"branch": "ewewew",
"hash": "ewewew",
"widgets": []
},
"vs-third": {
"version": "1.42.0",
"branch": "ewewew",
"hash": "ewewe",
"widgets": []
},
"vs-fourth": {
"version": "1.58.0",
"branch": "eewfwfef",
"hash": "vvfffsfsf",
"widgets": []
},
"vs-fifth": {
"version": "1.39.0",
"branch": "fvrvvsdvds",
"hash": "vvsvdsvds",
"widgets": [
"1",
"2",
"3",
"4"
]
}
}
}
This is my Script implementation:
jq -r '.streams|keys[]' $vsconfig | while read key ; do
if [ $key == "[" ] || [ $key == "]" ]; then
continue
fi
if [ $key == "vs-first" ]; then
version=$(jq -r '.streams.vs-first.version' $vsconfig)
branch=$(jq -r '.streams.vs-first.branch' $vsconfig)
hash=$(jq -r '.streams.vs-first.hash' $vsconfig)
filename="one_file-$version-$branch-$hash.zip"
createdUrl="$someurl/$version/$filename"
curl $createdUrl --output ./som/random/dir --create-dirs
...
else
version=$(jq -r --arg v keyvar $key 'streams.[$keyvar].branch' $vsconfig)
branch=`jq --arg keyvar "streams.$key.branch" '$keyvar' $vsconfig`
hash=`jq --arg keyvar "streams.$key.hash" '$keyvar' $vsconfig`
filename = "$key-$version"
if [ $branch == "some_branch" ]; then
filename="one_file-$version-$branch-$hash.zip"
else
filename="$filename.zip"
fi
curl $createdUrl --output ./som/random/dir --create-dirs
fi
echo "Version: $version Branch: $branch Hash: $hash"
done
I've tried multiple formats, i.e:
version=$(jq -r --arg v keyvar $key 'streams.[$keyvar].branch' $vsconfig)
And:
branch=`jq --arg keyvar "streams.$key.branch" '$keyvar' $vsconfig`
It gives this error:
jq: error: support/0 is not defined at <top-level>, line 1:
jq is quite sophisticated. The best bet is probably to do all of this inside a single jq invocation and have minimal or no bash scripting.
Values only
To start with, you can get each version/branch/hash object by applying [] to the .streams object. Using [] on an object—or key/value map—extracts the values and throws away the keys.
$ jq -c '.streams[]' vs.json
{"version":"2.33.0","branch":"ewewew","hash":"ewewewewe","widgets":[]}
{"version":"1.58.0","branch":"ewewew","hash":"ewewew","widgets":[]}
{"version":"1.42.0","branch":"ewewew","hash":"ewewe","widgets":[]}
{"version":"1.58.0","branch":"eewfwfef","hash":"vvfffsfsf","widgets":[]}
{"version":"1.39.0","branch":"fvrvvsdvds","hash":"vvsvdsvds","widgets":[]}
Then you can get the individual fields you're interested in by piping the above objects to a filter which grabs .version, .branch, and .hash and throws the three values into an array:
$ jq -c '.streams[] | [.version, .branch, .hash]' vs.json
["2.33.0","ewewew","ewewewewe"]
["1.58.0","ewewew","ewewew"]
["1.42.0","ewewew","ewewe"]
["1.58.0","eewfwfef","vvfffsfsf"]
["1.39.0","fvrvvsdvds","vvsvdsvds"]
To get it to format the results you can generate strings instead of lists and use \(...) to embed values. The -r flag tells it to print raw results: print the strings without quotes, in other words.
$ jq -r '.streams[] | "Version: \(.version) Branch: \(.branch) Hash: \(.hash)"' vs.json
Version: 2.33.0 Branch: ewewew Hash: ewewewewe
Version: 1.58.0 Branch: ewewew Hash: ewewew
Version: 1.42.0 Branch: ewewew Hash: ewewe
Version: 1.58.0 Branch: eewfwfef Hash: vvfffsfsf
Version: 1.39.0 Branch: fvrvvsdvds Hash: vvsvdsvds
Keys and values
To add the keys into the mix you can use to_entries, which extracts the key/value pairs from an object:
$ jq -c '.streams | to_entries[]' vs.json
{"key":"vs-first","value":{"version":"2.33.0","branch":"ewewew","hash":"ewewewewe","widgets":[]}}
{"key":"vs-second","value":{"version":"1.58.0","branch":"ewewew","hash":"ewewew","widgets":[]}}
{"key":"vs-third","value":{"version":"1.42.0","branch":"ewewew","hash":"ewewe","widgets":[]}}
{"key":"vs-fourth","value":{"version":"1.58.0","branch":"eewfwfef","hash":"vvfffsfsf","widgets":[]}}
{"key":"vs-fifth","value":{"version":"1.39.0","branch":"fvrvvsdvds","hash":"vvsvdsvds","widgets":[]}}
Pulling out the different fields then becomes:
$ jq -c '.streams | to_entries[] | [.key, .value.version, .value.branch, .value.hash]' vs.json
["vs-first","2.33.0","ewewew","ewewewewe"]
["vs-second","1.58.0","ewewew","ewewew"]
["vs-third","1.42.0","ewewew","ewewe"]
["vs-fourth","1.58.0","eewfwfef","vvfffsfsf"]
["vs-fifth","1.39.0","fvrvvsdvds","vvsvdsvds"]
Or equivalently, with the repeated .value lookups refactored out:
jq -c '.streams | to_entries[] | [.key, (.value | .version, .branch, .hash)]' vs.json
["vs-first","2.33.0","ewewew","ewewewewe"]
["vs-second","1.58.0","ewewew","ewewew"]
["vs-third","1.42.0","ewewew","ewewe"]
["vs-fourth","1.58.0","eewfwfef","vvfffsfsf"]
["vs-fifth","1.39.0","fvrvvsdvds","vvsvdsvds"]
Add bash processing
jq can't do everything, so if you do want to get the results out to bash to do additional processing—e.g., call curl—you could use -r to print each value on a separate line and use read to read the lines into variables. It would look something like this:
jq -r '.streams | to_entries[] | .key, (.value | .version, .branch, .hash)' vs.json |
while read -r key &&
read -r version &&
read -r branch &&
read -r hash
do
...
done

export command with variable name and value from other variable declaration in bash script

I am writing a bash script to read configs from AWS System parameter parameter store in json text format and then export to linux environment variables.
In summary I have the following:
name='COMPANY_NAME'
value='System Manager'
command = "$name=$value"
echo $command
export $command
After sourcing the script and try the to access the var
echo $COMPANY_NAME # output System instead of System Manager
It print output as System instead of System Manager.
I understood that the above might cause by the empty space.
How can I escape the empty space and other special chars in the above export command?
Below I also attach my source script as a ref.
#!/usr/bin/env bash
## to update the main process with >> . file_name
##function definition
process_json(){
#$1: json string
#$2: indicee
#$3: env name
json=$1
i=$2
app_env=$3
element_name=$(echo $json | jq -r ".Parameters[$i].Name")
element_value=$(echo $json | jq -r ".Parameters[$i].Value")
env_var_name=$(variable_name "$element_name" "$app_env")
command="export $env_var_name=$element_value"
echo $command
export $env_var_name=$element_value
}
variable_name(){
var_name=$1 # "/app_env/name" come with double quote
app_env=$2
app_env_path="/$app_env/"
app_env_path_length=${#app_env_path}
env_var_name="${var_name:($app_env_path_length)}"
echo $env_var_name
}
console_log(){
display_log=1
if [ $display_log -eq 1 ]
then
echo $#
fi
}
fetch_config(){
app_env=$1
json_str="
{
\"Parameters\": [
{
\"Name\": \"/$app_env/APP_NAME\",
\"Type\": \"String\",
\"Value\": \"BookMeBus\"
},
{
\"Name\": \"/$app_env/APP_VERSION\",
\"Type\": \"String\",
\"Value\": \"StgVersion\"
},
{
\"Name\": \"/$app_env/COMPANY_NAME\",
\"Type\": \"String\",
\"Value\": \"System Manager\"
},
{
\"Name\": \"/$app_env/ERROR_NOTIFICATION\",
\"Type\": \"String\",
\"Value\": \"no\"
},
{
\"Name\": \"/$app_env/SLACK_CHANNEL\",
\"Type\": \"String\",
\"Value\": \"#error-staging\"
},
{
\"Name\": \"/$app_env/SLACK_WEB_HOOK\",
\"Type\": \"String\",
\"Value\": \"https://hooks.slack.com/services/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\"
}
]
}
"
# command_load_params="aws --region=ap-southeast-1 ssm get-parameters-by-path --path /$app_env"
# echo $command_load_params
# json_str=$($command_load_params)
echo $json_str
}
if [[ -z "${APP_ENV}" ]]; then
error="ENV['APP_ENV'] must be set "
echo $error
exit 1
elif [[ $APP_ENV != "staging" && $APP_ENV != "production" ]]; then
error="ENV['APP_ENV'] must be staging or production"
exit 1
fi
app_env=$APP_ENV
json=$(fetch_config "$app_env")
# console_log "json is: $json"
# echo requires
# console_log $json | jq '.Parameters'
# console_log $json | jq '[.Parameters[] | {Name: .Name, Value: .Value}] '
length=$(echo $json | jq '.Parameters | length')
# console_log "length=$length"
i=0
while [ $i -lt $length ]
do
# Passing variables to a function quote is required surround variable name
process_json "$json" "$i" "$app_env"
((i++))
done
below is the output of my script:
Try as below:
name='COMPANY_NAME'
value='System Manager'
command="$name=$value"
echo $command
export "$command"
Notice the change in third and last statement (important)
Explanation: Unquoted expansion treats white space as token to separate.
It's export command
No $; otherwise you are exporting the value of the variable, not the variable (name) itself.
See https://ss64.com/bash/export.html
You'll want to double quote your variables (especially in this case with export):
name="COMPANY_NAME"
value="System Manager"
command=$name=$value
echo "$command"
export "${command?}" # ${var?} silences warning
Then when you source it:
$ source script.sh
$ echo "$COMPANY_NAME"
System Manager
↳ https://github.com/koalaman/shellcheck/wiki/SC2163

Replace pom version using sed in Jenkinsfile

I have a variable myStrthat contains the following value:
"app": {
"services": {
"app": [{
"groupID": "com.mycompany",
"artifactId": "myapp-versions",
"version": "1.0.0"
},
{
"groupID": "com.mycompany.xyz",
"artifactId": "car-stats",
"version": "1.0-master"
},
{
"groupID": "com.mycompany.service",
"artifactId": "my-differential-service",
"version": "1.0.0-master"
}
]
}
}
Now I want to replace the version of only my-differential-service artifactId to NEW_VERSION.
I tried using sed command on myStr variable but couldn't succeed as I am not much familiar with this command.
Can anyone please guide me on how should I proceed to achieve this?
Any help would be highly appreciated.
When you need sed for parsing this, look for the support of the option -z in your sed. This option ignores the special meaning of \n.
When you know that version is the first field after the artifactId you can try
new_version=1.0.1 # avoid uppercase variable names
echo "$myStr" |
sed -rz 's/("my-differential-service",[^:]*: ")[^"]*/\1'"${new_version}"'/'
When the order of fields can change, you might want to ask for jq in Jenkins or try awk.
When you want to use sed but don't have the -z option, you can translate first:
echo "$myStr" | tr '\n' '\r' | sed -r 's/..../' | tr '\r' '\n'
If jq is available, congrats! and please try the following:
echo "{ $myStr }" | jq '(.app.services.app[] | select(.artifactId == "my-differential-service") | .version) = "NEW_VERSION"'
which yields:
{
"app": {
"services": {
"app": [
{
"groupID": "com.mycompany",
"artifactId": "myapp-versions",
"version": "1.0.0"
},
{
"groupID": "com.mycompany.xyz",
"artifactId": "car-stats",
"version": "1.0-master"
},
{
"groupID": "com.mycompany.service",
"artifactId": "my-differential-service",
"version": "NEW_VERSION"
}
]
}
}
}
If you do not need the outermost curly braces, please remove them by bash's parameter expansion or something similar.
As a fallback, you can say with sed:
echo "$myStr" | sed '
:l
N
$!b l
s/\("my-differential-service"[^"]*"version": *\)"[^"]*"/\1"NEW_VERSION"/g'
Hope this helps.

Reindexing of elastic data for all the indices of a specific pattern

I have multiple elastic indices. The name of the indices are of a specific format. Following are an example of my indices:
abc_2ab99742-94d2-43f8-a582-ce10a0f031dc;
abc_e8241182-1a40-410b-a95d-c883472444f4;
Now I need to reindex all the data in these indices. For this I have written a shell script, which is doing the following thing.
1. Loop for all indices
1.1 create a temporary index like abc_2ab99742-94d2-43f8-a582-ce10a0f031dc_tmp.
1.2 reindix all the data from the original index to temp.
1.3 delete and re-create the original index.
1.4 reindex the data from temp to original index.
1.5 delete the temporary index.
Following is the shell script , I have written for the same.
#!/bin/bash
ES_HOST="localhost"
ES_PORT="9200"
TMP="_tmp"
indices=$(curl -s "http://${ES_HOST}:${ES_PORT}/_cat/indices/abc_*?h=index" | egrep 'abc_[0-9a-f]{8}-([0-9a-f]{4}-){3}[0-9a-f]{8}*')
# do for all abc elastic indices
for index in $indices
do
echo "Reindex process starting for index: $index"
tmp_index=$index${TMP}
output=$(curl -X PUT "http://${ES_HOST}:${ES_PORT}/$tmp_index" -H 'Content-Type: application/json' -d'
{
"settings" : {
"index" : {
"number_of_shards" : 16,
"number_of_replicas" : 1
}
}
}')
echo "Temporary index: $tmp_index created with output: $output"
echo "Starting reindexing elastic data from original index:$index to temporary index:$tmp_index"
output=$(curl -X POST "http://${ES_HOST}:${ES_PORT}/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": '"$index"'
},
"dest": {
"index": '"$tmp_index"'
}
}
')
echo "Reindexing completed from original index:$index to temporary index:$tmp_index with output: $output"
echo "Deleting $index"
output=$(curl -X DELETE "http://${ES_HOST}:${ES_PORT}/$index")
echo "$index deleted with status: $output"
echo "Creating index: $index"
output=$(curl -X PUT "http://${ES_HOST}:${ES_PORT}/$index" -H 'Content-Type: application/json' -d'
{
"settings" : {
"index" : {
"number_of_shards" : 16,
"number_of_replicas" : 1
}
}
}')
echo "Index: $index creation status: $output"
echo "Starting reindexing elastic data from temporary index:$tmp_index to original index:$index"
output=$(curl -X POST "http://${ES_HOST}:${ES_PORT}/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": '"$tmp_index"'
},
"dest": {
"index": '"$index"'
}
}
')
echo "Reindexing completed from temporary index:$tmp_index to original index:$index with output: $output"
echo "Deleting $tmp_index"
output=$(curl -X DELETE "http://${ES_HOST}:${ES_PORT}/$tmp_index")
echo "$tmp_index deleted with status: $output"
done
But I am getting exception in the reindex command. Following is the exception
Reindexing completed from original index:abc_58b888be-a90f-e3be-838d-88877aee572c to temporary index:abc_58b888be-a90f-e3be-838d-88877aee572c_tmp with output: {"error":{"root_cause":[{"type":"parsing_exception","reason":"[reindex] failed to parse field [source]","line":4,"col":9}],"type":"parsing_exception","reason":"[reindex] failed to parse field [source]","line":4,"col":9,"caused_by":{"type":"json_parse_exception","reason":"Unrecognized token 'abc_58b888be': was expecting ('true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput#39913ba; line: 4, column: 31]"}},"status":400}
Can anybody help me, as I am not very good in shell scripting.
The problem is with your shell script, Check the following part;
output=$(curl -X POST "http://${ES_HOST}:${ES_PORT}/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": '"$index"'
},
"dest": {
"index": '"$tmp_index"'
}
}
')
Here you are suppose to post a json, but the json is not valid, Change as following, then your script will work:
output=$(curl -XPOST "http://${ES_HOST}:${ES_PORT}/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": "'"$index"'"
},
"dest": {
"index": "'"$tmp_index"'"
}
}
')
"index": "'"$index"'"
This is creating a valid key value pair , as per json format
Here's the same modified script to reindex elasticsearch indices.
#!/bin/bash
#ES_HOST="localhost"
#ES_PORT="9200"
TMP="_v2"
echo "**** Script Execution Started ****"
echo " "
echo "**** Fetching List of Indices Elasticsearch Reindexing ****"
curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc'
echo " "
sleep 5
indices_list=$(curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc' | awk '{print $3}' | sed -n '1!p')
#indices=$(curl -s "http://${ES_HOST}:${ES_PORT}/_cat/indices/abc_*?h=index" | egrep 'abc_[0-9a-f]{8}-([0-9a-f]{4}-){3}[0-9a-f]{8}*')
# do for all abc elastic indices
count=0
echo "$indices_list"
for index in $indices_list
do
echo " "
echo "**** Index No: $count ****"
echo "**** Present Index we are iterating: ${index} ****"
count=`expr $count + 1`
echo " "
echo " "
echo "Reindex process starting for index: $index"
tmp_index=$index${TMP}
output=$(curl -s -X PUT "http://localhost:9200/$tmp_index" -H 'Content-Type: application/json' -d'
{
"settings" : {
"index" : {
"number_of_shards" : 5,
"number_of_replicas" : 1
}
}
}')
echo " "
echo "Temporary index: $tmp_index created with output: $output"
echo "Starting reindexing elastic data from original index: $index to temporary index: $tmp_index"
output=$(curl -s -X POST "http://localhost:9200/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": "'$index'"
},
"dest": {
"index": "'$tmp_index'"
}
}
')
echo " "
echo "Reindexing completed from original index: $index to temporary index: $tmp_index with output: $output"
echo " "
echo "Deleting $index"
output=$(curl -s -X DELETE "http://localhost:9200/$index")
echo "$index deleted with status: $output"
echo " "
echo "Creating index: $index"
output=$(curl -s -X PUT "http://localhost:9200/$index" -H 'Content-Type: application/json' -d'
{
"settings" : {
"index" : {
"number_of_shards" : 5,
"number_of_replicas" : 1
}
}
}')
echo " "
echo "Index: $index creation status: $output"
echo " "
echo "Starting reindexing elastic data from temporary index: $tmp_index to original index: $index"
output=$(curl -s -X POST "http://localhost:9200/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": "'$tmp_index'"
},
"dest": {
"index": "'$index'"
}
}
')
echo " "
echo "Reindexing completed from temporary index:$tmp_index to original index:$index with output: $output"
echo " "
echo "Deleting $tmp_index"
output=$(curl -s -X DELETE "http://localhost:9200/$tmp_index")
echo "$tmp_index deleted with status: $output"
echo " "
done
echo " "
sleep 5
echo "**** Fetching List of Indices After Elasticsearch Reindexing ****"
curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc'
echo " "
sleep 5
echo " "
echo "**** Original indices list ****"
echo "$indices_list"
echo "**** No. of indices in original list: $count ****"
echo " "
count1=0
MIndices_list=$(curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc' | awk '{print $3}' | sed -n '1!p')
echo " "
echo "**** Modified indices list ****"
echo "$MIndices_list"
for j in $MIndices_list
do
count1=`expr $count1 + 1`
done
echo " "
echo "**** No. of indices in modified list: $count1 ****"
echo "**** Script Execution Ended ****"

Iterate over a json using shell

I have a json in the following format. I want to iterate over this json file
{
"atest_engine": { "version": "96" },
"a_kdfvm": { "version": "68" },
"aseft_api": { "version": "" },
"push_psservice": { "version": "68" },
}
I tried jq utility and my script is as follows.
count=$( jq '. | length' test.json )
echo $count
for((i=0;i<$count;i++))
do
name=$(cat test.json | jq '.|keys['${i}']')
version=$(cat test.json | jq '.|keys['${i}'].version')
echo $name
echo $version
done
I am getting count and name properly but not able to fetch version information. How can I get it. I am new to scripting and any help in this regard is greatly appreciated.Thanks in Advance.
input.json
{
"atest_engine": { "version": "96" },
"a_kdfvm": { "version": "68" },
"aseft_api": { "version": "" },
"push_psservice": { "version": "68" }
}
command
jq -r 'to_entries[] | "\(.key)\t\(.value.version)"' input.json |
while read name version
do
echo "name:" $name
echo "version:" $version
done
result
name: atest_engine
version: 96
name: a_kdfvm
version: 68
name: aseft_api
version:
name: push_psservice
version: 68
First up your JSON example seems slightly malformed - the push_psservice line has a comma after it but this is most likely a typo.
You might find it easier to turn your object's fields into an array using jq's to_entries (see https://stackoverflow.com/a/24254365/4513656 ) e.g.:
to_entries | .[0].key
to_entries | .[0].value.version
Try this on https://jqplay.org/ .

Resources