I have a json in the following format. I want to iterate over this json file
{
"atest_engine": { "version": "96" },
"a_kdfvm": { "version": "68" },
"aseft_api": { "version": "" },
"push_psservice": { "version": "68" },
}
I tried jq utility and my script is as follows.
count=$( jq '. | length' test.json )
echo $count
for((i=0;i<$count;i++))
do
name=$(cat test.json | jq '.|keys['${i}']')
version=$(cat test.json | jq '.|keys['${i}'].version')
echo $name
echo $version
done
I am getting count and name properly but not able to fetch version information. How can I get it. I am new to scripting and any help in this regard is greatly appreciated.Thanks in Advance.
input.json
{
"atest_engine": { "version": "96" },
"a_kdfvm": { "version": "68" },
"aseft_api": { "version": "" },
"push_psservice": { "version": "68" }
}
command
jq -r 'to_entries[] | "\(.key)\t\(.value.version)"' input.json |
while read name version
do
echo "name:" $name
echo "version:" $version
done
result
name: atest_engine
version: 96
name: a_kdfvm
version: 68
name: aseft_api
version:
name: push_psservice
version: 68
First up your JSON example seems slightly malformed - the push_psservice line has a comma after it but this is most likely a typo.
You might find it easier to turn your object's fields into an array using jq's to_entries (see https://stackoverflow.com/a/24254365/4513656 ) e.g.:
to_entries | .[0].key
to_entries | .[0].value.version
Try this on https://jqplay.org/ .
Related
Json file:
{
"A": "jack",
"B": [
{
"name": "jack/jil",
"version": "0.1"
},
{
"name": "went/well",
"version": "1.2"
}
]
}
now I need to update every objects version: "$version-dev" where name starts from jack while retaining rest of the json intact.
the closest I can get.
jq '.B[] | select(.name|test("jack.")) | .version += "-dev"' old.json > new.json
in the above command I'm only getting the that particular object with updated value in the new.json but I need the whole json too.
any suggestions
You need to put parantheses around the whole selection to be updated: (… | .version) += …
jq '(.B[] | select(.name|test("jack.")) | .version) += "-dev"' old.json > new.json
{
"A": "jack",
"B": [
{
"name": "jack/jil",
"version": "0.1-dev"
},
{
"name": "went/well",
"version": "1.2"
}
]
}
Demo
I am trying to get values base on key of this JSON:
{
"streams": {
"vs-first": {
"version": "2.33.0",
"branch": "ewewew",
"hash": "ewewewewe",
"widgets": []
},
"vs-second": {
"version": "1.58.0",
"branch": "ewewew",
"hash": "ewewew",
"widgets": []
},
"vs-third": {
"version": "1.42.0",
"branch": "ewewew",
"hash": "ewewe",
"widgets": []
},
"vs-fourth": {
"version": "1.58.0",
"branch": "eewfwfef",
"hash": "vvfffsfsf",
"widgets": []
},
"vs-fifth": {
"version": "1.39.0",
"branch": "fvrvvsdvds",
"hash": "vvsvdsvds",
"widgets": [
"1",
"2",
"3",
"4"
]
}
}
}
This is my Script implementation:
jq -r '.streams|keys[]' $vsconfig | while read key ; do
if [ $key == "[" ] || [ $key == "]" ]; then
continue
fi
if [ $key == "vs-first" ]; then
version=$(jq -r '.streams.vs-first.version' $vsconfig)
branch=$(jq -r '.streams.vs-first.branch' $vsconfig)
hash=$(jq -r '.streams.vs-first.hash' $vsconfig)
filename="one_file-$version-$branch-$hash.zip"
createdUrl="$someurl/$version/$filename"
curl $createdUrl --output ./som/random/dir --create-dirs
...
else
version=$(jq -r --arg v keyvar $key 'streams.[$keyvar].branch' $vsconfig)
branch=`jq --arg keyvar "streams.$key.branch" '$keyvar' $vsconfig`
hash=`jq --arg keyvar "streams.$key.hash" '$keyvar' $vsconfig`
filename = "$key-$version"
if [ $branch == "some_branch" ]; then
filename="one_file-$version-$branch-$hash.zip"
else
filename="$filename.zip"
fi
curl $createdUrl --output ./som/random/dir --create-dirs
fi
echo "Version: $version Branch: $branch Hash: $hash"
done
I've tried multiple formats, i.e:
version=$(jq -r --arg v keyvar $key 'streams.[$keyvar].branch' $vsconfig)
And:
branch=`jq --arg keyvar "streams.$key.branch" '$keyvar' $vsconfig`
It gives this error:
jq: error: support/0 is not defined at <top-level>, line 1:
jq is quite sophisticated. The best bet is probably to do all of this inside a single jq invocation and have minimal or no bash scripting.
Values only
To start with, you can get each version/branch/hash object by applying [] to the .streams object. Using [] on an object—or key/value map—extracts the values and throws away the keys.
$ jq -c '.streams[]' vs.json
{"version":"2.33.0","branch":"ewewew","hash":"ewewewewe","widgets":[]}
{"version":"1.58.0","branch":"ewewew","hash":"ewewew","widgets":[]}
{"version":"1.42.0","branch":"ewewew","hash":"ewewe","widgets":[]}
{"version":"1.58.0","branch":"eewfwfef","hash":"vvfffsfsf","widgets":[]}
{"version":"1.39.0","branch":"fvrvvsdvds","hash":"vvsvdsvds","widgets":[]}
Then you can get the individual fields you're interested in by piping the above objects to a filter which grabs .version, .branch, and .hash and throws the three values into an array:
$ jq -c '.streams[] | [.version, .branch, .hash]' vs.json
["2.33.0","ewewew","ewewewewe"]
["1.58.0","ewewew","ewewew"]
["1.42.0","ewewew","ewewe"]
["1.58.0","eewfwfef","vvfffsfsf"]
["1.39.0","fvrvvsdvds","vvsvdsvds"]
To get it to format the results you can generate strings instead of lists and use \(...) to embed values. The -r flag tells it to print raw results: print the strings without quotes, in other words.
$ jq -r '.streams[] | "Version: \(.version) Branch: \(.branch) Hash: \(.hash)"' vs.json
Version: 2.33.0 Branch: ewewew Hash: ewewewewe
Version: 1.58.0 Branch: ewewew Hash: ewewew
Version: 1.42.0 Branch: ewewew Hash: ewewe
Version: 1.58.0 Branch: eewfwfef Hash: vvfffsfsf
Version: 1.39.0 Branch: fvrvvsdvds Hash: vvsvdsvds
Keys and values
To add the keys into the mix you can use to_entries, which extracts the key/value pairs from an object:
$ jq -c '.streams | to_entries[]' vs.json
{"key":"vs-first","value":{"version":"2.33.0","branch":"ewewew","hash":"ewewewewe","widgets":[]}}
{"key":"vs-second","value":{"version":"1.58.0","branch":"ewewew","hash":"ewewew","widgets":[]}}
{"key":"vs-third","value":{"version":"1.42.0","branch":"ewewew","hash":"ewewe","widgets":[]}}
{"key":"vs-fourth","value":{"version":"1.58.0","branch":"eewfwfef","hash":"vvfffsfsf","widgets":[]}}
{"key":"vs-fifth","value":{"version":"1.39.0","branch":"fvrvvsdvds","hash":"vvsvdsvds","widgets":[]}}
Pulling out the different fields then becomes:
$ jq -c '.streams | to_entries[] | [.key, .value.version, .value.branch, .value.hash]' vs.json
["vs-first","2.33.0","ewewew","ewewewewe"]
["vs-second","1.58.0","ewewew","ewewew"]
["vs-third","1.42.0","ewewew","ewewe"]
["vs-fourth","1.58.0","eewfwfef","vvfffsfsf"]
["vs-fifth","1.39.0","fvrvvsdvds","vvsvdsvds"]
Or equivalently, with the repeated .value lookups refactored out:
jq -c '.streams | to_entries[] | [.key, (.value | .version, .branch, .hash)]' vs.json
["vs-first","2.33.0","ewewew","ewewewewe"]
["vs-second","1.58.0","ewewew","ewewew"]
["vs-third","1.42.0","ewewew","ewewe"]
["vs-fourth","1.58.0","eewfwfef","vvfffsfsf"]
["vs-fifth","1.39.0","fvrvvsdvds","vvsvdsvds"]
Add bash processing
jq can't do everything, so if you do want to get the results out to bash to do additional processing—e.g., call curl—you could use -r to print each value on a separate line and use read to read the lines into variables. It would look something like this:
jq -r '.streams | to_entries[] | .key, (.value | .version, .branch, .hash)' vs.json |
while read -r key &&
read -r version &&
read -r branch &&
read -r hash
do
...
done
i ve this shell script ; it's a loop which set in the variable "a" each time result :
declare -a names=("one" "two" "three" "four")
for item in "${names[#]}";
do
a="$(cat <<-EOF
{
"NAME": "${item}_ABC",
"CHANGED": "${item}_CHANGING",
"VERSION": "${item}_GC",
}
EOF
)"
done
echo $a
My Purpose is how to change "a" by a dynamic variable name which be $item_MYPREFIX
(concatination :$item + _MYPREFIX )
So that my code would be generic , something like this :
for item in "${names[#]}";
do
$item_MYPREFIX="$(cat <<-EOF
{
"NAME": "${item}_ABC",
"CHANGED": "${item}_CHANGING",
"VERSION": "${item}_GC",
}
EOF
)"
done
and i would be able to display each variable : echo $one_MYPREFIX , echo $two_MYPREFIX ....
Of course it's not alerady working
Suggestions , to crrect it ?
Use an associative array.
declare -A foo
for item in "${names[#]}";
do
foo[$item]="{
\"NAME\": \"${item}_ABC\",
\"CHANGED\": \"${item}_CHANGING\",
\"VERSION\": \"${item}_GC\"
}"
done
Try it like this
#!/bin/bash
for item in "${names[#]}"; do
varname=${item}_MYPREFIX
declare $varname="
{
\"NAME\": \"${item}_ABC\",
\"CHANGED\": \"${item}_CHANGING\",
\"VERSION\": \"${item}_GC\",
}
"
echo "${!varname}"
done
But better use arrays for that.
#!/bin/bash
declare -A array
names=("one" "two" "three" "four")
for item in "${names[#]}"; do
indexname=${item}_MYPREFIX
array[$indexname]="
{
\"NAME\": \"${item}_ABC\",
\"CHANGED\": \"${item}_CHANGING\",
\"VERSION\": \"${item}_GC\",
}
"
echo "${array[$indexname]}"
done
This is really not best practice, but you can (in bash) do:
$ cat a.sh
#!/bin/bash
declare -a names=("one" "two" "three" "four")
for item in "${names[#]}"; do
eval "read -d '' ${item}_MYPREFIX" << EOF
{
"NAME": "${item}_ABC",
"CHANGED": "${item}_CHANGING",
"VERSION": "${item}_GC",
}
EOF
done
for item in "${names[#]}"; do
k="${item}_MYPREFIX"
echo "$k = ${!k}"
done
$ ./a.sh
one_MYPREFIX = {
"NAME": "one_ABC",
"CHANGED": "one_CHANGING",
"VERSION": "one_GC",
}
two_MYPREFIX = {
"NAME": "two_ABC",
"CHANGED": "two_CHANGING",
"VERSION": "two_GC",
}
three_MYPREFIX = {
"NAME": "three_ABC",
"CHANGED": "three_CHANGING",
"VERSION": "three_GC",
}
four_MYPREFIX = {
"NAME": "four_ABC",
"CHANGED": "four_CHANGING",
"VERSION": "four_GC",
}
I believe the only bashism there (besides the existence of arrays, but several shells have arrays) is the usage of ${!...} indirection, but that's just for output and is not really necessary. However, since you're using a shell that supports arrays, you might as well not do this at all. Instead of creating a variable named "two_MYPREFIX", you ought to create an array and store that value in either index 2, or use an associative array and store in with index "two". That would be much cleaner than using eval.
The heredocs is not needed, try this.
#!/usr/bin/env bash
declare -a names=("one" "two" "three" "four")
declare -a prefix=(foo bar baz more)
for item in "${!names[#]}"; do
array+=("${prefix[$item]} = {
"NAME": "${names[$item]}_ABC",
"CHANGED": "${names[$item]}_CHANGING",
"VERSION": "${names[$item]}_GC",
}"
)
done
printf '%s\n' "${array[#]}"
Output
foo = {
NAME: one_ABC,
CHANGED: one_CHANGING,
VERSION: one_GC,
}
bar = {
NAME: two_ABC,
CHANGED: two_CHANGING,
VERSION: two_GC,
}
baz = {
NAME: three_ABC,
CHANGED: three_CHANGING,
VERSION: three_GC,
}
more = {
NAME: four_ABC,
CHANGED: four_CHANGING,
VERSION: four_GC,
}
Bash solution without eval, without associative array and using a template:
#!/usr/bin/env bash
declare -a names=("one" "two" "three" "four")
read -r -d '' template <<'EOF'
{
"NAME": "%q_ABC",
"CHANGED": "%q_CHANGING",
"VERSION": "%q_GC"
}
EOF
for item in "${names[#]}"; do
# shellcheck disable=SC2059 # format with a template variable
printf -v "${item}_MYPREFIX" "$template" "$item" "$item" "$item"
done
# Dump variables for debug
IFS=$'\n' read -r -d '' -a k < <(printf '%s_MYPREFIX\n' "${names[#]}")
typeset -p "${k[#]}"
Output:
declare -- one_MYPREFIX="{
\"NAME\": \"one_ABC\",
\"CHANGED\": \"one_CHANGING\",
\"VERSION\": \"one_GC\"
}"
declare -- two_MYPREFIX="{
\"NAME\": \"two_ABC\",
\"CHANGED\": \"two_CHANGING\",
\"VERSION\": \"two_GC\"
}"
declare -- three_MYPREFIX="{
\"NAME\": \"three_ABC\",
\"CHANGED\": \"three_CHANGING\",
\"VERSION\": \"three_GC\"
}"
declare -- four_MYPREFIX="{
\"NAME\": \"four_ABC\",
\"CHANGED\": \"four_CHANGING\",
\"VERSION\": \"four_GC\"
}"
I have a variable myStrthat contains the following value:
"app": {
"services": {
"app": [{
"groupID": "com.mycompany",
"artifactId": "myapp-versions",
"version": "1.0.0"
},
{
"groupID": "com.mycompany.xyz",
"artifactId": "car-stats",
"version": "1.0-master"
},
{
"groupID": "com.mycompany.service",
"artifactId": "my-differential-service",
"version": "1.0.0-master"
}
]
}
}
Now I want to replace the version of only my-differential-service artifactId to NEW_VERSION.
I tried using sed command on myStr variable but couldn't succeed as I am not much familiar with this command.
Can anyone please guide me on how should I proceed to achieve this?
Any help would be highly appreciated.
When you need sed for parsing this, look for the support of the option -z in your sed. This option ignores the special meaning of \n.
When you know that version is the first field after the artifactId you can try
new_version=1.0.1 # avoid uppercase variable names
echo "$myStr" |
sed -rz 's/("my-differential-service",[^:]*: ")[^"]*/\1'"${new_version}"'/'
When the order of fields can change, you might want to ask for jq in Jenkins or try awk.
When you want to use sed but don't have the -z option, you can translate first:
echo "$myStr" | tr '\n' '\r' | sed -r 's/..../' | tr '\r' '\n'
If jq is available, congrats! and please try the following:
echo "{ $myStr }" | jq '(.app.services.app[] | select(.artifactId == "my-differential-service") | .version) = "NEW_VERSION"'
which yields:
{
"app": {
"services": {
"app": [
{
"groupID": "com.mycompany",
"artifactId": "myapp-versions",
"version": "1.0.0"
},
{
"groupID": "com.mycompany.xyz",
"artifactId": "car-stats",
"version": "1.0-master"
},
{
"groupID": "com.mycompany.service",
"artifactId": "my-differential-service",
"version": "NEW_VERSION"
}
]
}
}
}
If you do not need the outermost curly braces, please remove them by bash's parameter expansion or something similar.
As a fallback, you can say with sed:
echo "$myStr" | sed '
:l
N
$!b l
s/\("my-differential-service"[^"]*"version": *\)"[^"]*"/\1"NEW_VERSION"/g'
Hope this helps.
I am using the bash to json parser jq
Considering the following command:
jq '. * .transitive | del(.transitive) | del(.scope,.scopedName)' package.json > package.github.json$$
And the following input:
{
"name": "navigation",
"transitive": {
"name": "navigation",
"scope": "bs",
"scopedName": "#bs/navigation"
}
}
I am trying to get the following output:
{
"name": "#bs/navigation"
}
Is there a way before doing the delete of .scopedName, to use it's value to set .name?
Transforming your input to your output is as simple as:
jq '{"name": .transitive.scopedName}'
...and of course you could just reorder things to set name before deleting transitive:
jq '.name=.transitive.scopedName | del(.transitive)'
That said, if you really want to use del() first, you can save content in a variable and use it later:
jq '
.transitive as $transitive |
del(.transitive) |
.name=$transitive.scopedName
'