Accessing JQ Variables Using Arguments - linux

Question

I am trying to get values base on key of this JSON:
{
"streams": {
"vs-first": {
"version": "2.33.0",
"branch": "ewewew",
"hash": "ewewewewe",
"widgets": []
},
"vs-second": {
"version": "1.58.0",
"branch": "ewewew",
"hash": "ewewew",
"widgets": []
},
"vs-third": {
"version": "1.42.0",
"branch": "ewewew",
"hash": "ewewe",
"widgets": []
},
"vs-fourth": {
"version": "1.58.0",
"branch": "eewfwfef",
"hash": "vvfffsfsf",
"widgets": []
},
"vs-fifth": {
"version": "1.39.0",
"branch": "fvrvvsdvds",
"hash": "vvsvdsvds",
"widgets": [
"1",
"2",
"3",
"4"
]
}
}
}
This is my Script implementation:
jq -r '.streams|keys[]' $vsconfig | while read key ; do
if [ $key == "[" ] || [ $key == "]" ]; then
continue
fi
if [ $key == "vs-first" ]; then
version=$(jq -r '.streams.vs-first.version' $vsconfig)
branch=$(jq -r '.streams.vs-first.branch' $vsconfig)
hash=$(jq -r '.streams.vs-first.hash' $vsconfig)
filename="one_file-$version-$branch-$hash.zip"
createdUrl="$someurl/$version/$filename"
curl $createdUrl --output ./som/random/dir --create-dirs
...
else
version=$(jq -r --arg v keyvar $key 'streams.[$keyvar].branch' $vsconfig)
branch=`jq --arg keyvar "streams.$key.branch" '$keyvar' $vsconfig`
hash=`jq --arg keyvar "streams.$key.hash" '$keyvar' $vsconfig`
filename = "$key-$version"
if [ $branch == "some_branch" ]; then
filename="one_file-$version-$branch-$hash.zip"
else
filename="$filename.zip"
fi
curl $createdUrl --output ./som/random/dir --create-dirs
fi
echo "Version: $version Branch: $branch Hash: $hash"
done
I've tried multiple formats, i.e:
version=$(jq -r --arg v keyvar $key 'streams.[$keyvar].branch' $vsconfig)
And:
branch=`jq --arg keyvar "streams.$key.branch" '$keyvar' $vsconfig`
It gives this error:
jq: error: support/0 is not defined at <top-level>, line 1:

Answer 1

jq is quite sophisticated. The best bet is probably to do all of this inside a single jq invocation and have minimal or no bash scripting.
Values only
To start with, you can get each version/branch/hash object by applying [] to the .streams object. Using [] on an object—or key/value map—extracts the values and throws away the keys.
$ jq -c '.streams[]' vs.json
{"version":"2.33.0","branch":"ewewew","hash":"ewewewewe","widgets":[]}
{"version":"1.58.0","branch":"ewewew","hash":"ewewew","widgets":[]}
{"version":"1.42.0","branch":"ewewew","hash":"ewewe","widgets":[]}
{"version":"1.58.0","branch":"eewfwfef","hash":"vvfffsfsf","widgets":[]}
{"version":"1.39.0","branch":"fvrvvsdvds","hash":"vvsvdsvds","widgets":[]}
Then you can get the individual fields you're interested in by piping the above objects to a filter which grabs .version, .branch, and .hash and throws the three values into an array:
$ jq -c '.streams[] | [.version, .branch, .hash]' vs.json
["2.33.0","ewewew","ewewewewe"]
["1.58.0","ewewew","ewewew"]
["1.42.0","ewewew","ewewe"]
["1.58.0","eewfwfef","vvfffsfsf"]
["1.39.0","fvrvvsdvds","vvsvdsvds"]
To get it to format the results you can generate strings instead of lists and use \(...) to embed values. The -r flag tells it to print raw results: print the strings without quotes, in other words.
$ jq -r '.streams[] | "Version: \(.version) Branch: \(.branch) Hash: \(.hash)"' vs.json
Version: 2.33.0 Branch: ewewew Hash: ewewewewe
Version: 1.58.0 Branch: ewewew Hash: ewewew
Version: 1.42.0 Branch: ewewew Hash: ewewe
Version: 1.58.0 Branch: eewfwfef Hash: vvfffsfsf
Version: 1.39.0 Branch: fvrvvsdvds Hash: vvsvdsvds
Keys and values
To add the keys into the mix you can use to_entries, which extracts the key/value pairs from an object:
$ jq -c '.streams | to_entries[]' vs.json
{"key":"vs-first","value":{"version":"2.33.0","branch":"ewewew","hash":"ewewewewe","widgets":[]}}
{"key":"vs-second","value":{"version":"1.58.0","branch":"ewewew","hash":"ewewew","widgets":[]}}
{"key":"vs-third","value":{"version":"1.42.0","branch":"ewewew","hash":"ewewe","widgets":[]}}
{"key":"vs-fourth","value":{"version":"1.58.0","branch":"eewfwfef","hash":"vvfffsfsf","widgets":[]}}
{"key":"vs-fifth","value":{"version":"1.39.0","branch":"fvrvvsdvds","hash":"vvsvdsvds","widgets":[]}}
Pulling out the different fields then becomes:
$ jq -c '.streams | to_entries[] | [.key, .value.version, .value.branch, .value.hash]' vs.json
["vs-first","2.33.0","ewewew","ewewewewe"]
["vs-second","1.58.0","ewewew","ewewew"]
["vs-third","1.42.0","ewewew","ewewe"]
["vs-fourth","1.58.0","eewfwfef","vvfffsfsf"]
["vs-fifth","1.39.0","fvrvvsdvds","vvsvdsvds"]
Or equivalently, with the repeated .value lookups refactored out:
jq -c '.streams | to_entries[] | [.key, (.value | .version, .branch, .hash)]' vs.json
["vs-first","2.33.0","ewewew","ewewewewe"]
["vs-second","1.58.0","ewewew","ewewew"]
["vs-third","1.42.0","ewewew","ewewe"]
["vs-fourth","1.58.0","eewfwfef","vvfffsfsf"]
["vs-fifth","1.39.0","fvrvvsdvds","vvsvdsvds"]
Add bash processing
jq can't do everything, so if you do want to get the results out to bash to do additional processing—e.g., call curl—you could use -r to print each value on a separate line and use read to read the lines into variables. It would look something like this:
jq -r '.streams | to_entries[] | .key, (.value | .version, .branch, .hash)' vs.json |
while read -r key &&
read -r version &&
read -r branch &&
read -r hash
do
...
done