This question already has answers here:
Parsing JSON with Unix tools
(45 answers)
Closed 24 days ago.
I have json string like this
{"state":{"stateId":0,"nextPollingTime":null,"updateState":null},"config":{"privateIPs":null,"64Bit":false,"silent":false,"iid":14,"selfp":null,"sevlfp":null,"av":14,"aid":null,"aty":2,"sev":0,"seci":0,"sti":false,"scto":60000,"sci":5000,"stkd":5000,"sud":5000,"mpfa":3}}
I need to get "state" and "config" keys content to a shell custom variable or to a file.
tried with a command:
$RESPONSE is the API response, the above json string
echo $RESPONSE | sed 's/{"$state":"*\([0-9a-zA-Z]*\)"*,*.*}/\1/'
this prints nothing but suppose to get this output:
{"stateId":0,"nextPollingTime":null,"updateState":null}
Tried with saving the response to tmp file and executed this command
cat /tmp/a.json | grep -o -e "{"state":.*}"
this also print empty string but expected result:
{"stateId":0,"nextPollingTime":null,"updateState":null}
Am new to shell script and trying with various options available in the internet, please help me to write the command for the same.
This uses any sed:
$ sed 's/.*"state":\({[^}]*}\).*/\1/' file
{"stateId":0,"nextPollingTime":null,"updateState":null}
That will work for your posted input but fail given other json input (e.g. with "state" as a value or with } inside a value), just like any other solution that uses mandatory POSIX tools as none of them have a JSON parser (unless you write one with awk, but I expect that's more work than you or anyone here would put into this).
Related
This question already has answers here:
How to substitute shell variables in complex text files
(12 answers)
Closed 3 years ago.
Trying to variable replace a templated yaml file.
I'm using eval to take the environment shell variables and replace whats in the file dynamically. I can't figure out how to take the output of this and save to a file.
I just want to take the evaluated output and save to a file.
eval "cat <<EOF
$(<${baseFileName})
EOF"
Exmaple test.yaml
---
value: ${PORT}
Bash environment variable:
PORT=8888
output temp.test.yaml
---
value: 8888
Right now the code will just print the evaluated text to the console.
I've tried.
eval "cat <<EOF
$(<${baseFileName})
EOF" > $newBaseFileName
but no joy. Didn't even create the file.
The reason I'm not using sed is because the file could have unlimited variable decelerations, and I want to replace any value matching a defined bash variable or environment variable. This is part of a template engine. For the life of me I can't remember how I did it before with pure bash.
It didn't work for me but what I did is this
renderTemplate() {
eval "cat <<EOF
$(<${1})
EOF"
}
baseFileName=$(basename $fileName)
templateOutput=`renderTemplate ${baseFileName}`
echo "${templateOutput}"
I'm using this as a temp file anyways so what I'll do is save to variable and then pump that variable in to the command to apply the template as a file. That way it's only ever stored in memory. This is a middleware cli to another cli to add variable replacement to in-memory web hosted files before applying them.
Thanks for your help.
This question already has answers here:
Parsing JSON with Unix tools
(45 answers)
Closed 4 years ago.
In my shell script I am making an API call using cURL which is stored in variable named token_response, further I need to parse out value of key aws_access_key_id for which I am using sed as shown below. Wondering what's wrong with the pattern in sed which is not able to parse out desired key's value.
When trying to fetch using -
echo aws_access_key_id is:$(echo $token_response | sed -n 's/^.*"AccessKeyId":"\([^"]*\)",*$/\1/p')
JSON is stored in variable called - token_response
{
"Code":"Success",
"LastUpdated":"2018-12-27T07:16:31Z",
"Type":"fakedTypeValue",
"AccessKeyId":"fakedAccessKeyIdValue",
"Token":"fakedTokenValue"
}
Following is printed on console without the token value (Seems value for AccessKeyId is not parsed) -
aws_access_key_id is:
I expected following -
aws_access_key_id is:fakedAccessKeyIdValue
In the absence of jq, you may use this gnu grep:
grep -oP '"AccessKeyId"\h*:\h*"\K[^"]+' <<< "$token_response"
fakedAccessKeyIdValue
Or even this when token value appears after line breaks:
grep -zoP '"AccessKeyId"\s*:\s*"\K[^"]+' <<< "$token_response"
This question already has answers here:
File content into unix variable with newlines
(6 answers)
Closed 4 years ago.
In the bash shell, I'm trying to read the json file and load to a variable
eri#xyz:~/Documents/inbound>e1=$(eval echo $(cat ./deploy/request.json))
Upon fetching the output of that variable, I'm seeing -bash - command not found along with the actual contents of the .json file
eri#xyz:~/Documents/inbound>"$e1"
-bash: { type:Pipeline, category:Software, risk:4, short_description:sample short description text, description:sample detailed description text, assignment_group: Services - Retail Services, cmdb_ci:Retail Service, u_version:1.0, start_date:2017-01-04 18:00:00, end_date:2017-01-04 19:00:00, backout_plan:see department for standard backout plan, implementation_plan:sample implementation plan, test_plan:sample text plan, production_system:false }: command not found
Is there a way to suppress the -bash - command not found in the output?.
No need for eval - just e1=$(< ./deploy/request.json) should do the trick. (Thanks to #shellter for the syntax — you don't even need to use cat!)
To show the variable, you want
echo "$e1"
instead of just "$e1". "$e1" by itself on the command line does not print out the value of $e1, unlike many programming-language REPLs. Instead, it tells bash to try to interpret the entire contents of $e1 as the name of a command. It isn't the name of a command, so bash tells you a command by that name cannot be found.
I have a Jenkins console output that looks like this:
Started by remote host 10.16.17.13
Building remotely on ep9infrajen201 (ep9) in workspace d:\Jenkins\workspace\Tools\Provision
[AWS-NetProvision] $ powershell.exe -NonInteractive -ExecutionPolicy ByPass "& 'C:\Users\user\AppData\Local\Temp\jenkins12345.ps1'"
Request network range: 10.1.0.0/13
{
"networks": [
"10.1.0.0/24"
]
}
Finished: SUCCESS
I get this from a curl command that I run. to check the JENKINS_JOB_URL/lastBuild/consoleText
My question is, for the sake of some other automation I am doing, how do I get just "10.1.0.0/24" so I can assign it to a shell variable using LINUX tools?
Thank you
Since you listed jq among the tags of your duplicate question, I'll assume you have jq installed. You have to clean up your output to get JSON first, then get to the part of JSON you need. awk does the former, jq the latter.
.... | awk '/^{$/{p=1}{if(p){print}}/^}$/{p=0}' | jq -r .networks[0]
The AWK script looks for { on its own on a line to turn on a flag p; prints the current line if the flag is set; and switches off the flag when it encounters } all by itself.
EDIT: Since this output was generated on a DOS machine, it has DOS line endings (\r\n). To convert those before awk, additionally pipe through dos2unix.
This question already has answers here:
How to concatenate string variables in Bash
(30 answers)
Closed 9 years ago.
I am trying to download files from a database using wget and url. E.g.
wget "http://www.rcsb.org/pdb/files/1BXS.pdb"
So format of the url is as such: http://www.rcsb.org/pdb/files/($idnumber).pdb"
But I have many files to download; so I wrote a bash script that reads id_numbers from a text file, forms url string and downloads by wget.
!/bin/bash
while read line
do
url="http://www.rcsb.org/pdb/files/$line.pdb"
echo -e $url
wget $url
done < id_numbers.txt
However, url string is formed as
.pdb://www.rcsb.org/pdb/files/4H80
So, .pdb is repleced with http. I cannot figure out why. Does anyone have an idea?
How can I format it so url is
"http://www.rcsb.org/pdb/files/($idnumber).pdb"
?
Thanks a lot.
Note. This question was marked as duplicate of 'How to concatenate strings in bash?' but I was actually asking for something else. I read that question before asking this one and it turns out my problem was with preparing the txt file in Windows not really string concetanation. I edited question title. I hope it is more clear now.
It sounds like your id_numbers.txt file has DOS/Windows-style line endings (carriage return followed by linefeed characters) instead of plain unix line endings (just linefeed). The result is that read thinks the line ends with a carriage return, $line actually has a carriage return at the end, and that gets embedded in the url, causing various confusion.
There are several ways to solve this. You could have bash trim the carriage return from the variable when you use it:
url="http://www.rcsb.org/pdb/files/${line%$'\r'}.pdb"
Or you could have read trim it by telling it that carriage return counts as whitespace (read will trim leading and trailing whitespace from what it reads):
while IFS=$'\r' read line
Or you could use a command like dos2unix (or whatever the equivalent is on your OS) to convert the id_numbers.txt file.
The -e echo option is used to output the desired content without inserting a new line, you do not need it here.
Also I suspect your file containing the ids to be malformed, on which OS did you create it?
Anyway, you can simplify your script this way:
!/bin/bash
while read line
do
wget "http://www.rcsb.org/pdb/files/$line.pdb"
done < id_numbers.txt
I was able to successfully test it with an id_numbers.txt file generated like so:
for i in $(0 9) ; do echo "$i" >> id_numbers.txt ; done
Try this:
url="http://www.rcsb.org/pdb/files/"$line
$url=$url".pdb"
For more info, check How to concatenate string variables in Bash?