I have this shell script running in a Jenkins pipeline
def planResults = sh(returnStdout: true, script: "cd $it; PLAN=\$(terragrunt plan --terragrunt-source-update | landscape); echo "$PLAN"; CHANGES=$(echo "$PLAN" | tail -2); echo $CHANGES")
The issue is when I try to echo the "$PLAN" variables.
Here is the solution that groovy recommends, which works near where PLAN is set at \$(terragrunt, however it does not work for a $ inside double quotes. And I NEED double quotes for this command to work properly.
solution: either escape a literal dollar sign "\$5" or bracket the value
expression "${5}" # line 34, column 148.
ce-update | landscape); echo "$PLAN"; CH
Thank you!
For double quoted string, Groovy will do interpolation on the string firstly.
Because the it, PLAN and CHANGES are runtime variables of the shell, rather than the variables of Groovy runtime. Groovy can't find the responding value from Groovy variable stack to replace the $it/PLAN/CHANGS during interpolation.
So you need to escape all $ if you use double quote in your case:
script: "cd \$it; PLAN=\$(terragrunt plan --terragrunt-source-update | landscape);
echo \$PLAN; CHANGES=\$(echo \$PLAN | tail -2); echo \$CHANGES"
Or use single quote which not support interpolation:
script: 'cd $it; PLAN=$(terragrunt plan --terragrunt-source-update | landscape);
echo $PLAN; CHANGES=$(echo $PLAN | tail -2); echo $CHANGES'
More detail about Groovy string at here
Related
I am writing a bash script which has a json value stored in a variable now i want to extract the values in that json using Jq. The code used is.
json_val={"code":"lyz1To6ZTWClDHSiaeXyxg","redirect_to":"http://example.com/client-redirect-uri?code=lyz1To6ZTWClDHSiaeXyxg"}
code_val= echo"$json_val" | jq '.code'
This throws an error of no such file or direcotry.
If i change this to
json_val={"code":"lyz1To6ZTWClDHSiaeXyxg","redirect_to":"http://example.com/client-redirect-uri?code=lyz1To6ZTWClDHSiaeXyxg"}
code_val=echo" $json_val " | jq '.code'
This does not throws any error but the value in code_val is null.
If try to do it manually echo {"code":"lyz1To6ZTWClDHSiaeXyxg","redirect_to":"http://example.com/client-redirect-uri?code=lyz1To6ZTWClDHSiaeXyxg"} | jq '.code' it throws parse numeric letter error.
how can i do it in first case.
You may use this:
json_val='{"code":"lyz1To6ZTWClDHSiaeXyxg","redirect_to":"http://example.com/client-redirect-uri?code=lyz1To6ZTWClDHSiaeXyxg"}'
code_val=$(jq -r '.code' <<< "$json_val")
echo "$code_val"
lyz1To6ZTWClDHSiaeXyxg
Note following changes:
Wrap complete json string in single quotes
use of $(...) for command substitution
Use of <<< (here-string) to avoid a sub-shell creation
PS: If you're getting json text from a curl command and want to store multiple fields in shell variables then use:
read -r code_val redirect_to < <(curl ... | jq -r '.code + "\t" + .redirect_to')
Where ... is your curl command.
If try to do it manually:
$ echo {"code":"lyz1To6ZTWClDHSiaeXyxg","redirect_to":"http://example.com/client-redirect-uri?code=lyz1To6ZTWClDHSiaeXyxg"} | jq '.code'
...it throws parse numeric letter error.
seems like you did not escape the string of the echo command. in your case, escaping with a singe-quote (apostrophe ') will do - same as you did with the jq json-path argument ('.code')
$ echo '{"code":"lyz1To6ZTWClDHSiaeXyxg","redirect_to":"http://example.com/client-redirect-uri?code=lyz1To6ZTWClDHSiaeXyxg"}' | jq '.code'
"lyz1To6ZTWClDHSiaeXyxg"
I have the following json:
{
"feature/EBS_DDS_SC-27428": {
"auth": "http://test123:8080/service.jsp",
"publish": "http://test234:8080/service.jsp",
"general_name": "PG"
},
"feature/EBS_DDS_SC-27428": {
"auth": "http://ab123:8080/service.jsp",
"publish": "http://ab234:8080/service.jsp",
"general_name": "PG1"
}
}
when I do the following I get the expected result
jq --raw-output '."feature/EBS_DDS_SC-27428" | .auth'
But the following is not working,
export branch=feature/EBS_DDS_SC-27428
cat input.json | jq --raw-output '."${branch}" | .auth'
I get the following compilation error:
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${branch} | .auth
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${branch} | .auth
jq: 2 compile errors
Now I have an environmental variable called branch in my Linux machine
In your specific case I think you have a small bash quoting error causing ${branch} to be treated as a constant. I think you want to quote it like this:
'."' "${branch}" '" | .auth'
------ ------------------ -----------
single double quote so single
quote shell expands the quote
constant branch variable constant
Sample Run
$ echo '."'"${branch}"'" | .auth'
."feature/EBS_DDS_SC-27428" | .auth
$ cat input.json | jq --raw-output '."'"${branch}"'" | .auth'
http://ab123:8080/service.jsp
The variable substitution section of the Advanced Bash Scripting Guide is your friend.
I have an environmental variable called branch
Environment variables (as opposed to shell variables) can be dereferenced in jq programs using the env object. In your case, you'd write env.branch.
If you wanted to make the value of a variable accessible as $branch in the shell, it is best to use the --arg command-line option, along the lines of:
jq --arg branch "$branch"
That way, you can reference the value as $branch in the jq program. In the specific case of the Q, you could (for example) write:
jq -r '.[$branch].auth'
I have a directory /user/reports under which many files are there, one of them is :
report.active_user.30092018.77325.csv
I need output as number after date i.e. 77325 from above file name.
I created below command to find a value from file name:
ls /user/reports | awk -F. '/report.active_user.30092018/ {print $(NF-1)}'
Now, I want current date to be passed in above command as variable and get result:
ls /user/reports | awk -F. '/report.active_user.$(date +'%d%m%Y')/ {print $(NF-1)}'
But not getting required output.
Tried bash script:
#!/usr/bin/env bash
_date=`date +%d%m%Y`
active=$(ls /user/reports | awk -F. '/report.active_user.${_date}/ {print $(NF-1)}')
echo $active
But still output is blank.
Please help with proper syntax.
As #cyrus said you must use double quotes in your variable assignment because simple quote are use only for string and not for containing variables.
Bas use case
number=10
string='I m sentence with or wihtout var $number'
echo $string
Correct use case
number=10
string_with_number="I m sentence with var $number"
echo $string_with_number
You can use simple quote but not englobe all the string
number=10
string_with_number='I m sentence with var '$number
echo $string_with_number
Don't parse ls
You don't need awk for this: you can manage with the shell's capabilities
for file in report.active_user."$(date "+%d%m%Y")"*; do
tmp=${file%.*} # remove the extension
number=${tmp##*.} # remove the prefix up to and including the last dot
echo "$number"
done
See https://www.gnu.org/software/bash/manual/bashref.html#Shell-Parameter-Expansion
Given a list of paths separated by a single space:
/home/me/src/test /home/me/src/vendor/a /home/me/src/vendor/b
I want to remove the prefix /home/me/src/ so that the result is:
test vendor/a vendor/b
For a single path I would do: ${PATH#/home/me/src/} but how do I apply it to this series?
You can use // to replace all occurrences of substring. Replace it with null string to remove them.
$ path="/home/me/src/test /home/me/src/vendor/a /home/me/src/vendor/b"
$ echo ${path//\/home\/me\/src\/}
test vendor/a vendor/b
Reference: ${parameter/pattern/string} in Bash reference manual
Using shell parameter expansion doesn't seem to be the solution for this, since it would remove everything up to / from a given point is useful, as nu11p01n73R's answer reveals.
For clarity, I would use sed with the syntax sed 's#pattern#replacement#g':
$ str="/home/me/src/test /home/me/src/vendor/a /home/me/src/vendor/b"
$ sed 's#/home/me/src/##g' <<< "$str"
test vendor/a vendor/b
Like always a grep solution from my side :
echo 'your string' | grep -Po '^/([^ /]*/)+\K.+'
Please note that the above regex do this for any string like /x/y/z/test ... But if you are interested only in replacing /home/me/src/, try the following :
echo 'your string' | grep -Po '^/home/me/src/\K.+' --color
Here we have 2 vars in bash shell
ID="ABC"
ID_STRING="Here is the [${ID}]"
Is there any approach that we could dynamic replace the var ${ID} to "ABC" in the ID_STRING and echo to concole?
If I understand the question correctly the ID_STRING variable contains the literal string ${ID} when it is echoed to the screen and isn't assigned the way you show in the question - assigning it the way you have it in your question using double quotes means the ID_STRING variable never actually contains the literal string ${ID} because with double quotes the variable replacement is done as it is assigned. So first: to get the literal string ${ID} into ID_STRING you need to use single quotes. And second: you need to reevaluate ID_STRING as it is echoed. I do it like this (the $ is the bash prompt):
$ ID=ABC
$ echo $ID
ABC
$ ID_STRING='Here is the [${ID}]'
$ echo $ID_STRING
Here is the [${ID}]
$ eval echo $ID_STRING
Here is the [ABC]
$ echo $ID, $ID_STRING, `eval echo $ID_STRING`
ABC, Here is the [${ID}], Here is the [ABC]
You you want to replace ${ID} from string:
ID="ABC"
sed 's/\${ID}/'$ID'/' filename
Output:
ID_STRING="Here is the "[ABC]"
If you want to use that value of ID while assigning the string to ID_STRING, you can use:
ID="ABC"
ID_STRING="Here is the [$ID]"
Now:
echo $ID_STRING
gives
Here is the [ABC]
sed -i -e 's/old/new/g' file
Is basically what you need, to output the changes have a look here.