I want to run command:
curl -H "X-Auth-Token: $OS_TOKEN" "http://192.168.0.13:8774/v2.1/servers"
And, I made the script for it.
URL="http://192.168.0.13:8774/v2.1/servers"
HEADER="X-Auth-Token: 12345678"
METHOD="GET"
CMD="curl -H $HEADER $URL"
eval "$CMD"
But, as it doesn't include any double quotes in $CMD and separates parameters by space, it runs wrong command
$ bash request.sh
curl: (6) Could not resolve host: 12345678
How can I wrap it?
In command line, we can give double quotes to separate parameters.
But how can I put a variable with spaces to a parameter using scripts as same as command line.
You should change CMD="curl -H $HEADER $URL" in CMD="curl -H \"$HEADER\" \"$URL\"".
I would try:
URL="http://192.168.0.13:8774/v2.1/servers"
HEADER="X-Auth-Token: 12345678"
CMD='curl -H \"${HEADER}\" ${URL}'
eval "${CMD}"
Related
Below my .gitlab-ci.yml file:
image: docker:latest
services:
- docker:dind
stages:
- deploy_dev_env
Deploy DEV Environment:
stage: deploy_dev_env
script:
- curl -v -H "Content-Type: multipart/form-data" -X PUT -F uploadInput=#schema/schema.xml -F overwrite=true -F xmlaEnabledFlag=true -F parameters="DataSource=outputDS" -F parameters="EnableXmla=true" -u $PENTAHO_DEPLOY_USER:$PENTAHO_DEPLOY_PASSWORD http://$PENTAHO_HOST/pentaho/plugin/data-access/api/datasource/analysis/catalog/sca
This very simple script is causing an error because the colons(:) in the "Content-Type: multipart/form-data" piece.
Running the CI Lint in the script I get the following:
Status: syntax is incorrect
jobs:deploy dev environment:script config should be a string or an array containing strings and arrays of strings
If I replace "Content-Type: multipart/form-data" by "Content-Type multipart/form-data" (removed the ":"), I get I correct syntax for my .gitlab-ci.yml file.
Is this a bug or should I re-write my curl command in a different way ?
Docs warn about special characters in yaml script:
Note: Sometimes, script commands will need to be wrapped in single or double quotes. For example, commands that contain a colon (:) need to be wrapped in quotes so that the YAML parser knows to interpret the whole thing as a string rather than a “key: value” pair. Be careful when using special characters: :, {, }, [, ], ,, &, *, #, ?, |, -, <, >, =, !, %, #, `.
One of the option would be to use yaml block scalar
Deploy DEV Environment:
stage: deploy_dev_env
script: >
curl -v -H "Content-Type: multipart/form-data" -X PUT -F uploadInput=#schema/schema.xml -F overwrite=true -F xmlaEnabledFlag=true -F parameters="DataSource=outputDS" -F parameters="EnableXmla=true" -u $PENTAHO_DEPLOY_USER:$PENTAHO_DEPLOY_PASSWORD http://$PENTAHO_HOST/pentaho/plugin/data-access/api/datasource/analysis/catalog/sca
or use some other way to escape colon in yaml.
I faced this issue due to yaml syntax before and the workaround i found was
Store the "Content-Type: application/json" as an environment variable in the CICD settings
for example
Key: HEADER
Value: Content-Type: application/json
and in your script
- curl -v -H "$HEADER" -X PUT -F uploadInput=#schema/schema.xml -F overwrite=true -F xmlaEnabledFlag=true -F parameters="DataSource=outputDS" -F parameters="EnableXmla=true" -u $PENTAHO_DEPLOY_USER:$PENTAHO_DEPLOY_PASSWORD http://$PENTAHO_HOST/pentaho/plugin/data-access/api/datasource/analysis/catalog/sca
After going through several options, I found below simple solution
Use like this,
Content_Type="Content-Type:"
export RESULT="$(echo "$Content_Type" multipart/form-data)"
echo "Result is " $RESULT
Use $RESULT as variable there
Within a bash script, I am trying to append a command string that is single and double quoted to a file (.profile).
I would like to use echo and then >> the command to .profile. Of course, I am open to any solution that works.
The command I would like to use is echo "curl -X POST -H "Content-Type: application/json" -d '{"value1":"PHONENUMBER","value2":"MESSAGE"}' https://maker.ifttt.com/trigger/TRIGGER/with/key/KEY &> /dev/null" >> .profile but clearly this doesn't work within my bash script.
I am not clear on how printf %q works and don't understand how to apply it to my problem.
I have tried this
`CMDSTRING='curl -X POST -H "Content-Type: application/json" -d '`
`CMDSTRING=${CMDSTRING}"'"`
`CMDSTRING=${CMDSTRING}'{"value1":"+PHONENUMBER","value2":"MESSAGE"}'`
`CMDSTRING=${CMDSTRING}"'"`
`CMDSTRING=${CMDSTRING}' https://maker.ifttt.com/trigger/TRIGGER/with/key/KEY &> /dev/null'`
`echo $CMDSTRING`
Using printf '%q' to generate .profile content looks something like the following:
{
printf '%q ' \
curl -X POST -H "Content-Type: application/json" \
-d '{"value1":"PHONENUMBER","value2":"MESSAGE"}' \
https://maker.ifttt.com/trigger/TRIGGER/with/key/KEY
printf '%s\n' "&>/dev/null"
} >> .profile
Note that you cannot use the %q format string if you want &>/dev/null to be parsed as syntax, since by its very nature it formats everything it's passed to be parsed as data.
Thus, we use printf '%q ' "command name" "first argument" ... for the actual command itself, and format the redirection out-of-band.
That said, note that there's value to the above only if you're substituting variables from an untrusted source (rather than hardcoding them as in the example), and are worried about invalid values being abused for command injection. If you're truly just appending a constant string to the end of a file, a quoted heredoc will let you build more natural-looking shell quoting manually (indeed, as you've already done!), and pass it through verbatim:
cat >>.profile <<'EOF'
curl -X POST -H "Content-Type: application/json" \
-d '{"value1":"PHONENUMBER","value2":"MESSAGE"}' \
https://maker.ifttt.com/trigger/TRIGGER/with/key/KEY &> /dev/null
EOF
Here, everything between the <<'EOF' and the EOF are passed through exactly-as-given, including quotes and parameter expansions the shell might otherwise try to interpret.
I am trying to upload a JSON file into my noSQL database using a bash script, but it doesn't work and I don't understand why.
This is the script :
test='{"evaluation": "none"}'
test="'$test'"
command="curl -XPUT localhost:9200/test/evaluation/$i -d $test"
echo "$command"
$command
This is the error :
curl -XPUT localhost:9200/test/evaluation/0 -d '{"evaluation": "none"}'
{"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406}curl: (3) [globbing] unmatched close brace/bracket in column 7
When I do the command given in my command line it works fine though.
What is the error here ? Thank you
Don't store a command in a variable; if you absolutely must have something usable with logging, put the arguments in an array.
test='{"evaluation": "none"}'
args=( -XPUT localhost9200/test/evaluation/"$i" -d "$test" )
echo "curl ${args[*]}"
curl "${args[#]}"
My question is very simple. I want to use environment variables in a cURL command sth similar to this:
curl -k -X POST -H 'Content-Type: application/json' -d '{"username":"$USERNAME","password":"$PASSWORD"}'
When I run the command $USERNAME is passed to the command as a "$USERNAME" string not the value of the variable. Is there a way to escape this situation?
Thanks.
Single quotes inhibit variable substitution, so use double quotes. The inner double quotes must then be escaped.
... -d "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD\"}"
Since this answer was written in 2015, it has become clear that this technique is insufficient to properly create JSON:
$ USERNAME=person1
$ PASSWORD="some \"gnarly 'password"
$ echo "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD\"}"
{"username":"person1","password":"some "gnarly 'password"}
$ echo "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD\"}" | jq .
parse error: Invalid numeric literal at line 1, column 47
The quoting problem are clear. The (shell) solutions are not
Current best practice: use a JSON-specific tool to create JSON:
jq
$ jq -n -c --arg username "$USERNAME" --arg password "$PASSWORD" '$ARGS.named'
{"username":"person1","password":"some \"gnarly 'password"}
jo
$ jo "username=$USERNAME" "password=$PASSWORD"
{"username":"person1","password":"some \"gnarly 'password"}
And with curl:
json=$( jq -n -c --arg username "$USERNAME" --arg password "$PASSWORD" '$ARGS.named' )
# or
json=$( jo "username=$USERNAME" "password=$PASSWORD" )
# then
curl ... -d "$json"
For less quoting, read from standard input instead.
curl -k -X POST -H 'Content-Type: application/json' -d #- <<EOF
{ "username": "$USERNAME", "password": "$PASSWORD"}
EOF
-d #foo reads from a file named foo. If you use - as the file name, it reads from standard input. Here, standard input is supplied from a here document, which is treated as a double-quoted string without actually enclosing it in double quotes.
curl -k -X POST -H 'Content-Type: application/json' -d '{"username":"'$USERNAME'","password":"'$PASSWORD'"}'
Here the variable are placed outside of "'" quotes and will be expanded by shell (just like in echo $USERNAME). For example assuming that USRNAME=xxx and PASSWORD=yyy the argv[7] string passed to curl is {"username":"xxx","password":"yyy"}
And yes, this will not work when $USERNAME or $PASSWORD contain space characters.
Our: curl -k -X POST -H 'Content-Type: application/json' -d '{"username":"'"$USERNAME"'","password":"'"$PASSWORD"'"}'
You can wrap the environment variables with "'" and you should keep the single quote for the external json object.
e.g
-d '{"username":"'"$USERNAME"'","password":"'"$PASSWORD"'"}'
I use the following line to create database:
curl -X POST 'http://10.1.1.1:8086/db?u=root&p=root' -d '{name: test1}
if i try to do it from shell script:
ip=10.1.1.1
curl -X POST 'http://$ip:8086/db?u=root&p=root' -d '{name: test1}'
i have a problem with shell variable substitution within single quotas, if i try to use them within double quotas:
curl -X POST "http://$ip:8086/db?u=root&p=root" -d '{name: test1}'
variable is expanded to the right value, printing in terminal
curl -X POST "http://10.1.21.1:8086/db?u=root&p=root" -d '{name: test1}': **No such file or directory**
What would be the right solution to this problem?
Try this:
ip=10.1.1.1
curl -X POST 'http://'"$ip"':8086/db?u=root&p=root' -d '{name: test1}'