I need to call a REST API using a curl command inside a gitlab-ci script.
There are some variables that need to be replaced in the curl command, but if I use same variable, looks like it is not replaced by its value.
Below a piece of my gitlab-ci that works fine without any variable:
create_changelog:
stage: create_changelog
variables:
REDMINE_API_KEY: "cbf456d6c74ae85bfbcddfd130c75af4f0977448"
script:
- |
curl -X PUT \
"https://redmine.example.com/projects/flutter-2/wiki/abc1.xml" \
-H "cache-control: no-cache" \
-H "content-type: application/xml" \
-H "x-redmine-api-key: cbf456d6c74ae85bfbcddfd130c75af4f0977448" \
-d "<?xml version=\"1.0\"?><wiki_page><text>Example</text><parent title=\"Release\"/></wiki_page>"
But in case a replace, for example, the apk_key by a variable, like below, I dont get the wiki page created on the server.
create_changelog:
stage: create_changelog
variables:
REDMINE_API_KEY: "cbf456d6c74ae85bfbcddfd130c75af4f0977448"
script:
- |
curl -X PUT \
"https://redmine.example.com/projects/flutter-2/wiki/abc1.xml" \
-H "cache-control: no-cache" \
-H "content-type: application/xml" \
-H "x-redmine-api-key: ${REDMINE_API_KEY}" \
-d "<?xml version=\"1.0\"?><wiki_page><text>Example</text><parent title=\"Release\"/></wiki_page>"
How can I use variable in the curl command above ?
Related
I am referring to the thread: Passing a large text to bash curl url
I have a shell script that looks like:
#!bin/bash
curl --location --request POST 'example.com/abcxyz' \
--header 'Content-Type: application/json' \
--header 'Cookie: my_secret_key' \
--data-raw '{_my_very_long_json_content_here}'
Approach 1: I put everything after the curl, i.e. from --location ... to the end to a test.txt file then
tempfile = test.txt
curl "#$tempfile"
and I get the error curl: (6) Could not resolve host: test.txt
Approach 2: I put everything in a test.sh file:
#!bin/bash
curl --location --request POST 'example.com/abcxyz' --header 'Content-Type: application/json' --header 'Cookie: my_secret_key' --data-raw '#-' <<< 'my_very_long_json'
and I have an error {"error":"invalid character '#' looking for beginning of value","code":3}(base)
What should I do to pass a very long JSON content to curl POST?
Note that, if the JSON content is not too long, I can put everything in a SH file then run
bash test.sh
and it is okay.
--data-raw specifically disables # interpretation. Use --data #- or --data-binary #- to read from stdin.
curl --location --request POST 'example.com/abcxyz' \
--header 'Content-Type: application/json' \
--header 'Cookie: my_secret_key' \
--data #- <<< '{_my_very_long_json_content_here}'
You might find a heredoc more ergonomic than a long string:
curl --location --request POST 'example.com/abcxyz' \
--header 'Content-Type: application/json' \
--header 'Cookie: my_secret_key' \
--data #- <<'DATA'
{
'json': 'here'
}
DATA
I have this script in my gitlab yml file:
script:
- "curl -X POST -H 'Content-type: application/json' --data '{\"tag\":\"${CI_COMMIT_TAG}\", \"projectId\":\"${PROJECT_ID}\"}' http://localhost:1337/slack"**
Let's say when I create a tag named 'testing-v1' on gitlab, the variables $CI_COMMIT_TAG should be 'testing-v1', but instead it is showing as it is.
It's not replacing.
If I hard code the tag name and project id like this
- "curl -X POST -H 'Content-type: application/json' --data '{\"tag\":\"testing-v1\", \"projectId\":\"1111111\"}' http://localhost:1337/slack"**
It's working.
If you copy-paste your curl line (without the first and last double-quote) in shellcheck.net, you will be pointed to SC2016
Expressions don't expand in single quotes, use double quotes for that.
Single quotes prevent expansion of everything, including variables and command substitution.
If you want to use the values of variables and such, use double quotes instead.
Note that if you have other items that needs single quoting, you can use both in a single word:
echo '$1 USD is '"$rate GBP"
See an example of a working gitlab-ci.yml with curl here:
image: ruby:2.3
stages:
- clear-cache
CLOUDFLARE_URL: https://api.cloudflare.com/client/v4/zones/$zone/purge_cache
EMAIL: "X-Auth-Email: $email"
AUTH: "X-Auth-Key: $key"
CONTENT: 'Content-Type: application/json'
clear-cache:
stage: clear-cache
script:
- curl -X POST "${CLOUDFLARE_URL}" -H "${EMAIL}" -H "${AUTH}" -H "${CONTENT}" --data '{"purge_everything":true}'
only:
- master
curl -X POST https://api.commerce.coinbase.com/charges/ \
-H "Content-Type: application/json" \
-H "X-CC-Api-Key: YOUR_API_KEY" \
-H "X-CC-Version: 2018-03-22" \
-d "#data.json"
how do i make this api work?
-H attaches a header to your request. See manpage of curl.
The -H is adding a Header to the request so everything between the speech marks after the -H is a header.
Your CURL request has 3 headers.
See https://developer.mozilla.org/en-US/docs/Glossary/Request_header
I have the following code:
curl -s --insecure -H "Content-Type: application/json" -X POST -d "{\"username\":\"$1\",\"password\":\"$2\"}" http://apiurl
In the above curl command I want to escape the " and ! in the password. Right now the password is just given as $2
I have modified the curl command as below but it doesn't seem to work
curl -s --insecure -H "Content-Type: application/json" -X POST -d "{\"username\":\"$1\",\"password\":\"$2\"}" http://apiurl
Do not try to generate JSON manually; use a program like jq, which knows how to escape things properly, to generate it for you.
json=$(jq -n --arg u "$1" --arg p "$2" '{username: $u, password: $p}')
curl -s --insecure -H "Content-Type: application/json" -X POST -d "$json" http://apiurl
I'm trying to replace 1.2.3.4 with the contents of variable $wanip in the following script.
wanip="4.3.2.1"
echo $wanip
content=$(curl --insecure -H "X-DNSimple-Token: foo:bar" -H "Accept: application/json" -H "Content-Type: application/json" -X PUT -d "{\"record\": {\"name\": \"foo\",\"content\": \"1.2.3.4\"}}" https://acme.com/records/123)
echo $content
If I literally replace 1.2.3.4 with $wanip*, when I run the script I'm getting a message saying: "message":"Problems parsing JSON".
Try adding a layer of abstraction:
#!/bin/bash
wnip="4.3.2.1"
echo $wanip
command="curl --insecure -H 'X-DNSimple-Token: foo:bar' -H 'Accept: application/json' -H 'Content-Type: application/json' -X PUT -d '{\"record\": {\"name\": \"foo\",\"content\": \"${wnip}\"}}' https://acme.com/records/123"
echo $command
content=$($command)
echo $content
After some hacking, I got this to work. Strange.
wanip=\"4.3.2.1\"
echo $wanip
content=$(curl --insecure -H "X-DNSimple-Token: foo:bar" -H "Accept: application/json" -H "Content-Type: application/json" -X PUT -d "{\"record\": {\"name\": \"foo\",\"content\": $wanip }}" https://acme.com/records/123)
echo $content