How to chain curl such that the output of one become another's input? - linux

Could someone tell me how to pass the output of one CURL GET to another CURL POST? I mean something like this:
curl --header "Content-Type: application/json" --request POST --data "{\"specification\": curl --header "Content-Type: application/json" --request GET "https://user:pass#anotherhost"}" "https://user:pass#localhost"

Use jq to craft JSON in the shell.
response="$(curl --header "Content-Type: application/json" --request GET "https://user:pass#anotherhost")"
data="$(jq "{specification: .}" <<< "$response")"
curl --header "Content-Type: application/json" --request POST --data "$data" "https://user:pass#localhost"
Keep in mind, that passwords in command line arguments are public to the host.

It looks like you need to use command substitution. The result would be something like this.
curl --header "Content-Type: application/json" --request POST --data "{\"specification\": $(curl --header "Content-Type: application/json" --request GET "https://user:pass#anotherhost")}" "https://user:pass#localhost"
$() is used for command substitution and it invokes a subshell. The command in the parentheses (a.k.a. round brackets) of $() is executed in a subshell and the output is then placed in the original command.
So the curl GET will be executed in a subshell and the result will be passed to the curl POST
You can read more about it at the link below:
https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html

Related

Using Curl need to call a REST API

I am trying to generate a file using REST API by Header authorization "Bearer Token" and password encryption.
token=$(curl --header "Content-Type: application/json" --header "aesiv: xxxxxxxx" \
--request POST \
--data '{"login":"username","password":"encrypted_pswd"}' \
https://ssssss/api/login)
curl -X GET https://ssssss/api/generate-report -H "Accept: encryptionIntVeD" "Authorization: aesiv {xxxxxxxxxxx}" -H "Accept: application/json" -H "Authorization: Bearer {token}"
File not getting generated, can anyone guide me where I am stuck.

Passing a long JSON text to curl POST

I am referring to the thread: Passing a large text to bash curl url
I have a shell script that looks like:
#!bin/bash
curl --location --request POST 'example.com/abcxyz' \
--header 'Content-Type: application/json' \
--header 'Cookie: my_secret_key' \
--data-raw '{_my_very_long_json_content_here}'
Approach 1: I put everything after the curl, i.e. from --location ... to the end to a test.txt file then
tempfile = test.txt
curl "#$tempfile"
and I get the error curl: (6) Could not resolve host: test.txt
Approach 2: I put everything in a test.sh file:
#!bin/bash
curl --location --request POST 'example.com/abcxyz' --header 'Content-Type: application/json' --header 'Cookie: my_secret_key' --data-raw '#-' <<< 'my_very_long_json'
and I have an error {"error":"invalid character '#' looking for beginning of value","code":3}(base)
What should I do to pass a very long JSON content to curl POST?
Note that, if the JSON content is not too long, I can put everything in a SH file then run
bash test.sh
and it is okay.
--data-raw specifically disables # interpretation. Use --data #- or --data-binary #- to read from stdin.
curl --location --request POST 'example.com/abcxyz' \
--header 'Content-Type: application/json' \
--header 'Cookie: my_secret_key' \
--data #- <<< '{_my_very_long_json_content_here}'
You might find a heredoc more ergonomic than a long string:
curl --location --request POST 'example.com/abcxyz' \
--header 'Content-Type: application/json' \
--header 'Cookie: my_secret_key' \
--data #- <<'DATA'
{
'json': 'here'
}
DATA

How to SSH a curl command

env = GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
Situation:
SystemA=No internet. SystemB=Yes internet.
SystemA has a log file. SystemA wants SystemB to send a curl command for him.
SystemA$
ssh SystemB curl -X POST -H "Content-type: application/json" -d "$data" $hook
= fail
SystemB$
curl -X POST -H "Content-type: application/json" -d "$data" $hook = success
How do I achieve this without SystemA 'scp'ing the log file to SystemB?
It's heavily schedule related so I want SystemA let SystemB work.
EDIT:
I narrowed down the problem :
On SystemB:
curl -X POST -H "Content-type: application/json" -d '{$data}' $hookurl = success
curl -X POST -H "Content-type: application/json" -d {$data} $hookurl = fail
So when I type in SystemA
ssh SystemB curl -X POST -H "Content-type: application/json" -d "{$data}" $hookurl
It actually runs with -d {$data} on SystemB. How can I fix this?
Update:
ssh SystemB curl -X POST -H "Content-type: application/json" -d "'{$data}'" $hookurl
did work and actually sent data to url,
but curl: (6) Could not resolve host: application; Unknown error occurred again.
You can use this command :
ssh SystemB /bin/bash <<< "$(declare -p data hook);"'curl -X POST -H "Content-type: application/json" -d "$data" "$hook"'
"$(declare -p data hook);" takes variable definitions from SystemA and passes them to SystemB
Use sshfs perhaps
on systemB
sshfs -p'ssh port' -o password_stdin user#systemA:/home/user/dir /home/user/dir <<< 'passwd'

Gitlab API, Merge When Build Succeeds

I'd like to set the merge_when_build_succeeds attribute of a merge request with the Gitlab API. The docs says, that when the optional merged_when_build_succeeds parameter is true, the MR'll be accepted only after the build succeeds.
How should I provide this merged_when_build_succeeds parameter to the API? I tried the following curl commands without any success:
# In the request's body
vilmosnagy#vnagy-dell:~$ curl -X PUT --header "PRIVATE-TOKEN: zvzK7CNzx9WviV5iChyg" -d merged_when_build_succeeds=true "http://localhost:8080/api/v3/projects/1/merge_requests/9/merge"
{"message":"405 Method Not Allowed"}
# In the URL
vilmosnagy#vnagy-dell:~$ curl -X PUT --header "PRIVATE-TOKEN: zvzK7CNzx9WviV5iChyg" "http://localhost:8080/api/v3/projects/1/merge_requests/9/merge?merged_when_build_succeeds=true"
{"message":"405 Method Not Allowed"}
# I tried this also, maybe the __merged__ is a mistake in the docs
vilmosnagy#vnagy-dell:~$ curl -X PUT --header "PRIVATE-TOKEN: zvzK7CNzx9WviV5iChyg" -d merge_when_build_succeeds=true "http://localhost:8080/api/v3/projects/1/merge_requests/9/merge"
{"message":"405 Method Not Allowed"}
# In the URL
vilmosnagy#vnagy-dell:~$ curl -X PUT --header "PRIVATE-TOKEN: zvzK7CNzx9WviV5iChyg" "http://localhost:8080/api/v3/projects/1/merge_requests/9/merge?merge_when_build_succeeds=true"
{"message":"405 Method Not Allowed"}
# Some other CURL parameters
vilmosnagy#vnagy-dell:~$ curl -X PUT --header "PRIVATE-TOKEN: zvzK7CNzx9WviV5iChyg" -H "Content-Type: multipart/form-data;" -F "merge_when_build_succeeds=true" "http://localhost:8080/api/v3/projects/1/merge_requests/9/merge"
{"message":"405 Method Not Allowed"}
# Some other CURL parameters
vilmosnagy#vnagy-dell:~$ curl -X PUT --header "PRIVATE-TOKEN: zvzK7CNzx9WviV5iChyg" -H "Content-Type: multipart/form-data;" -F "merged_when_build_succeeds=true" "http://localhost:8080/api/v3/projects/1/merge_requests/9/merge"
{"message":"405 Method Not Allowed"}
The new feature, 'Only allow merge requests to be merged if the build succeeds' is turned on. When I turn off this feature any call of the previous calls merge the MR before the build finished, see: https://snag.gy/06nOEm.jpg
The private token, and the URL is right. I can query the MR's info:
vilmosnagy#vnagy-dell:~$ curl -X GET --header "PRIVATE-TOKEN: zvzK7CNzx9WviV5iChyg" "http://localhost:8080/api/v3/projects/1/merge_requests/9"
{"id":9,"iid":9,"project_id":1,"title":"enters","description":"","state":"opened","created_at":"2016-06-29T15:36:15.235Z","updated_at":"2016-06-29T15:36:15.235Z","target_branch":"master","source_branch":"features/long_build_05","upvotes":0,"downvotes":0,"author":{"name":"Vilmos Nagy","username":"vilmos.nagy","id":2,"state":"active","avatar_url":"http://www.gravatar.com/avatar/4f94d9571ec83f42a85651291296f503?s=80\u0026d=identicon","web_url":"http://172.21.0.3/u/vilmos.nagy"},"assignee":null,"source_project_id":2,"target_project_id":1,"labels":[],"work_in_progress":false,"milestone":null,"merge_when_build_succeeds":false,"merge_status":"can_be_merged","subscribed":false,"user_notes_count":0}
And I can accept the MR after the build succeeds:
vilmosnagy#vnagy-dell:~$ curl -X PUT --header "PRIVATE-TOKEN: zvzK7CNzx9WviV5iChyg" "http://localhost:8080/api/v3/projects/1/merge_requests/9/merge"
{"id":9,"iid":9,"project_id":1,"title":"enters","description":"","state":"merged","created_at":"2016-06-29T15:36:15.235Z","updated_at":"2016-06-29T16:13:41.242Z","target_branch":"master","source_branch":"features/long_build_05","upvotes":0,"downvotes":0,"author":{"name":"Vilmos Nagy","username":"vilmos.nagy","id":2,"state":"active","avatar_url":"http://www.gravatar.com/avatar/4f94d9571ec83f42a85651291296f503?s=80\u0026d=identicon","web_url":"http://172.21.0.3/u/vilmos.nagy"},"assignee":null,"source_project_id":2,"target_project_id":1,"labels":[],"work_in_progress":false,"milestone":null,"merge_when_build_succeeds":false,"merge_status":"can_be_merged","subscribed":true,"user_notes_count":0}
Could you help me, and give me the correct call of the given API?
Thanks!
Vilmos
It seems the documentation is wrong, the parameter is actually called merge_when_build_succeeds (without the "d" in "merge").
curl -X PUT --header "PRIVATE-TOKEN: zvzK7CNzx9WviV5iChyg" -d merge_when_build_succeeds=true "http://localhost:8080/api/v3/projects/1/merge_requests/9/merge" should work ("405 Method Not Allowed" was probably because of something else, like the MR was already merged or had conflicts).
I created an issue you can follow if you want to know when the documentation is fixed: https://gitlab.com/gitlab-org/gitlab-ce/issues/19448

Using a variable in a script calling curl

I'm trying to replace 1.2.3.4 with the contents of variable $wanip in the following script.
wanip="4.3.2.1"
echo $wanip
content=$(curl --insecure -H "X-DNSimple-Token: foo:bar" -H "Accept: application/json" -H "Content-Type: application/json" -X PUT -d "{\"record\": {\"name\": \"foo\",\"content\": \"1.2.3.4\"}}" https://acme.com/records/123)
echo $content
If I literally replace 1.2.3.4 with $wanip*, when I run the script I'm getting a message saying: "message":"Problems parsing JSON".
Try adding a layer of abstraction:
#!/bin/bash
wnip="4.3.2.1"
echo $wanip
command="curl --insecure -H 'X-DNSimple-Token: foo:bar' -H 'Accept: application/json' -H 'Content-Type: application/json' -X PUT -d '{\"record\": {\"name\": \"foo\",\"content\": \"${wnip}\"}}' https://acme.com/records/123"
echo $command
content=$($command)
echo $content
After some hacking, I got this to work. Strange.
wanip=\"4.3.2.1\"
echo $wanip
content=$(curl --insecure -H "X-DNSimple-Token: foo:bar" -H "Accept: application/json" -H "Content-Type: application/json" -X PUT -d "{\"record\": {\"name\": \"foo\",\"content\": $wanip }}" https://acme.com/records/123)
echo $content

Resources