I am doing this curl command successfully in bash console to retrieve data from a rest resource :
curl -H "Accept: application/json" -H "Content-Type: application/json" -H "X-content: test_content" -H "X-Public: public_key" -H "X-Hash: $(printf "testcompany1" | openssl sha256 -hmac "secret_key" | sed "s/^.* //" | tr -d "\n")" -X GET http://192.168.100.20/rest/v01/customer/csv > /home/user/customer.csv
What I want to do is to use this in a windows cmd shell. Cygwin and curl is installed. So I have tracked it down to this puzzling me, the hmac hashing, done as a nested command in my script using $(command) :
$(printf "testcompany1" | openssl sha256 -hmac "secret_key" | sed "s/^.* //" | tr -d "\n")
How do I get a windows shell cmd recognize this ? Or is there another smarter approach in windows to get rest data with hmac auth ?
I split the bash oneliner up i chunks in a batch file like this :
echo off
set content=content
set secret=secret_key
printf %content% | openssl sha256 -hmac %secret% | sed "s/^.* //" | tr -d "\n" > hash.txt
set /p hash=< hash.txt
curl -H "Accept: application/json" -H "Content-Type: application/json" -H "X-content: %content%" -H "X-Public: public_key -H "X-Hash: %hash%" -X GET http://192.168.100.20/rest/v01/customer/csv > out.csv
The printf command was very picky regarding generating the correct hash value, so I send it to a file first, and back again.
Related
env = GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
Situation:
SystemA=No internet. SystemB=Yes internet.
SystemA has a log file. SystemA wants SystemB to send a curl command for him.
SystemA$
ssh SystemB curl -X POST -H "Content-type: application/json" -d "$data" $hook
= fail
SystemB$
curl -X POST -H "Content-type: application/json" -d "$data" $hook = success
How do I achieve this without SystemA 'scp'ing the log file to SystemB?
It's heavily schedule related so I want SystemA let SystemB work.
EDIT:
I narrowed down the problem :
On SystemB:
curl -X POST -H "Content-type: application/json" -d '{$data}' $hookurl = success
curl -X POST -H "Content-type: application/json" -d {$data} $hookurl = fail
So when I type in SystemA
ssh SystemB curl -X POST -H "Content-type: application/json" -d "{$data}" $hookurl
It actually runs with -d {$data} on SystemB. How can I fix this?
Update:
ssh SystemB curl -X POST -H "Content-type: application/json" -d "'{$data}'" $hookurl
did work and actually sent data to url,
but curl: (6) Could not resolve host: application; Unknown error occurred again.
You can use this command :
ssh SystemB /bin/bash <<< "$(declare -p data hook);"'curl -X POST -H "Content-type: application/json" -d "$data" "$hook"'
"$(declare -p data hook);" takes variable definitions from SystemA and passes them to SystemB
Use sshfs perhaps
on systemB
sshfs -p'ssh port' -o password_stdin user#systemA:/home/user/dir /home/user/dir <<< 'passwd'
I had found these steps for use with a different API. They do not appear to work.
My understanding of nonce is that it is just a random hash you use once. so the creation method should matter little.
I think my issue is with my cURL formatting.
session:
# echo -n "/0" > tmp.bin
# echo -n "123nonce=123" | openssl sha256 -binary >> tmp.bin
# cat tmp.bin | openssl sha512 -binary -hmac $(echo -n "mmyykkeeyy" | base64 -d) | base64 -w long_hash
# curl -X GET -H "Accept: application/json" -H "long_hash" -d "nonce=123" "https://bittrex.com/Api/v2.0/key/market/GetOrderHistory?market=BTC-TRX&apikey=mmyyykkeeyy"
return data:
{"success":false,"message":"NONCE_NOT_PROVIDED","result":null}
also tried:
echo -n "/0/key" > tmp.bin
and I also tried creating nonce without the tmp.bin, outputing it to the buffer and using it raw. same nonce error.
adding
-H "Content-Type: application/json"
yields a server 500 error.
known working api v2.0 library, only python and does not suit my needs v2.0 python library
How do I correctly deliver the nonce to the bittrex api using cURL?
UPDATE:
I changed the url in the Bittrex python library to webserver not running https and captured the dead outbound packet in the raw (non encapsulated). Now I can at least see what the know good python library is throwing
GET packet from known good Python library request in wireshark
$ echo -n "/0" > tmp.bin
$ echo -n "123nonce=123" | openssl sha256 -hex
(stdin)= 353f9df92ab1d5e5afe06bb7d1bb42a8ef6654b633d94818007aeafbaf03ca3d
$ echo 353f9df92ab1d5e5afe06bb7d1bb42a8ef6654b633d94818007aeafbaf03ca3d| openssl sha512 -binary -hmac $(echo -n "mmyykkeeyy" | base64 -d) | base64 -w 0
biglonghash
$ curl -X GET -H "Accept: application/json" -H "biglonghash" "https://bittrex.com/Api/v2.0/key/market/GetOrderHistory?apikey=mmyykkeeyy&nonce=353f9df92ab1d5e5afe06bb7d1bb42a8ef6654b633d94818007aeafbaf03ca3d&market=BTC-TRX"
response:
{"success":false,"message":"APISIGN_NOT_PROVIDED","result":null}
update2:
$ echo -n "/key" > tmp.bin
$ echo -n "123nonce=123" | openssl sha256 -binary >> tmp.bin
$ cat tmp.bin | openssl sha512 -binary -hmac $(echo -n "mmyysseeccrreett" | base64 -d) | base64 -w 0
output:
biglonghash
session:
$ curl --get -H "Accept: application/json" -H "apisign:biglonghash" "https://bittrex.com/Api/v2.0/key/market/GetOrderHistory?apikey=mmyyaappiikkeeyy&nonce=353f9df92ab1d5e5afe06bb7d1bb42a8ef6654b633d94818007aeafbaf03ca3d&market=BTC-TRX"
{"success":false,"message":"INVALID_SIGNATURE","result":null}
I have the following code:
curl -s --insecure -H "Content-Type: application/json" -X POST -d "{\"username\":\"$1\",\"password\":\"$2\"}" http://apiurl
In the above curl command I want to escape the " and ! in the password. Right now the password is just given as $2
I have modified the curl command as below but it doesn't seem to work
curl -s --insecure -H "Content-Type: application/json" -X POST -d "{\"username\":\"$1\",\"password\":\"$2\"}" http://apiurl
Do not try to generate JSON manually; use a program like jq, which knows how to escape things properly, to generate it for you.
json=$(jq -n --arg u "$1" --arg p "$2" '{username: $u, password: $p}')
curl -s --insecure -H "Content-Type: application/json" -X POST -d "$json" http://apiurl
My question is very simple. I want to use environment variables in a cURL command sth similar to this:
curl -k -X POST -H 'Content-Type: application/json' -d '{"username":"$USERNAME","password":"$PASSWORD"}'
When I run the command $USERNAME is passed to the command as a "$USERNAME" string not the value of the variable. Is there a way to escape this situation?
Thanks.
Single quotes inhibit variable substitution, so use double quotes. The inner double quotes must then be escaped.
... -d "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD\"}"
Since this answer was written in 2015, it has become clear that this technique is insufficient to properly create JSON:
$ USERNAME=person1
$ PASSWORD="some \"gnarly 'password"
$ echo "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD\"}"
{"username":"person1","password":"some "gnarly 'password"}
$ echo "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD\"}" | jq .
parse error: Invalid numeric literal at line 1, column 47
The quoting problem are clear. The (shell) solutions are not
Current best practice: use a JSON-specific tool to create JSON:
jq
$ jq -n -c --arg username "$USERNAME" --arg password "$PASSWORD" '$ARGS.named'
{"username":"person1","password":"some \"gnarly 'password"}
jo
$ jo "username=$USERNAME" "password=$PASSWORD"
{"username":"person1","password":"some \"gnarly 'password"}
And with curl:
json=$( jq -n -c --arg username "$USERNAME" --arg password "$PASSWORD" '$ARGS.named' )
# or
json=$( jo "username=$USERNAME" "password=$PASSWORD" )
# then
curl ... -d "$json"
For less quoting, read from standard input instead.
curl -k -X POST -H 'Content-Type: application/json' -d #- <<EOF
{ "username": "$USERNAME", "password": "$PASSWORD"}
EOF
-d #foo reads from a file named foo. If you use - as the file name, it reads from standard input. Here, standard input is supplied from a here document, which is treated as a double-quoted string without actually enclosing it in double quotes.
curl -k -X POST -H 'Content-Type: application/json' -d '{"username":"'$USERNAME'","password":"'$PASSWORD'"}'
Here the variable are placed outside of "'" quotes and will be expanded by shell (just like in echo $USERNAME). For example assuming that USRNAME=xxx and PASSWORD=yyy the argv[7] string passed to curl is {"username":"xxx","password":"yyy"}
And yes, this will not work when $USERNAME or $PASSWORD contain space characters.
Our: curl -k -X POST -H 'Content-Type: application/json' -d '{"username":"'"$USERNAME"'","password":"'"$PASSWORD"'"}'
You can wrap the environment variables with "'" and you should keep the single quote for the external json object.
e.g
-d '{"username":"'"$USERNAME"'","password":"'"$PASSWORD"'"}'
I have a curl command which I want to execute from inside a shell script.
My script is as below:
iddn="`jq -r '.APPID_DN' /scr/resp.json`"
appid = `echo "$iddn" | awk -F',' '{print $1}'`
id =`echo $appid | awk '{print substr($0,4)}'`
apppwd = `jq -r '.APPID_PWD' /scr/resp.json`
tnname = `jq '.tnName' /scr/resp.json`
curl -i --user "$id":"$apppwd" -H "Content-Type: application/xml" -H "Accept: application/xml" -H "X-USER-IDENTITY-DOMAIN-NAME: tn11" --request GET "http://hostname.XX.XXXX.com:port/XXX/services/rest/version/auth/administrator/Clients"
But I am getting below error
HTTP/1.1 401 Unauthorized
Not authorized. Provide Authorization Header.
I tried by echoing the command to see how the curl command is forming,I am getting it as(userid and passwd not passed at all),
curl -i --user -H "Content-Type: application/xml" -H "Accept: application/xml" -H "X-USER-IDENTITY-DOMAIN-NAME: tn11" --request GET "http://hostname.XX.XXXX.com:port/XXX/services/rest/version/auth/administrator/Clients"
But if i am running the same command directly from command line,its running fine.Please help on this.
Regards,
Shilpi
id =`echo $appid | awk '{print substr($0,4)}'`
apppwd = `jq -r '.APPID_PWD' /scr/resp.json`
You must not put spaces around = in a shell variable assignment.