Format in which the files that are sent as input to curl --data and --data-encode need to written - linux

This works:
curl -k -d 'VARTEST=vartest2&TEXTPARAM=text1%0Atext2&COMMENT=abc22' http://localhost:8080/job/TEST1/buildWithParameters?delay=0sec
curl -X POST -k 'http://localhost:8080/job/TEST1/buildWithParameters?delay=0sec&VARTEST=vartest2&TEXTPARAM=text1%0Atext2&COMMENT=abc22'
I want to be able to write the parameters to -d into a file and run the command line like this
curl -k -d #persargfile 'http://localhost:8080/job/TEST1/buildWithParameters?delay=0sec' -o abc.html
OR like this
curl -k --data-urlencode #persargfile 'http://localhost:8080/job/TEST1/buildWithParameters?delay=0sec' -o abc.html
Question:
What should be the format of the persargfile?

As the man page states, there are several formats available:
ascii (--data-ascii)
urlencoded (--data-urlencoded)
binary (--data-binary)
It looks like for your application the ascii option is required. Meaning your file should contain:
VARTEST=vartest2&TEXTPARAM=text1%0Atext2&COMMENT=abc22

Related

How can I CURL a remote file to a remote server

I am able to upload a local file to a remote server using the command
curl -T abc.pom -uusername:password URL
But I am not able to upload a remote file to that URL. The command I am using is this
curl -T remote-file-url -uusername:password URL
Is it not possible to do this? Is downloading it and then uploading it again the only option here?
My approach:
TF=/tmp/temp && curl <REMOTE_FILE_URL> -o $TF && curl -T $TF <UPLOAD_URL> && rm -f $TF
It might be possible to pipe the content of file from 1st to 2nd cURL but then the second one has to prepare the HTML form-encoded body by itself. The -T is a shorthand for this - it creates the form and populates it directly:
curl <REMOTE_FILE_URL> | curl -i -X POST -H "Content-Type: multipart/form-data" -d #- <UPLOAD_URL>
You may curl directly from the remote host by sending a command through ssh to the destination host :
ssh user#destination_host 'curl -o destination_file_path remote_file_url'

Using Environment Variables in cURL Command - Unix

My question is very simple. I want to use environment variables in a cURL command sth similar to this:
curl -k -X POST -H 'Content-Type: application/json' -d '{"username":"$USERNAME","password":"$PASSWORD"}'
When I run the command $USERNAME is passed to the command as a "$USERNAME" string not the value of the variable. Is there a way to escape this situation?
Thanks.
Single quotes inhibit variable substitution, so use double quotes. The inner double quotes must then be escaped.
... -d "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD\"}"
Since this answer was written in 2015, it has become clear that this technique is insufficient to properly create JSON:
$ USERNAME=person1
$ PASSWORD="some \"gnarly 'password"
$ echo "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD\"}"
{"username":"person1","password":"some "gnarly 'password"}
$ echo "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD\"}" | jq .
parse error: Invalid numeric literal at line 1, column 47
The quoting problem are clear. The (shell) solutions are not
Current best practice: use a JSON-specific tool to create JSON:
jq
$ jq -n -c --arg username "$USERNAME" --arg password "$PASSWORD" '$ARGS.named'
{"username":"person1","password":"some \"gnarly 'password"}
jo
$ jo "username=$USERNAME" "password=$PASSWORD"
{"username":"person1","password":"some \"gnarly 'password"}
And with curl:
json=$( jq -n -c --arg username "$USERNAME" --arg password "$PASSWORD" '$ARGS.named' )
# or
json=$( jo "username=$USERNAME" "password=$PASSWORD" )
# then
curl ... -d "$json"
For less quoting, read from standard input instead.
curl -k -X POST -H 'Content-Type: application/json' -d #- <<EOF
{ "username": "$USERNAME", "password": "$PASSWORD"}
EOF
-d #foo reads from a file named foo. If you use - as the file name, it reads from standard input. Here, standard input is supplied from a here document, which is treated as a double-quoted string without actually enclosing it in double quotes.
curl -k -X POST -H 'Content-Type: application/json' -d '{"username":"'$USERNAME'","password":"'$PASSWORD'"}'
Here the variable are placed outside of "'" quotes and will be expanded by shell (just like in echo $USERNAME). For example assuming that USRNAME=xxx and PASSWORD=yyy the argv[7] string passed to curl is {"username":"xxx","password":"yyy"}
And yes, this will not work when $USERNAME or $PASSWORD contain space characters.
Our: curl -k -X POST -H 'Content-Type: application/json' -d '{"username":"'"$USERNAME"'","password":"'"$PASSWORD"'"}'
You can wrap the environment variables with "'" and you should keep the single quote for the external json object.
e.g
-d '{"username":"'"$USERNAME"'","password":"'"$PASSWORD"'"}'

expanding shell variable in curl POST?

I use the following line to create database:
curl -X POST 'http://10.1.1.1:8086/db?u=root&p=root' -d '{name: test1}
if i try to do it from shell script:
ip=10.1.1.1
curl -X POST 'http://$ip:8086/db?u=root&p=root' -d '{name: test1}'
i have a problem with shell variable substitution within single quotas, if i try to use them within double quotas:
curl -X POST "http://$ip:8086/db?u=root&p=root" -d '{name: test1}'
variable is expanded to the right value, printing in terminal
curl -X POST "http://10.1.21.1:8086/db?u=root&p=root" -d '{name: test1}': **No such file or directory**
What would be the right solution to this problem?
Try this:
ip=10.1.1.1
curl -X POST 'http://'"$ip"':8086/db?u=root&p=root' -d '{name: test1}'

Linux cURL XML file POST - how to check for error/success?

I am using cURL to post a XML file on Linux as follows:
curl -X POST --header "Content-Type: text/xml" -d #test.xml "https://www.example.com"
How can I check the status of the cURL command to see if the file was posted or not?
Thanks for any help.
I'm not sure if I got you but you can basically check the return value of the curl command. The return value of the last command is stored in the variable $?.
Example:
curl -X POST --header "Content-Type: text/xml" -d #test.xml "https://www.example.com"
ret=$? # store return value for later usage in the error message
if [ $ret != 0 ] ; then
echo "POST failed with exit code $ret"
fi
This list of possible error codes can be found at the bottom of the man page. They are very helpful for debugging.
You can get the response status of your operation using -w %{http_code}
curl -s -o out.txt -w %{http_code} http://www.example.com/
In this example -s means silent mode, and -o out.txt means to save the response(usually html) into a file.
For this above command, you'l have output 200 when its success.

Curl: How to insert value to a cookie?

Ho to insert cookies value in curl? from firebug request headers I can see in the following
Cookie: PHPSESSID=gg792c2ktu6sch6n8q0udd94o0; was=1; uncheck2=1; uncheck3=1; uncheck4=1; uncheck5=0; hd=1; uncheck1=1"
I have tried the following:
curl http://site.com/ -s -L -b cookie.c -c cookie.c -d "was=1; uncheck2=1; uncheck3=1; uncheck4=1; uncheck5=0; hd=1; uncheck1=1" > comic
and the only thing i see in cookie.c is
PHPSESSID=gg792c2ktu6sch6n8q0udd94o0; was=1;
To pass keys/values to cURL cookie, you need the -b switch, not -d.
For the forms -d, the data will be separated by & and not by ; in your curl command.
So :
curl http://site.com/ \
-s \
-L \
-b cookie.c \
-c cookie.c \
-b "was=1; uncheck2=1; uncheck3=1; uncheck4=1; uncheck5=0; hd=1; uncheck1=1"
> comic
If you need to know the names of the forms to be POSTed, you can run the following command :
mech-dump --forms http://site.com/
It comes with libwww-mechanize-perl package with debian or derivated.

Resources