I need to login to a page and then submit an entry.
i'm able to login with the following code
curl $HOST -s -L -b $COOKIE -c $COOKIE -d "login=submit&login_name=$USER&login_password=$PASS" -o $OUTPUT
but when i try to submit an entry i'm not logged in anymore
curl $HOSTADDNEWS -L -b $COOKIE -c $COOKIE -A $HEADER -d "title=$TITLE" -o $OUTPUT.add.news
How to retain cookies between requests?
Thanks to Daniel Stenberg! $COOKIE variable had a "slash" in it... it still saved to file but wasn't able to use it.
Related
I use this curl command to download a setup.exe :
curl -k -u login:Password -O "https://url/to/my/setup.exe"
I want to get the time total :
curl -s -w 'total : %{time_total}\n' -k -u login:Password -O "https://url/to/my/setup.exe"
total : 41.165108
I want to redirect the output ( total : 41.165108 ) in an external file. I try :
curl -o test.txt -s -w 'total : %{time_total}\n' -k -u login:Password -O "https://url/to/my/setup.exe"
curl -s -w 'total : %{time_total}\n' -k -u login:Password -O "https://url/to/my/setup.exe" > test.txt
But it doesn't work...
Someone to show me ?
Thanks !
Just combine the two things you've tried:
curl -s -w 'total : %{time_total}\n' -k -u login:Password \
-O "https://url/to/my/setup.exe" -o setup.exe > total.txt
# ~~~~~~~~~~~~ ~~~~~~~~~~~
setup.exe will contain the downloaded file
total.txt will contain the total
The -o doesn't influence where the output of -w goes.
The -o parameter is in lowercase, and the file to output to, is to be entered after the parameter, an example is below,
curl -K myconfig.txt -o output.txt
I am attempting to run the following script on a virtual instance I have created on the google cloud:
#!/bin/bash
set -eu
DS=$(date "+%Y-%m-%d" -d "7 days ago")
DE=$(date "+%Y-%m-%d" -d "1 day ago")
account=123
## above specifies last weeks delivery
rm -f cookiejar
curl /dev/null -s -S -L -f -c cookiejar 'https://url.io/auth/authenticate' -d name=usr -d passwd='pwd'
curl -o /dev/null -s -S -L -f -b cookiejar -c cookiejar 'https://adloox.io/auth/adminaccounts' -d account=$account
curl -s -S -L -f -o "report1.xlsx" -J -b cookiejar -c cookiejar "https://url.io/adquality/ajax-adblocking?categoryFw=&platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&keywords=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=2628&zfTableItemPerPage=10000&zfTableExport=xlsx"
curl -s -S -L -f -o "report2.xlsx" -J -b cookiejar -c cookiejar "https://url.io/report/ajax-by-tag2?platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=10000&zfDetails=true&by_viewability=imps_sivt&device_id[]=all&tag_type_id[]=all&support_id[]=all&by_website=1&zfTableItemPerPage=10000&zfTableExport=xlsx"
Set Up
I have curl installed on my local desktop (running windows 10 with `Ubuntu 18.04.3 LTS)
I have curl installed on my remote (google cloud virtual instance set up with Ubuntu 18.04.3 LTS`)
Both have version curl 7.58.0
Issue
When run on my local desktop there is no issue and the files download. When run on my remote I am able to log in however I receive the following error for the next curl line:
+ curl -o /dev/null -s -S -L -f -b cookiejar -c cookiejar https://url.io/auth/adminaccounts -d account=123
curl: (22) The requested URL returned error: 500 Internal Server Error
Can someone confirm what else I should be looking at here? I would have thought if my linux and curl versions are the same, there wouldn't be an issue. Sorry if there are some other straight forward checks to do. This is my first time setting up a server.
If your virtual instance does not have a public IP address, this might help you out.
#!/bin/bash
set -eu
DS=$(date "+%Y-%m-%d" -d "7 days ago")
DE=$(date "+%Y-%m-%d" -d "1 day ago")
account=123
## above specifies last weeks delivery
rm -f cookiejar
curl /dev/null -s -S -L -f -c cookiejar 'https://url.io/auth/authenticate' -d name=usr -d passwd='pwd'
curl -o /dev/null -s -S -f -b cookiejar -c cookiejar 'https://adloox.io/auth/adminaccounts' -d account=$account
curl -s -S -L -f -o "report1.xlsx" -J -b cookiejar -c cookiejar "https://url.io/adquality/ajax-adblocking?categoryFw=&platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&keywords=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=2628&zfTableItemPerPage=10000&zfTableExport=xlsx"
curl -s -S -L -f -o "report2.xlsx" -J -b cookiejar -c cookiejar "https://url.io/report/ajax-by-tag2?platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=10000&zfDetails=true&by_viewability=imps_sivt&device_id[]=all&tag_type_id[]=all&support_id[]=all&by_website=1&zfTableItemPerPage=10000&zfTableExport=xlsx"
I am able to upload a local file to a remote server using the command
curl -T abc.pom -uusername:password URL
But I am not able to upload a remote file to that URL. The command I am using is this
curl -T remote-file-url -uusername:password URL
Is it not possible to do this? Is downloading it and then uploading it again the only option here?
My approach:
TF=/tmp/temp && curl <REMOTE_FILE_URL> -o $TF && curl -T $TF <UPLOAD_URL> && rm -f $TF
It might be possible to pipe the content of file from 1st to 2nd cURL but then the second one has to prepare the HTML form-encoded body by itself. The -T is a shorthand for this - it creates the form and populates it directly:
curl <REMOTE_FILE_URL> | curl -i -X POST -H "Content-Type: multipart/form-data" -d #- <UPLOAD_URL>
You may curl directly from the remote host by sending a command through ssh to the destination host :
ssh user#destination_host 'curl -o destination_file_path remote_file_url'
I am using cURL to post a XML file on Linux as follows:
curl -X POST --header "Content-Type: text/xml" -d #test.xml "https://www.example.com"
How can I check the status of the cURL command to see if the file was posted or not?
Thanks for any help.
I'm not sure if I got you but you can basically check the return value of the curl command. The return value of the last command is stored in the variable $?.
Example:
curl -X POST --header "Content-Type: text/xml" -d #test.xml "https://www.example.com"
ret=$? # store return value for later usage in the error message
if [ $ret != 0 ] ; then
echo "POST failed with exit code $ret"
fi
This list of possible error codes can be found at the bottom of the man page. They are very helpful for debugging.
You can get the response status of your operation using -w %{http_code}
curl -s -o out.txt -w %{http_code} http://www.example.com/
In this example -s means silent mode, and -o out.txt means to save the response(usually html) into a file.
For this above command, you'l have output 200 when its success.
Ho to insert cookies value in curl? from firebug request headers I can see in the following
Cookie: PHPSESSID=gg792c2ktu6sch6n8q0udd94o0; was=1; uncheck2=1; uncheck3=1; uncheck4=1; uncheck5=0; hd=1; uncheck1=1"
I have tried the following:
curl http://site.com/ -s -L -b cookie.c -c cookie.c -d "was=1; uncheck2=1; uncheck3=1; uncheck4=1; uncheck5=0; hd=1; uncheck1=1" > comic
and the only thing i see in cookie.c is
PHPSESSID=gg792c2ktu6sch6n8q0udd94o0; was=1;
To pass keys/values to cURL cookie, you need the -b switch, not -d.
For the forms -d, the data will be separated by & and not by ; in your curl command.
So :
curl http://site.com/ \
-s \
-L \
-b cookie.c \
-c cookie.c \
-b "was=1; uncheck2=1; uncheck3=1; uncheck4=1; uncheck5=0; hd=1; uncheck1=1"
> comic
If you need to know the names of the forms to be POSTed, you can run the following command :
mech-dump --forms http://site.com/
It comes with libwww-mechanize-perl package with debian or derivated.