How can I send POST request with a csv or a text file to the server running on a localhost using cURL.
I have tried curl -X POST -d #file.csv http://localhost:5000/upload but I get
{
"message": "The browser (or proxy) sent a request that this server could not understand."
}
My server is flask_restful API. Thanks a lot in advance.
There are many alternate ways to accomplish this. One way is
I have used the following:
curl -F ‘data=#<file_location>’ <URL>
Eg. curl -F data=#data.csv localhost:5000/h
Your command can also be changed slightly like this
curl -X POST -H 'Content-Type: text/csv' -d #file.csv http://localhost:5000/upload
The above is one of the many ways.It can be sent either as a part of form or data, or multipart, etc. You can refer Medium Post
Curl's default Content-Type is application/x-www-form-urlencoded so your problem is probably that the data you are POSTing is not actually form data. It might work if you set the content type header properly:
-H "Content-Type: text/csv"
Though it does depend on the server.
Related
I am new to using CURL. I am trying to connect to a SharePoint URL and pull the data in the form of json. My URL looks like this:
.../_api/web/lists/GetByTitle('titles list')/items
If I give the URL as it is without any encoding, it fails with a HTTP/1.1 400 Bad Request error.
I tried using -G and --data-urlencode like below:
curl -v -G -L --ntlm --user user:password -H 'Accept: application/json;odata=verbose' ".../_api/web/lists/GetByTitle" --data-urlencode "('titles list')" -d "/items"
doing this will convert my URL to .../_api/web/lists/GetByTitle?%28%27titles%20list%27%29&/items
But it fails with HTTP/1.1 404 Not Found since using -G will append to the URL with a ? and & . Putting ? and & to append to the URL will give me a different URL and hence the 404 not found error.
I have no issues accessing other end points like ../_api/web/lists since there is no need to encode it I guess.
How do I properly encode my URL and get the data without any errors?
Any help would be appreciated. Thank you.
I was able to solve the issue by directly giving URL with the encoded characters to cURL command. Like this:
curl -v -L --ntlm --user user:password -H 'Accept: application/json;odata=verbose' ".../_api/web/lists/GetByTitle%28%27titles%20List%27%29/items"
I hope this helps someone. I know I am no expert with linux or cURL and any better answers to this are welcome.
Before you post this as a duplicate; I've tried many of the suggestions I found around SO.
So far I've been using postman to post data to a Java web service. That works great as follows:
I now want to do the same using curl, so I tried it using the following ways:
$ curl -X POST --data "this is raw data" http://78.41.xx.xx:7778/
$ curl -X POST --data-binary "this is raw data" http://78.41.xx.xx:7778/
$ curl -X POST --data "#/home/kramer65/afile.txt" http://78.41.xx.xx:7778/
$ curl -X POST --data-binary "#/home/kramer65/afile.txt" http://78.41.xx.xx:7778/
Unfortunately, all of those show an empty raw body on the receiving side.
Does anybody know what I'm doing wrong here? How is my curl request different from my postman request? All tips are welcome!
curl's --data will by default send Content-Type: application/x-www-form-urlencoded in the request header. However, when using Postman's raw body mode, Postman sends Content-Type: text/plain in the request header.
So to achieve the same thing as Postman, specify -H "Content-Type: text/plain" for curl:
curl -X POST -H "Content-Type: text/plain" --data "this is raw data" http://78.41.xx.xx:7778/
Note that if you want to watch the full request sent by Postman, you can enable debugging for packed app. Check this link for all instructions. Then you can inspect the app (right-click in Postman) and view all requests sent from Postman in the network tab :
I have written an Application, should upload some files to several mobile devices. By now, I am using dropbox, I have an App (https://www.dropbox.com/developers), so I can upload any data to the clients Dropbox.
Now I would like to switch to owncloud because of some security concerns. I already have read this:
Uploading files to an ownCloud server programatically
But unfortunately it didn't help.
I tried
curl -X PUT -u username:password "http://myserver.com/owncloud/remote.php/webdav/test" -F f=#"/tmp/test"
The file was uploaded, but there was a Problem: Some kind of Header was added to my file.
Original test-file:
test
Uploaded File:
--------------------------00c5e21306fd0b2d Content-Disposition: form-data; name="f"; filename="test" Content-Type:
application/octet-stream
Dies ist ein Test.
--------------------------00c5e21306fd0b2d--
While this is really annoying on any text-Files it is a desaster on any binary files like JPGs etc, because they can't be opened any more after uploading.
That's why, I tried the other possible way, which was described:
mischka#lappy:/tmp$ curl -X PUT -u 'username:password' "http://myserver/owncloud/remote.php/webdav/test" --data-binary #"/tmp/test"
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
<s:exception>Sabre\DAV\Exception\BadRequest</s:exception>
<s:message>expected filesize 19 got 0</s:message>
</d:error>
But the result was even worse!
Can anyone tell me, what I am doing wrong?
-F means form upload, you should use --data-binary instead:
curl -X PUT -u username:password
"http://myserver.com/owncloud/remote.php/webdav/test" --data-binary #"/tmp/test"
Upload a file you have to make a PUT request to the destiny of the file for example: http://yourOwnCloudServer/remote.php/webdav/text.txt
Here is the CURL command:
curl -X PUT -u username:password "https://yourOwnCloudServer/remote.php/webdav/text.txt" -F myfile=#"/Users/gokul/Desktop/text.txt"
You can also use --data-binary for media files.
"https://yourOwnCloudServer/remote.php/webdav/image.jpg" --data-binary #"/Users/gokul/Desktop/image.jpg"
Remember to use HTTPS or in most owncloud installations will reply
<p>The document has moved here.</p>
Is there a way to tell cURL command not to use server's side cache?
e.g; I have this curl command:
curl -v www.example.com
How can I ask curl to send a fresh request to not use the cache?
Note: I am looking for an executable command in the terminal.
I know this is an older question, but I wanted to post an answer for users with the same question:
curl -H 'Cache-Control: no-cache' http://www.example.com
This curl command servers in its header request to return non-cached data from the web server.
The -H 'Cache-Control: no-cache' argument is not guaranteed to work because the remote server or any proxy layers in between can ignore it. If it doesn't work, you can do it the old-fashioned way, by adding a unique querystring parameter. Usually, the servers/proxies will think it's a unique URL and not use the cache.
curl "http://www.example.com?foo123"
You have to use a different querystring value every time, though. Otherwise, the server/proxies will match the cache again. To automatically generate a different querystring parameter every time, you can use date +%s, which will return the seconds since epoch.
curl "http://www.example.com?$(date +%s)"
Neither -H 'Pragma: no-cache' nor -H 'Cache-Control: no-cache' helped me. In browser with "cmd+shift+r" (full reload) I was seeing a new version than the output of curl in terminal.
How to debug for yourself
To get the exact same result, I went to browser > F12 (Dev Tools) > Network/Requests > Right-click on the request > "Copy as cURL" and got the equivalent cURL command to the browser's call.
Then, I pasted that in the terminal and started removing the params one by one, until I found that surprisingly --compressed was making a difference in my case. (Calling CloudFront AWS)
You could try following ways to force not to keep Cache when curl.
Note: The server may or may not be configured to respect the Cache-Control header. Therefore, whether this method will work is dependent on the server or website we’re sending the HTTP request to.
curl command with the Cache-Control header
$ curl -H 'Cache-Control: no-cache, no-store' http://www.example.com
Adding the Pragma HTTP Header
$ curl -H 'Pragma: no-cache' http://www.example.com
Finally, the most common way: bypass the cache by changing the URL
curl -H 'Cache-Control: no-cache, no-store' http://www.example.com?$(date +%s)
My problem is I should use single quotes in a query string.
My code
if (event.queryStringParameters && event.queryStringParameters['Name']) {
responseMessage = 'Hello, ' + event.queryStringParameters['Name'] + '!';
}
My requests
curl https://cssrq1srud.execute-api.us-east-1.amazonaws.com/serverless_lambda_stage/hello?Name=Terraform
returns zsh: no matches found
curl 'https://cssrq1srud.execute-api.us-east-1.amazonaws.com/serverless_lambda_stage/hello?Name=Terraform'
returns {"message":"Hello, Terraform!"}
This didn't work for me:
curl -H 'Cache-Control: no-cache' http://www.example.com
but this ended up doing the trick:
curl -H 'Cache-Control: no-cache' http://www.example.com&someFakeParam=$RANDOM
the &someFakeParam=$RANDOM makes the URL unique each time and bypasses caching.
Simple GET request with curl returns empty body (Content-Length: 0):
curl -v https://www.flyorientthai.com/booking/en/index.php
On the other hand wget can handle that url just fine:
wget https://www.flyorientthai.com/booking/en/index.php
What's wrong with curl?
Turned out 'Connection: Keep-Alive' header is required. It's added by default to request with wget but not with curl.
For anyone using "Copy as cURL" in the Chrome developer tools network tab to generate a curl command — one of the lines in the generated curl command is a header that looks something like...
-H 'If-None-Match: W/"18f6a6-6p8AL7X/p71IhN/WztZm60Aue4k"'
That causes the server to return an empty 304 response if nothing's changed. Just whip it out.
Let's try "Content-Type: application/x-www-form-urlencodeds"