I am new to using CURL. I am trying to connect to a SharePoint URL and pull the data in the form of json. My URL looks like this:
.../_api/web/lists/GetByTitle('titles list')/items
If I give the URL as it is without any encoding, it fails with a HTTP/1.1 400 Bad Request error.
I tried using -G and --data-urlencode like below:
curl -v -G -L --ntlm --user user:password -H 'Accept: application/json;odata=verbose' ".../_api/web/lists/GetByTitle" --data-urlencode "('titles list')" -d "/items"
doing this will convert my URL to .../_api/web/lists/GetByTitle?%28%27titles%20list%27%29&/items
But it fails with HTTP/1.1 404 Not Found since using -G will append to the URL with a ? and & . Putting ? and & to append to the URL will give me a different URL and hence the 404 not found error.
I have no issues accessing other end points like ../_api/web/lists since there is no need to encode it I guess.
How do I properly encode my URL and get the data without any errors?
Any help would be appreciated. Thank you.
I was able to solve the issue by directly giving URL with the encoded characters to cURL command. Like this:
curl -v -L --ntlm --user user:password -H 'Accept: application/json;odata=verbose' ".../_api/web/lists/GetByTitle%28%27titles%20List%27%29/items"
I hope this helps someone. I know I am no expert with linux or cURL and any better answers to this are welcome.
Related
as my question mention it, how do i need to change this statement to send an image in base64 to Azure?
curl -H "Ocp-Apim-Subscription-Key: ***hidden***" "https://***hidden***.cognitiveservices.azure.com/face/v1.0/detect?detectionModel=detection_03&returnFaceId=true&returnFaceLandmarks=false"
-H "Content-Type: application/json" --data-ascii "{\"url\":\"https://upload.wikimedia.org/wikipedia/commons/c/c3/RH_Louise_Lillian_Gish.jpg\"}"
Thank you!
how do i need to change this statement to send an image in base64 to Azure?
As per this documentation, your curl command seems fine.
Alternatively, you can try as suggested by muru:
'{"image" : "'"$( base64 ~/Pictures/1.jpg)"'"}
Updated answer:
As per Use the Face client library:
curl -v -X POST "https://westus.api.cognitive.microsoft.com/face/v1.0/detect?returnFaceId=true&returnFaceLandmarks=false&returnFaceAttributes={string}&recognitionModel=recognition_04&returnRecognitionModel=false&detectionModel=detection_03&faceIdTimeToLive=86400" -H "Content-Type: application/json" -H "Ocp-Apim-Subscription-Key: {subscription key}" --data-ascii "{\"url\":\"https://csdx.blob.core.windows.net/resources/Face/Images/identification1.jpg\"}"
You can refer to similar issues: can't send/request base64 to azure face api with python and Face API need support for base64 string or Local File URL or Native File URL instead of Image URL
curl -X POST "https://.cognitiveservices.azure.com/face/v1.0/detect" -H "Content-Type: application/octet-stream" -H "Ocp-Apim-Subscription-Key: 27e993*********6aaa2464" --data-binary #'/home/rafael/Downloads/person.jpg'
is the solution that works. As far as I can tell, the documentation seems outdated.
How can I send POST request with a csv or a text file to the server running on a localhost using cURL.
I have tried curl -X POST -d #file.csv http://localhost:5000/upload but I get
{
"message": "The browser (or proxy) sent a request that this server could not understand."
}
My server is flask_restful API. Thanks a lot in advance.
There are many alternate ways to accomplish this. One way is
I have used the following:
curl -F ‘data=#<file_location>’ <URL>
Eg. curl -F data=#data.csv localhost:5000/h
Your command can also be changed slightly like this
curl -X POST -H 'Content-Type: text/csv' -d #file.csv http://localhost:5000/upload
The above is one of the many ways.It can be sent either as a part of form or data, or multipart, etc. You can refer Medium Post
Curl's default Content-Type is application/x-www-form-urlencoded so your problem is probably that the data you are POSTing is not actually form data. It might work if you set the content type header properly:
-H "Content-Type: text/csv"
Though it does depend on the server.
Is there a way to tell cURL command not to use server's side cache?
e.g; I have this curl command:
curl -v www.example.com
How can I ask curl to send a fresh request to not use the cache?
Note: I am looking for an executable command in the terminal.
I know this is an older question, but I wanted to post an answer for users with the same question:
curl -H 'Cache-Control: no-cache' http://www.example.com
This curl command servers in its header request to return non-cached data from the web server.
The -H 'Cache-Control: no-cache' argument is not guaranteed to work because the remote server or any proxy layers in between can ignore it. If it doesn't work, you can do it the old-fashioned way, by adding a unique querystring parameter. Usually, the servers/proxies will think it's a unique URL and not use the cache.
curl "http://www.example.com?foo123"
You have to use a different querystring value every time, though. Otherwise, the server/proxies will match the cache again. To automatically generate a different querystring parameter every time, you can use date +%s, which will return the seconds since epoch.
curl "http://www.example.com?$(date +%s)"
Neither -H 'Pragma: no-cache' nor -H 'Cache-Control: no-cache' helped me. In browser with "cmd+shift+r" (full reload) I was seeing a new version than the output of curl in terminal.
How to debug for yourself
To get the exact same result, I went to browser > F12 (Dev Tools) > Network/Requests > Right-click on the request > "Copy as cURL" and got the equivalent cURL command to the browser's call.
Then, I pasted that in the terminal and started removing the params one by one, until I found that surprisingly --compressed was making a difference in my case. (Calling CloudFront AWS)
You could try following ways to force not to keep Cache when curl.
Note: The server may or may not be configured to respect the Cache-Control header. Therefore, whether this method will work is dependent on the server or website we’re sending the HTTP request to.
curl command with the Cache-Control header
$ curl -H 'Cache-Control: no-cache, no-store' http://www.example.com
Adding the Pragma HTTP Header
$ curl -H 'Pragma: no-cache' http://www.example.com
Finally, the most common way: bypass the cache by changing the URL
curl -H 'Cache-Control: no-cache, no-store' http://www.example.com?$(date +%s)
My problem is I should use single quotes in a query string.
My code
if (event.queryStringParameters && event.queryStringParameters['Name']) {
responseMessage = 'Hello, ' + event.queryStringParameters['Name'] + '!';
}
My requests
curl https://cssrq1srud.execute-api.us-east-1.amazonaws.com/serverless_lambda_stage/hello?Name=Terraform
returns zsh: no matches found
curl 'https://cssrq1srud.execute-api.us-east-1.amazonaws.com/serverless_lambda_stage/hello?Name=Terraform'
returns {"message":"Hello, Terraform!"}
This didn't work for me:
curl -H 'Cache-Control: no-cache' http://www.example.com
but this ended up doing the trick:
curl -H 'Cache-Control: no-cache' http://www.example.com&someFakeParam=$RANDOM
the &someFakeParam=$RANDOM makes the URL unique each time and bypasses caching.
Simple GET request with curl returns empty body (Content-Length: 0):
curl -v https://www.flyorientthai.com/booking/en/index.php
On the other hand wget can handle that url just fine:
wget https://www.flyorientthai.com/booking/en/index.php
What's wrong with curl?
Turned out 'Connection: Keep-Alive' header is required. It's added by default to request with wget but not with curl.
For anyone using "Copy as cURL" in the Chrome developer tools network tab to generate a curl command — one of the lines in the generated curl command is a header that looks something like...
-H 'If-None-Match: W/"18f6a6-6p8AL7X/p71IhN/WztZm60Aue4k"'
That causes the server to return an empty 304 response if nothing's changed. Just whip it out.
Let's try "Content-Type: application/x-www-form-urlencodeds"
I am trying to make a POST request using curl to retrieve exam results from the page : http://wbutech.net/result_odd.php .... I am using the following syntax but various errors are showing up (like content has moved permanently) although I am using the -L option...
curl -L --data-urlencode "semno=7&rectype=1&rollno=10400111005" http://wbutech.net/show-result.php
can anyone please post the working syntax here which works on the above said link correctly and retrieves the results?
(Sample Usage is enter roll as 10400111005 and click semester7 button under regular students...that is what I am trying to do using the above curl syntax)
Try:
curl -XPOST http://wbutech.net/show-result.php -d '{"semno": 7, "rectype": 1, "rollno": 10400111005}'
Other than 301/302 redirects, this website requires using credentials, try:
curl -u user:passwd -XPOST http://50.28.104.43/show-result.php -d'{"semno": 7, "rectype": 1, "rollno": 10400111005}'
replacing user:passwd with your real username and password.