From this question I am able to get a wget url for the oracle jdk.
I intend to use it in a script vis
wget_opts="-c --no-check-certificate --no-cookies --header --load-cookies="Cookie: oraclelicense=accept-securebackup-cookie""
jdk_download_url="http://download.oracle.com/otn-pub/java/jdk/7u55-b13/jdk-7u55-linux-x64.tar.gz"
/usr/bin/wget $wget_opts $jdk_download_url
When I echo the above command it appears ok and is able to correctly download the file.
But on running the command in the script I get the below
--2014-06-04 14:19:43-- http://oraclelicense=accept-securebackup-cookie%22/
Resolving oraclelicense=accept-securebackup-cookie"... failed: Name or service not known.
wget: unable to resolve host address “oraclelicense=accept-securebackup-cookie"”
--2014-06-04 14:20:03-- http://download.oracle.com/otn-pub/java/jdk/7u55-b13/jdk-7u55-linux-x64.tar.gz
Resolving download.oracle.com...
Wget gets the wrong URL.
How do I correct this?
Use an array:
wget_opts=( -c
--no-check-certificate
--no-cookies
--header
--load-cookies="Cookie: oraclelicense=accept-securebackup-cookie"
)
jdk_download_url="http://download.oracle.com/otn-pub/java/jdk/7u55-b13/jdk-7u55-linux-x64.tar.gz"
# use the exact quoting below
/usr/bin/wget "${wget_opts[#]}" "$jdk_download_url"
Try this:
wget_opts='-c --no-check-certificate --no-cookies --header --load-cookies="Cookie: oraclelicense=accept-securebackup-cookie"'
jdk_download_url="http://download.oracle.com/otn-pub/java/jdk/7u55-b13/jdk-7u55-linux-x64.tar.gz"
/usr/bin/wget $wget_opts $jdk_download_url
Check the difference between single and double quotes in the bash manual.
EDIT: In fact you have some mistakes in your wget command line. Here is the correct line.
OPTS="-c --no-check-certificate --no-cookies --header Cookie:oraclelicense=accept-securebackup-cookie"
URL="http://download.oracle.com/otn-pub/java/jdk/7u55-b13/jdk-7u55-linux-x64.tar.gz"
wget $OPTS $URL
The --load-cookies option take a file as argument and not a string. We have to use the --header option with Cookie: oraclelicense=accept-securebackup-cookie. After tests, I have seen that wget does not care about spaces in the header field. So we can use directly Cookie:oraclelicense=accept-securebackup-cookie
If you use the --debug option you will see the correct formatted request :
GET /otn-pub/java/jdk/7u55-b13/jdk-7u55-linux-x64.tar.gz HTTP/1.1
Range: bytes=5307-
User-Agent: Wget/1.15 (linux-gnu)
Accept: */*
Host: download.oracle.com
Connection: Keep-Alive
Cookie: oraclelicense=accept-securebackup-cookie
You need to escape the double quotes that are within other double quotes:
wget_opts="-c --no-check-certificate --no-cookies --header --load-cookies=\"Cookie: oraclelicense=accept-securebackup-cookie\""
Or enclose the string in single quotes if you don't need variable interpolation:
wget_opts='-c --no-check-certificate --no-cookies --header --load-cookies="Cookie: oraclelicense=accept-securebackup-cookie"'
Also, in your command, you need a $ in front of jdk_download_url:
/usr/bin/wget $wget_opts $jdk_download_url
Related
I am running out of ideas to have this to work in my gitlab-ci.yml.
I have a variable called TARGET, and I want to successfully inject its value inside the json payload of my http POST request in form of curl bash command.
job:
stage: deploy
script:
- curl -k --request POST ${URL} --header "Authorization:Basic ${TOKEN}" --header 'Content-Type:application/json' --data "{"extra_vars":{"target":${TARGET}}}"
variables:
TARGET: host1
the pipeline output keeps complaining with error message:
{"detail":"JSON parse error - Expecting property name enclosed in double quotes: line 1 column 2 (char 1)\nPossible cause: trailing comma."}
I also tried escaping some special characters with \ as below, but not working either:
curl -k --request POST ${URL} --header "Authorization:Basic ${TOKEN}" --header 'Content-Type:application/json' --data "\{"extra_vars":\{"target":${TARGET}\}\}"
your helps are very appreciated.
There are two issues here. You are missing double quotes around the value for target inside the JSON since TARGET is a string value.
You are also wrapping your data argument with double quotes hence you should not use them inside the payload without either escaping them or switching to single quotes.
Keeping that in mind the payload may look something like this:
'{"extra_vars":{"target":"'${TARGET}'"}}'
Here's a quick tip on how to test the output using httpbin.org:
TARGET=host1 && curl -v https://httpbin.org/post -H "Content-Type: application/json" -d '{"extra_vars":{"target":"'${TARGET}'"}}'
Which will respond with the sent payload (which is ofcourse escaped as it is also wrapped inside JSON:
"data": "{\"extra_vars\":{\"target\":\"host1\"}}",
I have a cURL call that I'm trying to use to send file data to a remote server.
curl -X POST -u username:password -d 'data=#/path/to/file.ext&version=2&action=Parse' http://fqdn.to.server.i.control/Parser.cgi
curl -X POST -u username:password -d 'data=#localFile.ext&version=2&action=Parse' http://fqdn.to.server.i.control/Parser.cgi
cat file.ext | curl -X POST -u username:password -d 'data=#-&version=2&action=Parse' http://fqdn.to.server.i.control/Parser.cgi
The file contents are URI encoded already. Using Perl and CGI on the server side.
My problem is that when the server tries to access that "data" line, value I have is only "file.ext" - the path is stripped out and the file's contents are not used ($cgi->param("data") is just "file.ext", "localFile.ext" or "-" respectively).
Any indication as to what I'm doing wrong?
#MattJacob was correct; my syntax was wrong. data=#... should have been #... and the data= portion should have been in the file. Boy am I thick.
I am using curl command to call rest api. I want to post data and my curl command looks like:
curl –x POST -u 'username:PW' -k -H "Content-Type:application/json" -d '{"json-input":{"handler":"getContent","image":true,"video":false,"text":false,"source":"1","lage":"testlage1"}}' -i http://localhost:8080/com.knime.enterprise.server/rest/v4/jobs/3fd2ca61-c173-4160-a20d-45c387f65f64
I am getting following message:
curl: (6) Could not resolve host: xn--x-5gn curl: (6) Could not
resolve host: POST
The letter before X is wrong. It is supposed to be an ascii minus ('-', ascii code 0x2d / 45) and not the unicode dash character (U+2013) as used in the question.
curl will treat all options that don't start with a minus as a URL, which makes it convert the dash-X string to a IDN hostname and try it. It then continues to try the "POST" host name as that follows the dash-X... None of those host names can be resolved, which is the curl error messages you see.
Then finally: don't use -X POST when you do a post with -d (or with -F)! Just remove the -X POST entirely and things will work better.
check the codepage settings of the terminal software in use and compare these to the host settings
in our case we saw the same strange hostname error returned to a simple
curl -v http://{hostname}:{port}
we discovered the problem was the dash character; the codepage specified in Putty was not consistent with the codepage used at the host so curl treated the -v expression as the hostname.
Is there a way to tell cURL command not to use server's side cache?
e.g; I have this curl command:
curl -v www.example.com
How can I ask curl to send a fresh request to not use the cache?
Note: I am looking for an executable command in the terminal.
I know this is an older question, but I wanted to post an answer for users with the same question:
curl -H 'Cache-Control: no-cache' http://www.example.com
This curl command servers in its header request to return non-cached data from the web server.
The -H 'Cache-Control: no-cache' argument is not guaranteed to work because the remote server or any proxy layers in between can ignore it. If it doesn't work, you can do it the old-fashioned way, by adding a unique querystring parameter. Usually, the servers/proxies will think it's a unique URL and not use the cache.
curl "http://www.example.com?foo123"
You have to use a different querystring value every time, though. Otherwise, the server/proxies will match the cache again. To automatically generate a different querystring parameter every time, you can use date +%s, which will return the seconds since epoch.
curl "http://www.example.com?$(date +%s)"
Neither -H 'Pragma: no-cache' nor -H 'Cache-Control: no-cache' helped me. In browser with "cmd+shift+r" (full reload) I was seeing a new version than the output of curl in terminal.
How to debug for yourself
To get the exact same result, I went to browser > F12 (Dev Tools) > Network/Requests > Right-click on the request > "Copy as cURL" and got the equivalent cURL command to the browser's call.
Then, I pasted that in the terminal and started removing the params one by one, until I found that surprisingly --compressed was making a difference in my case. (Calling CloudFront AWS)
You could try following ways to force not to keep Cache when curl.
Note: The server may or may not be configured to respect the Cache-Control header. Therefore, whether this method will work is dependent on the server or website we’re sending the HTTP request to.
curl command with the Cache-Control header
$ curl -H 'Cache-Control: no-cache, no-store' http://www.example.com
Adding the Pragma HTTP Header
$ curl -H 'Pragma: no-cache' http://www.example.com
Finally, the most common way: bypass the cache by changing the URL
curl -H 'Cache-Control: no-cache, no-store' http://www.example.com?$(date +%s)
My problem is I should use single quotes in a query string.
My code
if (event.queryStringParameters && event.queryStringParameters['Name']) {
responseMessage = 'Hello, ' + event.queryStringParameters['Name'] + '!';
}
My requests
curl https://cssrq1srud.execute-api.us-east-1.amazonaws.com/serverless_lambda_stage/hello?Name=Terraform
returns zsh: no matches found
curl 'https://cssrq1srud.execute-api.us-east-1.amazonaws.com/serverless_lambda_stage/hello?Name=Terraform'
returns {"message":"Hello, Terraform!"}
This didn't work for me:
curl -H 'Cache-Control: no-cache' http://www.example.com
but this ended up doing the trick:
curl -H 'Cache-Control: no-cache' http://www.example.com&someFakeParam=$RANDOM
the &someFakeParam=$RANDOM makes the URL unique each time and bypasses caching.
What is the difference between this two curl calls:
curl -X POST -d "/path/file.txt" http://api-path
curl -X POST -d "#/path/file.txt" http://api-path
The first form sends the string "/path/file.txt" as data, probably not what you want. The # makes curl interpret it as a filename to read from. See http://curl.haxx.se/docs/manpage.html, search for the "--data" option.