This is a really basic question but I somehow can't figure it out:
I am using netcat to get HTTP response from a website. Example:
request="GET / HTTP/1.1"
echo -ne $request | nc 127.0.0.1 80
If the website requires authentication I can send a POST request and include user/pass in form-data.
But how about apache password protected directories? Let's say this is in my apache.conf:
<Directory "/var/www/html/">
...
AuthType Basic
AuthName "Authentication Required"
AuthUserFile "/etc/htpasswd/.htpasswd"
Require valid-user
</Directory>
When I visit 127.0.0.1, I get prompted for user/pass. Browser shows request as "stalled". After I provide my credentials, request is finished. If I inspect request headers, I can't find my credentials or any other form data there.
My question is: how are these credentials sent? How can I send them using netcat utility?
p.s.: I am not dead set on using netcat, if there is another command line utility that can achive this in a simple way, that's also great.
They are sent in a Authorization header, encoded in base64 (Wireshark auto-decodes them for you if you expand the header).
You can specify the username and password easily using curl:
curl --user name:password localhost
You can see how it works by using -v to see the headers, for example:
curl -sv --user name:password localhost
In the header output you will see a line like this:
> Authorization: Basic bmFtZTpwYXNzd29yZA==
Try to use CURL insted of netcat
Related
We have been struggling for months trying to enable the LDAS connection with our LAMP application. Its an internal server and we don´t need any special cypher but just accessing to LDAPS port.
lampstack-7.2.26-0
Apache/2.4.6 (CentOS)
PHP 5.4.16
OpenSSL 1.0.2k-fips
Current .htaccess config that works ok with LDAP:
AuthType Basic
AuthName "Login with your Company ID"
AuthBasicProvider ldap
AuthLDAPURL "ldap://xxx/DC=xx,DC=x,DC=grp?sAMAccountName"
AuthLDAPBindDN "svc-apache-ldap-auth"
AuthLDAPBindPassword "<password>"
AuthLDAPCompareAsUser on
LDAPReferrals off
Require ldap-group CN=sxx,OU=xx,OU=Services,OU=Groups,OU=Enterprise,DC=xx,DC=x,DC=xx
From the client we can successfully do the ldapsearch test from root user with both LDAP and LDAPs
as root user:
ldapsearch -ZZ --> OK
ldapsearch (unsecured) --> OK
with domain user:
ldapsearch -ZZ --> ldap_start_tls: Connect error (-11)
ldapsearch (unsecured) --> OK
One possible issue could be a permissions one? LAMP uses the "daemon" especial user to run the services.
We have tried many things. The most obvious is to add the "s" to
AuthLDAPURL "ldap://xxx/DC=xx,DC=x,DC=grp?sAMAccountName"
Like this
AuthLDAPURL "ldaps://xxx/DC=xx,DC=x,DC=grp?sAMAccountName"
When we try that, we get a 500 error at the webpage and this message in the logs
AH01695: auth_ldap authenticate: user XXX authentication failed; URI / [LDAP: ldap_set_option failed.Could not set LDAP_OPT_X_TLS to LDAP_OPT_X_TLS_HARD][Can't contact LDAP server]
We also try adding
TLS_CRLCHECK none
TLS_REQCERT never
to the ldap.conf files but the same error appears.
Other things have been tried but I have just put the most relevant ones. Not sure what else to try to get working LDAPS
Thanks for the help
SOLVED: It was finally fixed by using a different Lamp Stack version 7.4.30-1
Seems that version 7.2.26-0 had this bug that LDAPS was not working.
I have a x.example which serves traffic for both a.example and b.example.
x.example has certificates for both a.example and b.example. The DNS for a.example and b.example is not yet set up.
If I add an /etc/hosts entry for a.example pointing to x.example's ip and run curl -XGET https://a.example, I get a 200.
However if I run curl --header 'Host: a.example' https://x.example, I get:
curl: (51) SSL: no alternative certificate subject name matches target
host name x.example
I would think it would use a.example as the host. Maybe I'm not understanding how SNI/TLS works.
Because a.example is an HTTP header the TLS handshake doesn't have access to it yet? But the URL itself it does have access to?
Indeed SNI in TLS does not work like that. SNI, as everything related to TLS, happens before any kind of HTTP traffic, hence the Host header is not taken into account at that step (but will be useful later on for the webserver to know which host you are connecting too).
So to enable SNI you need a specific switch in your HTTP client to tell it to send the appropriate TLS extension during the handshake with the hostname value you need.
In case of curl, you need at least version 7.18.1 (based on https://curl.haxx.se/changes.html) and then it seems to automatically use the value provided in the Host header. It alo depends on which OpenSSL (or equivalent library on your platform) version it is linked to.
See point 1.10 of https://curl.haxx.se/docs/knownbugs.html that speaks about a bug but explains what happens:
When given a URL with a trailing dot for the host name part: "https://example.com./", libcurl will strip off the dot and use the name without a dot internally and send it dot-less in HTTP Host: headers and in the TLS SNI field.
The --connect-to option could also be useful in your case. Or --resolve as a substitute to /etc/hosts, see https://curl.haxx.se/mail/archive-2015-01/0042.html for am example, or https://makandracards.com/makandra/1613-make-an-http-request-to-a-machine-but-fake-the-hostname
You can add --verbose in all cases to see in more details what is happening. See this example: https://www.claudiokuenzler.com/blog/693/curious-case-of-curl-ssl-tls-sni-http-host-header ; you will also see there how to test directly with openssl.
If you have a.example in your /etc/hosts you should just run curl with https://a.example/ and it should take care of the Host header and hence SNI (or use --resolve instead)
So to answer your question directly, replace
curl --header 'Host: a.example' https://x.example
with
curl --connect-to a.example:443:x.example:443 https://a.example
and it should work perfectly.
The selected answer helped me find the answer, even though it does not contain the answer. The answer in the mail/archive link Patrick Mevzek provided has the wrong port number. So even following that answer will cause it to continue to fail.
I used this container to run a debugging server to inspect the requests. I highly suggest anyone debugging this kind of issue do the same.
Here is how to address the OP's question.
# Instead of this:
# curl --header 'Host: a.example' https://x.example
# Do:
host=a.example
target=x.example
ip=$(dig +short $target | head -n1)
curl -sv --resolve $host:443:$ip https://$host
If you want to ignore bad certificates matches, use -svk instead of -sv
curl -svk --resolve $host:443:$ip https://$host
Note: Since you are using https, you must use 443 in the --resolve argument instead of 80 as was stated on the mail/archive
I had a similar need. Didn't have sudo access to update hosts file.
I use resolve parameter and also added the DNS host name as a header parameter.
--resolve <dns name>:<port>:<ip addr>
curl --request POST --resolve dns_name:443:a.b.c.d 'https://dns_name/x/y' --header 'Host: dns_name' ....
Cheers..
I want to verify that my web application does not have a path traversal vulnerability.
I'm trying to use curl for that, like this:
$ curl -v http://www.example.com/directory/../
I would like the HTTP request to be explicitly made to the /directory/../ URL, to test that a specific nginx rule involving proxy is not vulnerable to path traversal. I.e., I would like this HTTP request to be sent:
> GET /directory/../ HTTP/1.1
But curl is rewriting the request as to the / URL, as can be seen in the output:
* Rebuilt URL to: http://www.example.com/
(...)
> GET / HTTP/1.1
Is it possible to use curl for this test, forcing it to pass the exact URL in the request? If not, what would be an appropriate way?
The curl flag you are looking for is curl --path-as-is .
I'm not aware of a way to do it via curl, but you could always use telnet. Try this command:
telnet www.example.com 80
You'll see:
Trying xxx.xxx.xxx.xxx...
Connected to www.example.com.
Escape character is '^]'.
You now have an open connection to www.example.com. Now just type in your command to fetch the page:
GET /directory/../ HTTP/1.1
And you should see your result. e.g.
HTTP/1.1 400 Bad Request
You can use an intercepting proxy to capture a request to your application and repeat the request with parameters changed, such as the raw URL that is requested from the application.
The free version of Burp Suite will allow this using the Repeater.
However, there are alternatives that should also allow this such as Zap, WebScarab and Fiddler2.
I have to make some request to a https site via openssl. In the browser there is no problem at all, it's a simple GET request without any cookies, I watch it via fiddler. My problem is that I have to make this request via openssl and it waits too long and then I get an empty response.
It looks like:
(cat <REQUEST>; sleep 5) | openssl s_client -quiet -connect <HOST>:<PORT> > <variable>
I have many https sites and only one of them causes this. What could be the problem?
Make sure you send all the headers your browser does. In particular, Referer and Accept may play a role in it (and Cookie, but you said there are no cookies).
Other than that, the server's SSL certificate may be unverifiable by your client. The browser is usually more forgiving, so your request would still be successful. If you can translate your request into a wget command (i.e. wget https://host:port/URL), if the certificate has a problem it will report something like "cannot verify <host>'s certificate". If you attempt the same request with wget --no-check-certificate https://host:port/URL and it succeeds, you'll know the certificate is the problem.
I have this proxy address: 125.119.175.48:8909
How can I perform a HTTP request using cURL like curl http://www.example.com, but specifying the proxy address of my network?
From man curl:
-x, --proxy <[protocol://][user:password#]proxyhost[:port]>
Use the specified HTTP proxy.
If the port number is not specified, it is assumed at port 1080.
General way:
export http_proxy=http://your.proxy.server:port/
Then you can connect through proxy from (many) application.
And, as per comment below, for https:
export https_proxy=https://your.proxy.server:port/
The above solutions might not work with some curl versions I tried them for myself(curl 7.22.0). But what worked for me was:
curl -x http://proxy_server:proxy_port --proxy-user username:password -L http://url
Hope it solves the issue better!
Beware that if you are using a SOCKS proxy, instead of a HTTP/HTTPS proxy, you will need to use the --socks5 switch instead:
curl --socks5 125.119.175.48:8909 http://example.com/
You can also use --socks5-hostname instead of --socks5 to resolve DNS on the proxy side.
as an adition to airween, another good idea is to add this into your .bashrc, so you'll be able to switch from non proxied to proxied environment:
alias proxyon="export http_proxy='http://YOURPROXY:YOURPORT';export https_proxy='http://YOURPROXY:YOURPORT'"
alias proxyoff="export http_proxy='';export https_proxy=''"
WHERE YOURPROXY:YOURPORT is exactly that, your ip and port proxy :-).
Then, simply doing
proxyon
your system will start to use the proxy, and just the opposite with:
proxyoff
use the following
curl -I -x 192.168.X.X:XX http://google.com
192.168.X.X:XX put your proxy server ip and port.
-v verbose mode it will give more details including headers and response.
I like using this in order to get the IP under which I am seen
curl -x http://proxy_server:proxy_port https://api.ipify.org?format=json && echo
Hope this helps someone.
For curl you can configure proxy in your ~/.curlrc (_curlrc on Windows) file by adding proxy value, the syntax is:
proxy = http://username:password#proxy-host:port
curl -I "https://www.google.com" -x 1.1.1.1:8080
Just summarizing all great mentioned answers:
curl -x http://<user>:<pass>#<proxyhost>:<port>/ -o <filename> -L <link>
With a proxy with authentication I use:
curl -x <protocol>://<user>:<password>#<host>:<port> --proxy-anyauth <url>
because, I don't know why curl doesn't use/catch http[s]_proxy environment variables.
You don't need to export the http[s]_proxy shell variable if you're just setting the proxy for a one off command. e.g.
http_proxy=http://your.proxy.server:port curl http://www.example.com
That said, I'd prefer curl -x if I knew I was always going to use a proxy.
sudo curl -x http://10.1.1.50:8080/ -fsSL https://download.docker.com/linux/ubuntu/gpg
This worked perfectly for me, the error comes because curl need to set
the proxy
Remmember replace the proxy with your proxy, mine, "example" was
http://10.1.1.50:8080/.
curl -vv -ksL "https://example.com" -x "http://<proxy>:<port>"
Depending on your workplace, you may also need to specify the -k or the --insecure option for curl in order to get past potential issues with CA certificates.
curl -x <myCompanyProxy>:<port> -k -O -L <link to file to download>
In case the proxy is using automatic proxy with PAC file. We can find the actual proxy from the javascript from the PAC URL.
And if the proxy needs authentication, we can first use a normal web-browser to access the website which will promote authentication dialog. After authentication, we can use wireshark to capture the http package sends to the proxy server, from the http package, we can get the auth token from http header: Proxy-Authorization
Then we can set the http_proxy environment variable and also include auth token in the http header: Proxy-Authorization
export http_proxy=http://proxyserver:port
curl -H "Proxy-Authorization: xxxx" http://targetURL
curl -x socks5://username:password#ip:port example.com
For http proxy tunnels (needed for the TLS protocol), you need to specify -p (aka --proxytunnel) instead of -x.
curl post about proxies
tl;dr the proxy tunnel uses a newer "CONNECT" keyword instead of a modified "GET"
This was needed for the node http-proxy-middleware library.
Only got a clue once I used wget which worked out of the box.