I have a question about SSL. As I know, when we use browser to request from https server, it will make an SSL handshake first then all data will be encryption in the connection. But if I make a request without browser (like request module in nodejs, postman...), will it be an SSL handshake and data encryption on the connection?
Anyone know please explain to me, thank you.
First, stop saying SSL. Its successor is TLS, and it will have 20 years next January.
TLS is a protocol sitting on top of TCP typically (other variants can also use UDP), and provides on top of TCP features some new features about endpoints authentication and transport confidentiality and integrity.
In a way, you can understand it as being sandwiched between TCP and the higher level application protocol, like HTTP.
Saying otherwise you can use many others protocols on top of TLS: you have all email related ones (SMTP, IMAP, POP, etc.), you can have FTP on top of it (while probably not a good idea nowadays), XMPP for realtime communications, etc.
In short, any protocol using TCP could use TLS with some adaptation.
So HTTP is one case among others. HTTP is between an HTTP client and an HTTP server, or webserver for short.
A browser is an HTTP client. One among many ones. When you use curl or wget you are also an HTTP client. So if any HTTP client access an http:// link it will first do the TLS handshake, after the TCP connection and before starting to do anything really related to the HTTP protocol.
You have specialized libraries dealing with TLS so that not all program need to recode everything about this again, since it is also complicated.
Related
I'm trying to create a very secure connection between client and server using Node.js, Express.js and TLS (1.2).
I think my problem is in understanding what TLS actually is - meaning what is being exchanged, when and how by who.
Anyhow, I'm searching the internet like a nutter (crazy person) to try and figure out following:
what does var tls = require('tls'); invoke?
what does var https = require('https'); invoke?
I can get tls working when using another node as a client, but in this case the client will be a user in a browser. Can I use both for a browser or only https??
Thanks
Let's indeed start with what TLS is.
TLS is a way to provide secure connections between a client and a server. It does this by providing a safe way for clients and servers to exchange keys so they can then use public-key cryptography to secure their transmission. The exact mechanism is found here, but it's really not important for this answer.
Now, what is https? Well first, let's talk about HTTP. HTTP is a protocol that defines how web servers and clients talk and exchange web pages or data. Basically, it includes a request from a client and the server responds with a numerical message, headers, and (optionally) a body. If you're familiar with writing web pages, this is obvious.
So now, finally, what is HTTPS? HTTPS is a version of HTTP using TLS to secure data. This means that clients and servers can use the same protocol they're used to, wrapped in encryption.
Now, let's talk about these in node.js.
When you use require('tls'), you're only using the encryption layer, without defining the protocol. This will work fine for anything that doesn't expect an exact protocol, such as your other node.js client.
When you use require('https'), you're specifically using HTTP over TLS. The https module is actually a subclass of the tls module! (Oops, actually, the https.Server is a subclass of tls.Server) This means that whenever you're using the https module, you're also using the tls one.
Now, the final question: What does the browser want? If you've been following everything I've said, you can see that the browser wants https. In fact, it's likely that most of the webpages you've visited today has been over https.
In my setup, I have 2 layers of transparent proxies. When a client makes an SSL request, I wish to have the first proxy it meets simply forward the traffic to another one without attempting to do the handshake with the client.
The setup seems funny, but it is justified in my case - the 2nd proxy registers itself to the first one (through some other service) only occassionally. It tells the first: "I'm interested in some traffic that looks like___". In most cases, the 1st proxy simply does the work.
Can an httpProxy (in node-proxy) proxy SSL requests? Must I use an httpsProxy (which will then do the handshake with the client)?
You could do all of this with the existing httpsProxy if you wanted to. Unless you are wanting to use a non-Node proxy or proxy to a different server, I can't see what you would gain by having two.
Simply add the required the logging/signing logic to the existing httpsProxy.
Typically, I use https on the proxy to both restrict the number of open ports and to remove the need to do https on all of the Node servers running. You can also add Basic Auth using http-basic library too.
See my example code: https://github.com/TotallyInformation/node-proxy-https-example/blob/master/proxy.js
EDIT 2012-05-15: Hmm, after some thought, I wonder if you shouldn't be looking at something like stunnel to do what you want rather than Node?
(For reference, I've already made some of those points in my answer to your similar question on ServerFault.)
If you are after a MITM proxy (that is, a proxy that can look inside the SSL content by using its own certificates, which can work provided the clients are configured to trust them), it will hardly be fully transparent, since you will at least have to configure its clients to trust its certificates.
In addition, unless all your client use the server name indication extension, the proxy itself will be unable to determine reliably which host to issue its certificate for (something that a normal HTTPS proxy would have been able to know by looking at the CONNECT request issued by the client).
If you're not after a MITM proxy, then you might as well let the initial connection through via your router. If you want to record that traffic, your router might be able to log the encrypted packets.
Having your router catch the SSL/TLS packets to send them transparently to a proxy that will merely end up relaying that traffic untouched anyway to the target server doesn't make much sense. (By nature, the transparent proxy will imply the client isn't configured to know about it, so it won't even send its CONNECT method with which you could have had the requested host and port. Here, you'll really have nothing more than what the router can do.)
EDIT: Once again, you simply won't be able to use an HTTP proxy to analyse the content of the connection transparently. Even when using a normal proxy, an HTTPS connection is relayed straight through to the target server. The SSL/TLS connection itself is established between the original client and the target server. The point of using SSL/TLS is to protect this connection, and to make the client notice if something is trying to look inside the connection.
Plain HTTP transparent proxy servers work because (a) the traffic can be seen (in particular, the request line and the HTTP Host header are visible so that the proxy can know which request to make itself) and (b) the traffic can be altered transparently so that the initial client doesn't notice that the request wasn't direct and works as if it was.
Neither of these conditions are true with HTTPS. HTTPS connections that go through an HTTP proxy are simply tunnel, after explicit request from the client, which has sent a CONNECT command and was configured to make use of such a proxy.
To do something close to what you're after, you'd need an SSL/TLS server that accepts the SSL/TLS connection and deciphers it (perhaps something like STunnel) before your HTTP proxy. However, this won't be transparent, because it won't be able to generate the right certificates.
What is the difference between HTTPS and SSL? I read about them and found following:
HTTPS: HTTPS is a combination of HTTP with SSL/TLS. It means that HTTPS is basically HTTP connection which is delivering the data secured using SSL/TLS.
SSL: SSL is a secure protocol that works on the top of HTTP to provide security. That means SSL encrypted data will be routed using protocols like HTTP for communication.
I am wondering where is the difference between these two? Or both are identical?
The explanation of SSL that you've found is wrong.
SSL (Secure Socket Layer) or TLS (Transport Layer Security) works on top of the transport layer, in your examples TCP. TLS can be used for more or less any protocol, HTTPS is just one common instance of it.
HTTP is an application layer protocol.
In regular, non-encrypted HTTP, the protocol stack can look like this:
HTTP
TCP
IP
Ethernet
When using HTTPS, the stack looks like this:
HTTP
TLS (SSL)
TCP
IP
Ethernet
HTTPS runs over SSL (as it's name suggests, HTTP-over-SSL), not SSL over HTTP. First SSL session is established, then all HTTP data are wrapped into secured SSL packets before sending and after receiving.
SSL (Secure Sockets Layer) is a standard security technology to create an encrypted link between a server and a client. This link ensures that all data passed between the server and the client remain private and secure.
It was designed to support protocols such as FTP, HTTP, TELNET.
Hypertext Transfer Protocol Secure (HTTPS) or “HTTP Secure,” is an application specific implementation that is a combination of the Hypertext Transfer Protocol (HTTP) with the SSL/TLS. HTTPS is used to provide encrypted communication and secure identification of a server, so that no middle man can intercept the data easily.
As everything in HTTP is in plain text (or encoded) , it is used with SSL/TLS to encrypt it.
Found this link which explains SSL, TLS, HTTPS :
http://nexsniper.blogspot.com/2017/11/what-is-ssl-tls-and-https.html
I have an interesting network security challenge that I can't figure out the best way to attack.
I need to provide a way to allow two computers (A and B) that are behind firewalls to make a secure connection to each other using only a common "broker" untrusted server on the internet (somewhere like RackSpace). (the server is considered untrusted because the customers behind the firewalls won't trust it since it is on an open server) I can not adjust the firewall settings to allow the networks to directly connect to each other because the connections are no known ahead of time.
This is very similar to a NAT to NAT connection problem like that handled by remote desktop help tools (crossloop, copilot, etc).
What I would really like to find is a way to open an SSL connection between the two hosts and have the public server broker the connection. Preferably when host A tries to connect to host B, it should have to provide a token that the broker can check with host B before establishing the connection.
To add another wrinkle to this, the connection mechanism needs to support two types of communication. First, HTTP request/response to a REST web service and second persistent socket connection(s) to allow for real-time message passing.
I have looked at the techniques I know about like OpenSSL using certificates, OAuth, etc, but I don't see anything that quite does what I need.
Has anyone else handled something like this before? Any pointers?
You can solve your problem with plain SSL.
Just have the untrusted server forward connections between the client hosts as opaque TCP connections. The clients then establish an end-to-end SSL connection over that forwarded TCP tunnel - with OpenSSL, one client calls SSL_accept() and the other calls SSL_connect().
Use certificates, probably including client certificates, to verify that the other end of the SSL connection is who you expect it to be.
(This is conceptually similar to the way that HTTPS connections work over web proxies - the browser just says "connect me to this destination", and establishes an SSL connection with the desired endpoint. The proxy just forwards encrypted SSL data backwards and forwards, and since it doesn't have the private key for the right certificate, it can't impersonate the desired endpoint).
In general, SSL is packet-based protocol (for the purpose of solving your task). If you can have the host forward the packets back and forth, you can easily have SSL-secured communication channel. One thing you need is something like our SSL/TLS components, which allow any transport and not just sockets. I.e. the component tells your code "send this packet to the other side" or "do you have anything for me to receive?" and your code communicates with your intermediate server.
Would like to understand whether the HTTPS body part of the Response is encrypted. Also, in a HTTPS request whether the header are transmitted as plain text / encrypted?
Is there any tool with which I can observe the raw HTTPS traffic without decrypting it.
HTTPS is HTTP over SSL. So the whole HTTP communication is encrypted.
As the other posts say - HTTPS is HTTP (plaintext) wrapped in SSL on top of the TCP/IP layer. Every part of the HTTP message is encrypted. So the stack looks like:
TCP/IP
SSL
HTTP
As far as encryption goes, there is no way to see any part of the HTTP message with SSL around it.
If you need to debug your traffic I suggest the following:
Use a network traffic watcher (like Ethereal) to watch the creation of connections. This will let you see the connection be initiated. It will show you the start of the SSL Handshake, details on failures, and when the session is set up, there will be chains of cipher text. The ciphertext is not very useful, but its presence lets you know data is going back and forth.
Deubg your http layer in the clear prior to setting up HTTPS. Every application or web server I've ever worked with has let me turn off HTTPS, and host the same set of URLs in the clear. Do this, and watch it with the same network tool.
If you get both sides talking with HTTP and everything breaks on HTTPS, it's time to look at either the SSL session establishment or anything in between the two points that may be interrupting the flow.
YES https flow is encrypted. When an https connection is initialized, it uses a strong encryption algorithm to handshake and agree with other part on a less strong, but much faster encryption algorithm for the flow.
To observe network packets, you can use sniffers like http://www.ethereal.com/.
When using HTTPS, the entire content of the request and reply are encrypted, including the headers and body. The HTTP protocol in plaintext happens on top of, TLS or SSL, so what's on the wire is encrypted.
The entire HTTP session is encrypted including both the header and the body.
Any packet sniffer should be able to show you the raw traffic, but it'll just look like random bytes to someone without a deep understanding of SSL, and even then you won't get beyond seeing the key exchange as a third party.
Any packet capture/sniffing tool can show you the raw HTTPS traffic. To view the actual contents (by decrypting it), use Fiddler.
Anything sent over https is encrypted using SSL transport
Try WireShark or Fiddler as helpful tools for this.