I'm trying to create a very secure connection between client and server using Node.js, Express.js and TLS (1.2).
I think my problem is in understanding what TLS actually is - meaning what is being exchanged, when and how by who.
Anyhow, I'm searching the internet like a nutter (crazy person) to try and figure out following:
what does var tls = require('tls'); invoke?
what does var https = require('https'); invoke?
I can get tls working when using another node as a client, but in this case the client will be a user in a browser. Can I use both for a browser or only https??
Thanks
Let's indeed start with what TLS is.
TLS is a way to provide secure connections between a client and a server. It does this by providing a safe way for clients and servers to exchange keys so they can then use public-key cryptography to secure their transmission. The exact mechanism is found here, but it's really not important for this answer.
Now, what is https? Well first, let's talk about HTTP. HTTP is a protocol that defines how web servers and clients talk and exchange web pages or data. Basically, it includes a request from a client and the server responds with a numerical message, headers, and (optionally) a body. If you're familiar with writing web pages, this is obvious.
So now, finally, what is HTTPS? HTTPS is a version of HTTP using TLS to secure data. This means that clients and servers can use the same protocol they're used to, wrapped in encryption.
Now, let's talk about these in node.js.
When you use require('tls'), you're only using the encryption layer, without defining the protocol. This will work fine for anything that doesn't expect an exact protocol, such as your other node.js client.
When you use require('https'), you're specifically using HTTP over TLS. The https module is actually a subclass of the tls module! (Oops, actually, the https.Server is a subclass of tls.Server) This means that whenever you're using the https module, you're also using the tls one.
Now, the final question: What does the browser want? If you've been following everything I've said, you can see that the browser wants https. In fact, it's likely that most of the webpages you've visited today has been over https.
Related
I have a question about SSL. As I know, when we use browser to request from https server, it will make an SSL handshake first then all data will be encryption in the connection. But if I make a request without browser (like request module in nodejs, postman...), will it be an SSL handshake and data encryption on the connection?
Anyone know please explain to me, thank you.
First, stop saying SSL. Its successor is TLS, and it will have 20 years next January.
TLS is a protocol sitting on top of TCP typically (other variants can also use UDP), and provides on top of TCP features some new features about endpoints authentication and transport confidentiality and integrity.
In a way, you can understand it as being sandwiched between TCP and the higher level application protocol, like HTTP.
Saying otherwise you can use many others protocols on top of TLS: you have all email related ones (SMTP, IMAP, POP, etc.), you can have FTP on top of it (while probably not a good idea nowadays), XMPP for realtime communications, etc.
In short, any protocol using TCP could use TLS with some adaptation.
So HTTP is one case among others. HTTP is between an HTTP client and an HTTP server, or webserver for short.
A browser is an HTTP client. One among many ones. When you use curl or wget you are also an HTTP client. So if any HTTP client access an http:// link it will first do the TLS handshake, after the TCP connection and before starting to do anything really related to the HTTP protocol.
You have specialized libraries dealing with TLS so that not all program need to recode everything about this again, since it is also complicated.
Is it possible to have Node.js use multiple SSL certificates? I am currently using one certificate but had a new certificate issued that matches other domains.
Since my server is behind a load balancer, there are two ways to get to it and I'd like to match them. Is there a way to use two certificates, instead of creating one with both matches?
See this answer for hosting multiple domains on a single https server
In short, you can use the SNI callback from the https server.
SNI stands for Server Name Identification and it is implemented by all modern browsers.
How it works:
The browser sends the hostname unencrypted (if it supports SNI). The rest of the request is encrypted by the certificate. The HTTPs module can then let you decide which SSL certificate is going to be used to decrypt the connection.
SNI Notes:
SNI is used by AWS Cloudfront and other services, if you require a secure connection
SNI requires a modern browser for it to work.. However given that AWS uses it gives me confidence in using it too.
Depending on how you implement it, it may slow down the request.
It may be better to put a nginx proxy in front of this.
Your connection then travels like this: Client -> (HTTPS) -> NGINX -> (HTTP) -> Node
Nginx could also serve static files, which may optimise your site.
I hope this helps you. I am posting this for documentation purposes.
In my setup, I have 2 layers of transparent proxies. When a client makes an SSL request, I wish to have the first proxy it meets simply forward the traffic to another one without attempting to do the handshake with the client.
The setup seems funny, but it is justified in my case - the 2nd proxy registers itself to the first one (through some other service) only occassionally. It tells the first: "I'm interested in some traffic that looks like___". In most cases, the 1st proxy simply does the work.
Can an httpProxy (in node-proxy) proxy SSL requests? Must I use an httpsProxy (which will then do the handshake with the client)?
You could do all of this with the existing httpsProxy if you wanted to. Unless you are wanting to use a non-Node proxy or proxy to a different server, I can't see what you would gain by having two.
Simply add the required the logging/signing logic to the existing httpsProxy.
Typically, I use https on the proxy to both restrict the number of open ports and to remove the need to do https on all of the Node servers running. You can also add Basic Auth using http-basic library too.
See my example code: https://github.com/TotallyInformation/node-proxy-https-example/blob/master/proxy.js
EDIT 2012-05-15: Hmm, after some thought, I wonder if you shouldn't be looking at something like stunnel to do what you want rather than Node?
(For reference, I've already made some of those points in my answer to your similar question on ServerFault.)
If you are after a MITM proxy (that is, a proxy that can look inside the SSL content by using its own certificates, which can work provided the clients are configured to trust them), it will hardly be fully transparent, since you will at least have to configure its clients to trust its certificates.
In addition, unless all your client use the server name indication extension, the proxy itself will be unable to determine reliably which host to issue its certificate for (something that a normal HTTPS proxy would have been able to know by looking at the CONNECT request issued by the client).
If you're not after a MITM proxy, then you might as well let the initial connection through via your router. If you want to record that traffic, your router might be able to log the encrypted packets.
Having your router catch the SSL/TLS packets to send them transparently to a proxy that will merely end up relaying that traffic untouched anyway to the target server doesn't make much sense. (By nature, the transparent proxy will imply the client isn't configured to know about it, so it won't even send its CONNECT method with which you could have had the requested host and port. Here, you'll really have nothing more than what the router can do.)
EDIT: Once again, you simply won't be able to use an HTTP proxy to analyse the content of the connection transparently. Even when using a normal proxy, an HTTPS connection is relayed straight through to the target server. The SSL/TLS connection itself is established between the original client and the target server. The point of using SSL/TLS is to protect this connection, and to make the client notice if something is trying to look inside the connection.
Plain HTTP transparent proxy servers work because (a) the traffic can be seen (in particular, the request line and the HTTP Host header are visible so that the proxy can know which request to make itself) and (b) the traffic can be altered transparently so that the initial client doesn't notice that the request wasn't direct and works as if it was.
Neither of these conditions are true with HTTPS. HTTPS connections that go through an HTTP proxy are simply tunnel, after explicit request from the client, which has sent a CONNECT command and was configured to make use of such a proxy.
To do something close to what you're after, you'd need an SSL/TLS server that accepts the SSL/TLS connection and deciphers it (perhaps something like STunnel) before your HTTP proxy. However, this won't be transparent, because it won't be able to generate the right certificates.
This question already has answers here:
Is a HTTPS query string secure?
(9 answers)
Closed 9 years ago.
Suppose I setup a simple php web server with a page that can be accessed by HTTPS. The URL has simple parameters, like https://www.example.com/test?abc=123.
Is it true that the parameter here in this case will be safe from people sniffing the packets? And would this be true if the server does not employ any SSL certificate?
Yes your URL would be safe from sniffing; however, one hole that is easily overlooken is if your page references any third party resources such as Google Analytics, Add Content anything, your entire URL will be sent to the third party in the referer. If its really sensitive it doesn't belong in the query string.
As for your second part of the question, you can't use SSL if you don't have a certificate on the server.
http://answers.google.com/answers/threadview/id/758002.html
HTTPS Establishes an underlying SSL
connection before any HTTP data is
transferred. This ensures that all URL
data (with the exception of hostname,
which is used to establish the
connection) is carried solely within
this encrypted connection, and is
protected from man-in-the-middle
attacks in the same way that any HTTPS
data is.
All HTTP-level transactions within an
HTTPS connection are conducted within
the established SSL session, and no
query data is transferred before the
secure connection is established.
From the outside the only data that is
visible to the world is the hostname
and port you are connecting to.
Everything else is simply a stream of
binary data which is encrypted using a
private key shared only between you
and the server.
In the example you provide your
browser would do this:
Derive
hostname (and port if present)
from URL.
Connect to host.
Check certificate (it must be 'signed'
by a known authority, applied specifically
to correct IP address and port, and be
current).
The browser and server
exchange cryptographic data and the
browser receives a private key.
The
HTTP request is made, and encrypted with
established cryptography.
HTTP response is received. Also encrypted.
HTTP is an 'Application Layer'
protocol. It is carried on top of the
secure layer. According to the SSL
specification, drawn up by Netscape,
it dictates that no application layer
data may be transmitted until a secure
connection is established - as
outlined in the following paragraph:
"At this point, a change cipher spec
message is sent by the client, and the
client copies the pending Cipher Spec
into the current Cipher Spec. The
client then immediately sends the
finished message under the new
algorithms, keys, and secrets. In
response, the server will send its own
change cipher spec message, transfer
the pending to the current Cipher
Spec, and send its finished message
under the new Cipher Spec. At this
point, the handshake is complete and
the client and server may begin to
exchange application layer data."
http://wp.netscape.com/eng/ssl3/draft302.txt
So yes. The data contained in the URL
query on an HTTPS connection is
encrypted. However it is very poor
practice to include such sensitive
data as a password in a 'GET'
request. While it cannot be
intercepted, the data would be logged
in plaintext server logs on the
receiving HTTPS server, and quite
possibly also in browser history. It
is probably also available to browser
plugins and possibly even other
applications on the client computer.
At most a HTTPS URL could be
reasonably allowed to include a
session ID or similar non-reusable
variable. It should NEVER contain
static authentication tokens.
The HTTP connection concept is most
clearly explained here:
http://www.ourshop.com/resources/ssl_step1.html
The requested URI (/test?abc=123) is sent to the web server as part of the HTTP request header and thus encrypted.
However URLs can leak in other ways, usually web browser toolbars, bookmarks, and sending links to friends. POSTing data may be more appropriate depending on the context/sensitivity of the data you're sending.
I believe an HTTPS connection requires an SSL certificate, even a self-generated one if you don't want to buy one.
Hope that helps a bit!
depends on what you mean by safe
SSL encrypts the entire HTTP request/response, so the URL in the GET portion will be encrypted. This does not stop MITM attacks and corruption of the integrity of the SSL session itself. If a non-authoritative certificate is used, this makes potential attack vectors simpler.
Are REST request headers encrypted by SSL?
Is a similar question.
The url:s will be stored both in the server logs and in the browser history so even if they aren't sniffable they are far from safe.
On the wire, yes. At the end points (browser and server) not necessarily. SSL/TLS is transport layer security. It will encrypt your traffic between the browser and the server. It is possible on the browser-side to peek at the data (a BHO for example). Once it reaches the server-side, it is available to the recipient of course and is only as secure as he treats it. If the data needs to move securely beyond the initial exchange and protected from prying eyes on the client, you should also look at message layer security.
The SSL/TSL is a Transport Layer Security, yes the data can be picked with BHO (as #JP wrote) or any add on but also with "out of browser" HTTP sniffers. They read messaging between winsock32 and the application. The encryption takes place in the winsock32 not in the browser.
Take a look (this part was taked from the page of IEinspector):
IEInspector HTTP Analyzer is such a handy tool that allows you to monitor, trace, debug and analyze HTTP/HTTPS traffic in real-time.
Can a (||any) proxy server cache content that is requested by a client over https? As the proxy server can't see the querystring, or the http headers, I reckon they can't.
I'm considering a desktop application, run by a number of people behind their companies proxy. This application may access services across the internet and I'd like to take advantage of the in-built internet caching infrastructure for 'reads'. If the caching proxy servers can't cache SSL delivered content, would simply encrypting the content of a response be a viable option?
I am considering all GET requests that we wish to be cachable be requested over http with the body encrypted using asymmetric encryption, where each client has the decryption key. Anytime we wish to perform a GET that is not cachable, or a POST operation, it will be performed over SSL.
The comment by Rory that the proxy would have to use a self-signed cert if not stricltly true.
The proxy could be implemented to generate a new cert for each new SSL host it is asked to deal with and sign it with a common root cert. In the OP's scenario of a corportate environment the common signing cert can rather easily be installed as a trusted CA on the client machines and they will gladly accept these "faked" SSL certs for the traffic being proxied as there will be no hostname mismatch.
In fact this is exactly how software such as the Charles Web Debugging Proxy allow for inspection of SSL traffic without causing security errors in the browser, etc.
No, it's not possible to cache https directly. The whole communication between the client and the server is encrypted. A proxy sits between the server and the client, in order to cache it, you need to be able to read it, ie decrypt the encryption.
You can do something to cache it. You basically do the SSL on your proxy, intercepting the SSL sent to the client. Basically the data is encrypted between the client and your proxy, it's decrypted, read and cached, and the data is encrypted and sent on the server. The reply from the server is likewise descrypted, read and encrypted. I'm not sure how you do this on major proxy software (like squid), but it is possible.
The only problem with this approach is that the proxy will have to use a self signed cert to encrypt it to the client. The client will be able to tell that a proxy in the middle has read the data, since the certificate will not be from the original site.
I think you should just use SSL and rely on an HTTP client library that does caching (Ex: WinInet on windows). It's hard to imagine that the benefits of enterprise wide caching is worth the pain of writing a custom security encryption scheme or certificate fun on the proxy. Worse, on the encryption scheme you mention, doing asymmetric ciphers on the entity body sounds like a huge perf hit on the server side of your application; there is a reason that SSL uses symmetric ciphers for the actual payload of the connection.
How about setting up a server cache on the application server behind the component that encrypts https responses? This can be useful if you have a reverse-proxy setup.
I am thinking of something like this:
application server <---> Squid or Varnish (cache) <---> Apache (performs SSL encryption)