I have a question regarding SSL verification within the requests library for Python, but I believe it to me more general than that.
I am currently ignoring certificate verification because the third party API I need to connect to is using a self-signed certificate.
What are the implications for turning SSL verification off in requests? And what are the implications for not verifying SSL certificates in the real-world. Can I gaurantee the data transported is secure/encrypted?
This is a security sin, as anyone could spoof this certificate and intercept your traffic. You should just add the self-signed certificate to the trusted certificate chain of the machine which is using the API.
How you do that depends on the operating system and specific setup, but a quick google will guide you to the right solution.
Can I gaurantee the data transported is secure/encrypted?
The data is encrypted (this is TLS confidentiality guarantee) but since you did not authenticate the remote part (if you disable certificate validation or bypass all errors) you could be as well sending the encrypted content to anyone, including an attacker, which of course on his side will read it in plain, as the TLS handshake succeeded if you do not validate the remote party.
TLS provides multiple features, two major ones being authentication and confidentiality. They are orthogonal (you can have one without the other) but it may not be so useful to not have all of them.
Contrary to natural thinking, authentication is more important than confidentiality because if you have no insurance about who is the remote party, what do you gain by sending it encrypted? Nothing.
Related
Our frontend at domain.com uses an API located at api.domain.com.
For domain.com, we use SSL by LetsEncrypt. For the backend, however, it is much simpler to use self signed certificates.
Will users be presented with Red Warning Banner if they go to domain.com, which connects to https://api.domain.com with self signed certificates? Is this okay practice? Furthermore, can we replace https://api.domain.com with external IP instead?
That's not a good idea in general. The purpose of a certificate is to allow a client to verify that it is actually talking to the right server, and not for example to a man-in-the-middle attacker. The way it does that (somewhat simplified) is the certificate includes the public key of the server, and the domain name ("common name") of the server the client intended to communicate with. The certificate is then signed by another certificate of about the same type and contents and so on, until the chain reaches a certificate that does not need to be signed by another, because it is already trusted by the client (it is in the trusted roots list of your OS for example).
A self-signed certificate doesn't have this chain of signatures, it is called self-signed, because the certificate used to sign it is itself. There is no way for a client to verify the certificate (unless it explicitly has it listed as trusted of course). This means an attacker can spoof (impersonate) your API by self-signing a different certificate for the same domain name, but with a different keypair. This might allow stealing credentials or serving fake data. Note that the attacker might also relay the information entered by the user (all the requests made) to the real API, so responses (also received by the attacker first, but relayed to the victim) can easily look very real without much background information.
This could (in theory) be solved by certificate pinning, but in case of a Javascript client that's going to be difficult (if at all possible). HPKP would seem like a solution, but HPKP won't work with a self-signed (not verifiable) certificate. I'm not sure if Javascript has an appropriate level of access to the server certificate to implement pinning.
Even if you did implement pinning, a self-signed certificate also cannot be revoked. Think about what would happen if you discovered a compromise of the TLS key used for https in your api domain. You would have no way to revoke the key, so clients would still accept a MitM attacker serving the compromised key.
There goes a lot of effort, and you could implement something that is not standard, hard to get right and error-prone.
Or you could just use a free letsencrypt certificate for api.domain.com as well, for which all the necessary infrastructure and setup is already done on the main domain. :)
I am new to encryption and mutual authentication. I am supposed to have a server which issues certificates (maybe on a self-signed Linux machine) and clients which access this server to get a certificate and later these clients establish mutual authentication between themselves by verifying the certificate issued by the server without contacting the server (i.e in an offline mode).
Can anyone please point me to the right video tutorials or code and description links that can be useful to achieve the above stated scenario.
Thanks.
Can anyone please point me to the right video tutorials or code and description links that can be useful to achieve the above stated scenario.
You have not provided enough information. Here's some things that may get you pointed in the right direction.
If the same Issuer issues certificates for the server and the clients (like an organizations internal CA), then its traditional PKIX with client certificates.
If a client wants to use their own certificate not issued by anyone, then check out Origin Bound Certificates. These are 'tear off" certificates, and clients generate them on the fly as needed.
There's also a technique created by a fellow named Jake Thompson which cleverly uses the <keygen> tag to build a channel with mutual authentication to thwart MitM attacks that are facilitated by browsers.
Browsers claim interception and MitM is a valid use case in their security models. Incidentally, the usefulness of the <keygen> to thwart MitM is one of the unspoken reasons the <keygen> tag is going away. It breaks the browser's interception model.
I've been tasked with development of an intranet interface for command line software, and now I'm researching security options. Our command line application is finished, but I haven't started writing the web interface. I don't know exactly what the security requirements are for potential customers, although I believe ssh is generally acceptable for the command line interface. With this in mind, I'm asking for help developing a menu of choices with their associated pros/cons. Some day, we may consider releasing our web interface to the internet, so I'm willing to consider more security than currently necessary if it's easy and/or free.
I've been doing a lot of reading, and my tentative conclusion is that SSL security with no certificate is the best approach, not because less security is unacceptable, but because SSL is the standard and because it doesn't appear to be difficult to set up. I, a security non-expert, wouldn't need to explain why less security is acceptable to security non-experts. I could upgrade my application to use a certificate in the future if necessary.
Here's a list of SSL related security choices, sorted by my perception of security level with my comments. What level of protection do I need?
No SSL. This might be acceptable if our customers aren't worried about their employees seeing/changing each others' data. Their employees might want to share results with each other anyway, and I could use IP based access control and/or passwords for security.
Do SSL with no certificate. This encrypts the communication, which at least protects the data from being read by unauthorized employees. Using a password, this is the same level of security as ssh on the command line, right? I don't need to worry about man-in-the-middle attacks in an intranet, right? A con for this approach would be if there were loads of browser warning messages.
Do SSL with a self-signed certificate. What does this give me that no certificate gives me? If the DNS can be changed inappropriately, then the customer then my application is the least of their concerns. Worded another way, if the DNS can change, then I think ssh would be vulnerable too.
Do SSL with a local Certificate Authority. OpenSSL lets me make my own Certificate Authority. What does this give me that a self-signed certificate does not? I'm assuming that on a LAN, it's less important for the server to be verified.
Do SSL with an external Certificate Authority. Is there ever a reason to go this route for an intranet? I found some "intranet certificates" for sale online -- but it's not clear what they're offering I can't do myself.
For reference, this page might be useful for comparing certificates:
http://httpd.apache.org/docs/trunk/ssl/ssl_faq.html#aboutcerts
[update]
Here's an article discussing the risks and rules of obtaining an internal certificate from a public CA.
Yes, certificates are still useful for Intranet SSL.
There's an important difference between SSH and SSL-without-a-certificate: when you first connect to a server with SSH, your SSH stores the server's fingerprint. If you then try to connect to what the SSH client believes to be the same machine but gets back a different fingerprint, it alerts you that there might be someone intercepting your communications.
SSL-without-a-certificate, on the other hand, does not store the server's fingerprint. Your communications will still be encrypted, but if someone somehow hijacks the DNS server as you mentioned, or, as Rushyo notes, does ARP poisoning or something similar, they would be able to perform a man-in-the-middle attack. SSH, as previously mentioned, would (supposing you had connected to the correct server some time in the past) notice that the fingerprint had changed and alert you.
A self-signed certificate would be comparable in security to SSH. A man in the middle could generate their own self-signed certificate, but as long as your applications are configured to only accept that self-signed certificate, you should get an alert similar to that that SSH will give you.
A local certificate authority gives you security similar to self-signed certificates, but may be more scalable. Should you have multiple servers, each can have their own certificate, but a client only needs the top-level one to trust all of them. If a server is compromised, you can revoke its certificate rather than having to change every server's certificate.
I don't believe an external certificate authority has any advantages, other than possibly less configuration if your machines already have the certificate authority trusted.
Lastly, I don't know enough about two-factor authentication to evaluate it, but for most applications, SSL should be sufficient.
Disclaimer: I am not a security expert.
Do SSL with an external Certificate Authority. Is there ever a reason to go this route for an intranet? I found some "intranet certificates" for sale online -- but it's not clear what they're offering I can't do myself.
The benefit is that you don't need to learn how to setup your own Certificate Authority if you need to manage a decent number of certificates and/or machines. Such a certificate would already be trusted by all browsers without you needing to install your own certificates into the trusted store.
However, this is actually less secure because somebody could purchase a certificate for a different intranet and use it on your network. For this reason, SSL vendors no longer offer this service. For more information, see: https://www.godaddy.com/help/phasing-out-intranet-names-and-ip-addresses-in-ssls-6935
If you only have a very small intranet, then I would recommend using a self-signed certificate, and then just add each self-signed certificate to each computer's trusted store.
However, it quickly becomes impractical to install a new certificate on every computer in your intranet whenever you want to add a new computer. At this point, you want to setup your own Certificate Authority so that you only need to install a single CA certificate in each computer's trusted store.
I am developing an API for a web application. Desktop clients will interact with the API using simple HTTP posts (REST). I will be using SSL, there is no question about that. My question is this: should I also be encrypting the data before it is sent over SSL? The information being sent may contain confidential information. Is SSL enough or should I be doing more? My only concern with adding additional layers of security is that it will make it substantially more difficult for people to interact with the API. Any thoughts on this would be much appreciated.
No, SSL provides strong encryption as it is. Just make sure you force clients to use HTTPS, and if you're really paranoid, check if the cypher is strong enough.
The only reason you'd want to encrypt a second time is if your web application passes the data straight on to some other system. In that case, you could keep the web application ignorant of the actual data and provide end-to-end encryption between the client and the final destination.
If you trust your certification authority, your clients do, and your key is sufficiently secure (RSA 2048 will not be breakable for a while), there are no problems at all with SSL. You don't need to encrypt your data before SSLing, as SSL itself provides encryption.
If this API is to be used over the internet, the list of certification authorities here will be useful to you in choosing one.
You can also have your clients authenticate over SSL, with SSL client certificates.
IMHO, I would not add another layer of encryption on top of the already existing encryption. It will add overhead and as you say, complexity to the API. SSL exists to send secure data between two nodes, so why reinvent the wheel?
As Shtééf already pointed out, if you need end-to-end encryption instead of point-to-point then you need encryption. Other cases this might be relevant in is if your client application communicate with the server through integration services and service busses. In this case the SSL encryption is not enforced while the message is in an intermediary node and that node may do whatever it wants with the unencrypted confidential data.
Furthermore, if your clients use these integration services then they might not enforce SSL connections between the client and the integration service.
While working with highly confidential information I tend to go for end-to-end security instead of SSL encrypted communications channels for this reason.
Yes.
SSL would help with man-in-the-middle and wiretapping but there are other attacks SSL won't help with such as replay attacks.
Whenever I see it being talked about, it sounds like one simply 'turns on' SSL and then all requests/responses to/from an online server are magically secure.
Is that right? Is SSL just about code - can I write two apps and make them communicate via SSL, or do you have to somehow register/certificate them externally?
Secure web pages are requested on port 443 instead of the normal port 80. The SSL protocol (plenty complicated in and of itself) is responsible for securing communication, and using the certificate information on both the SERVER and the BROWSER to authenticate the server as being who they say they are.
Generating an SSL certificate is easy. Generating one that is based on the information embedded in 99% of web browsers costs money. But the technical aspects are not different.
You see, there are organizations (Verisign, Globalsign, etc...) that have had their certificate authority information INCLUDED with browsers for many years. That way, when you visit a site that has a certificate that they produced (signed), your browser says:
"well, if Verisign trusts XYZ.com, and I trust Verisign, then I trust XYZ.com"
The process is easy:
Go to a competent SSL vendor, such as GlobalSign. Create a KEY and Certificate Request on the webserver. Use them (and your credit card) to buy a certificate. Install it on the server. Point the web-browser to HTTPS (port 443). The rest is done for you.
SSL is a protocol for encrypted communications over a TCP connection (or some other reliable scheme). The encryption uses public key encryption using X.509 certificates. SSL handles both privacy and trust. These are related: if you don't trust the server, you don't believe that the server hasn't handed out its private key to everyone in North America.
Thus, the client has to trust the server's certificate. For public sites, this is arranged via a hierarchy of certificate authorities, with the root authorities trusted, automatically, by browsers and things like the JRE's socket implementation.
Anyone can generate a self-signed certificate for a server, but then the client has to be manually configured to trust it.
SSL is not, in itself, a magic bullet that makes everything secure. Security has no such things.
SSL is, however, an already-designed, ready-to-use system for solving a common problem: secure stream communication over a network connection.
There are two things you need to do to secure your application with SSL:
Modify the application's code to use SSL.
Determine the certificate trust model (and deploy and configure the application respectively).
Other answers and documentation provide better answers to how to do each of these things than I could provide.
I'll throw caution to the wind and attempt to condense an enormous subject.
SSL attempts to solve two problems:
1) Authentication and hence trust i.e can the client trust the server and vice versa
2) Communication without eavesdropping
1) Is handled by means of an intermediary i.e a trusted 3rd party - these are called 'Root Certificate Authorities' ( or Root CAs ) examples include Verisign, RSA etc
If a company wants to authenticate users and more importantly if a user wants to authenticate the company's website it's connecting to i.e your bank then the Root CA issues the company a certificate which effectively says 'I the trusted Root CA verify that I trust that Company X are who they say they are and am issuing a certificate accordingly'. So you get a chain of trust i.e I trust the certificate from ACME Co because Root CA Verisign created and issued it.
2) Once the two parties have authenticated then the certificate ( typically X590 ) is used to form a secure connection using public/private key encryption.
Hopelessly simple and incomplete but hope that gives a rough idea
Yes and no. You should self-sign a certificate and test the site with SSL internally before deploying it with SSL, first of all. To make the public site secure under SSL, you will need to purchase a certificate from one of any number of certificate providers. Then you will have a certificate signed by a trusted third party, tied to your domain name, so that users' browsers won't complain that the certificate is invalid, etc. Turning SSL on is pretty much just flipping a switch, otherwise.
For the most part you need to buy and register a certificate externally.
You need to have your server certificate signed by a Certificate Authority (CA), for which they will charge you. The client needs to trust that CA and have a copy of the relevant CA public key. The client can then check that you are who you claim to be (including domain name (from DNS) and display name for https).
This is a good tutorial on how to create self signed certificates for Apache.
If you want to know how SSL works on either the Server or the Client, then I suggest Googling it. As you suspected, it is a ridiculesly complex procedure, with lots of communication between the client and server, a lot of very peculiar math, and tons of processing. There is also a lot of theory involved, several protocols and many different algorithms and encryption standards. It's quite incredible how changing http:// to https:// is so simple to the user, but results in so much work for both sides, and is so secure. To really understand it you need to take a security course (multiple courses to fully understand it), as the entire history of encryption goes into making your login to Gmail secure.
Turning on TLS (colloquially "SSL") does not make your site magically secure. You may still be vulnerable to application-level vulnerabilities like stack overflows, SQL injection, XSS, and CSRF.
As other answers have explained, TLS only protects against a man in the middle. Traffic between a client and a properly-configured TLS server cannot be intercepted or modified, and the client can reliably confirm the identity of the server by validating the X.509 certificate. This prevents an attacker from impersonating your TLS server.
SSL actually does two things:
Encrypts the communication so that an observer seeing the data stream will not be able to read the conversation.
Guarantees that you are talking to who you think you are talking to.
It is only for #2 that you need to get official certificates. If you only care to encrypt the communication without setting up a trust relationship, you can use self-signed certificates or you can use an algorithm that does not require certificates (i.e. Diffie-Hellman).