Looking at LetsEncrypt FAQ page, they have clearly stated that email encryption and code signing require a different type of certificate and therefore they are not supported by LetsEncrypt.
My understanding is that https and S/MIME both require X.509 certificates. What is the difference between certificates these two technologies require?
Among other things (like the encryption key), an X.509 certificate also specifies what it may be used for. X.509 certificates for HTTPS and S/MIME usage basically have different usages.
The structure of an X.509 certificate is fairly complex. Its possible usage depends on attributes and extensions within the certificate, and require that specific combinations of them with specific values must be present.
For example, an S/MIME certificate, requires a Key Usage attribute with something like Encrypt, Verify, Wrap, Derive, a Key Usage extension that must have the Critical attribute set to Yes and include a Usage attribute with Key Encipherment plus in the Extended Key Usage extension (sic!) it should list the Purpose attribute value Email Protection.
Note that Key Usage extension Data Encipherment is not required, because in S/MIME, the data contents are encrypted by a random symmetric key which then is encrypted with the private key from the certificate. This is called hybrid encryption and it is done for performance and scalability reasons.
The certificate requirements even extend into the certificate chain, meaning that the certificate must be signed by another certificate that has been issued to sign certificates for that usage.
Note that the above example may not be 100% correct, because the subject is so complex, and I don't fully understand every aspect of it myself. I found this quote which I think describes the situation fairly well:
I think a lot of purists would rather have PKI be useless to anyone in
any practical terms than to have it made simple enough to use, but
potentially "flawed". — Chris Zimman
Resources that helped me:
X.509 Style Guide by Peter Gutmann: an attempt to shed light on the relations of attributes and extensions and their interpretations
RFC 4262 - X.509 Certificate Extension for Secure/Multipurpose Internet Mail Extensions (S/MIME) Capabilities
Related
I am not an expert (not even a novice :))) in security certificates and such - thus the question - but I was wondering if one can generate a new certificate by using a known fingerprint? :D (lost my certificate for a google play app and was wondering about this workaround)
Thanks.
This is practically impossible. The fingerprint uses a cryptographic hash like SHA-1 or SHA-256. With current technology it is impossible for a normal user to create the input which results in a specific hash value. It is doable with lots of efforts (i.e. still very very expensive) when MD5 was used for the fingerprint (not common), it might be doable by the NSA for SHA-1 and it is considered impossible to do it today for SHA-256.
Apart from that certificates are commonly not validated by the fingerprint but because a trusted CA has signed this certificate. This signature is part of the certificate so that you would not only need to recreate the certificate but also somehow get a valid signature which is impossible without having the private key of the CA which of course you don't.
I want to ensure that client libraries (currently in Python, Ruby, PHP, Java, and .NET) are configured correctly and failing appropriately when SSL certificates are invalid. Shmatikov's paper, The Most Dangerous Code in the World:
Validating SSL Certificates in Non-Browser Software, reveals how confusing SSL validation is so I want to thoroughly test the possible failures.
Based on research a certificate is invalid if:
It is used before its activation date
It is used after its expiry date
It has been revoked
Certificate hostnames don't match the site hostname
Certificate chain does not contain a trusted certificate authority
Ideally, I think I would have one test case for each of the invalid cases. To that end I am currently testing an HTTP site accessed over HTTPS, which leads to a failure that I can verify in a test like so:
self.assertRaises(SSLHandshakeError, lambda: api.call_to_unmatched_hostname())
This is incomplete (only covering one case) and potentially wrong, so...
How can you test that non-browser software properly validates SSL certificates?
First off, you'll need a collection of SSL certificates, where each has just one thing wrong with it. You can generate these using the openssl command line tool. Of course, you can't sign them with a trusted root CA. You will need to use your own CA. To make this validate correctly, you'll need to install your CA certificate in the client libraries. You can do this in Java, for example, using the control panel.
Once you have the certificates, you can use the "openssl s_server" tool to serve an SSL socket using each one. I suggest you put one certificate on each port.
You now have to use the client library to connect to a port, and verify that you get the correct error message.
I know that Python by default does no certificate validation (look at the manual for httplib.HTTPSConnection). However, m2crypto does do validation. Java by default does do validation. I don't know about other languages.
Some other cases you could test:
1) Wildcard host names.
2) Certificate chaining. I know there was a bug in old browsers where if you had a certificate A signed by the root, A could then sign B, and B would appear valid. SSL is supposed to stop this by having flags on certificates, and A would not have the "can sign" flag. However, this was not verified in some old browsers.
Good luck! I'd be interested to hear how you get on.
Paul
Certificate hostnames don't match the site hostname
This is probably the easiest to check, and failure (to fail) there is certainly a good indication that something is wrong. Most certificates for well-known services only use host names for their identity, not IP addresses. If, instead of asking for https://www.google.com/, you ask for https://173.194.67.99/ (for example) and it works, there's something wrong.
For the other ones, you may want to generate your own test CA.
Certificate chain does not contain a trusted certificate authority
You can generate a test certificate using your test CA (or a self-signed certificate), but let the default system CA list be used for the verification. Your test client should fail to verify that certificate.
It is used before its activation date, It is used after its expiry date
You can generate test certificates using your test CA, with notBefore/notAfter dates that make the current date invalid. Then, use your test CA as a trusted CA for the verification: your test client should fail to validate the certificate because of the dates.
It has been revoked
This one is probably the hardest to set up, depending on how revocation is published. Again, generate some test certificates that you've revoked immediately, using your own test CA.
Some tools expect to be configured with a set of CRL files next to the set of trusted CAs. This requires some setup for the test itself, but very little online setup: this is probably the easiest. You can also set up a local online revocation repository, e.g. using CRL distribution points or OCSP.
PKI testing can be more complex than that more generally. A full test suite would require a fairly good understanding of the specifications (RFC 5280). Indeed, you may need to check the dates for all intermediate certificates, as well as various attributes for each certificate in the chain (e.g. key usage, basic constraints, ...).
In general, client libraries separate the verification process into two operations: verifying that the certificate is trusted (the PKI part) and verifying that it was issued to the entity you want to connect to (the host name verification part). This is certainly due to the fact these are specified in different documents (RFC 3280/5280 and RFC 2818/6125, respectively).
From a practical point of view, the first two points to check when using an SSL library are:
What happens when you connect to a known host, but with a different identifier for which the certificate isn't valid (such as its IP address instead of the host)?
What happens when you connect to a certificate that you know cannot be verified by any default set of trusted anchors (for example, a self-signed certificate or from your own CA).
Failure to connect/verify should happen in both cases. If it all works, short of implementing a full PKI test suite (which require a certain expertise), it's often the case that you need to check the documentation of that SSL library to see how these verifications can be turned on.
Bugs aside, a fair number of problems mentioned in this paper are due to the fact that some library implementations have made the assumption that it was up to their users to know what they were doing, whereas most of their users seem to have made the assumption that the library was doing the right thing by default. (In fact, even when the library is doing the right thing by default, there is certainly no shortage of programmers who just want to get rid of the error message, even if it makes their application insecure.)
I would seem fair to say that making sure the verification features are turned on would be sufficient in most cases.
As for the status of a few existing implementations:
Python: there was a change between Python 2.x and Python 3.x. The ssl module of Python 3.2 has a match_hostname method that Python 2.7 doesn't have. urllib.request.urlopen in Python 3.2 also has an option to configure CA files, which its Python 2.7 equivalent doesn't have. (This being said, if it's not set, verification won't occur. I'm not sure about the host name verification.)
Java: verification is turned on by default for both PKI and host name for HttpsUrlConnection, but not for the host name when using SSLSocket directly, unless you're using Java 7 and you've configure its SSLParameters using setEndpointIdentificationAlgorithm("HTTPS") (for example).
PHP: as far as I'm aware, fopen("https://.../") won't perform any verification at all.
What is the difference between s/mime ,tsp and pgp? Reference : Bouncycastle.
Bouncycastle has packages for all of them, just don't understand what is the difference and what is the purspose of each package
S/MIME means Secure/Multipurpose Internet Mail Extensions. It is a standard for public key encryption and signing of MIME data. Mostly used in email. It uses certificate authorities issuing certificates for users, similar to how SSL certificates work.
PGP means Pretty Good Privacy. It is a data encryption and decryption software that provides cryptographic privacy and authentication for data communication. It follows the OpenPGP standard. Also used in email, among other things. It uses a decentralized network of trust. No certificate authorities.
TSP is a lot of things.
In addition to Zed's answer, TSP stands for Timestamping protocol, defined in RFC 3161. As is this protocol is not very usable. It's used as a third-party proof of signing time in CMS, CAdES, PDF signatures, PAdES and XAdES. Note, that Authenticode doesn't use TSP for timestamping (another format is used).
Both S/MIME and the PGP mail system uses the hybrid cryptography.But they use different formats for key exchange. Wile PGP depends upon each user’s key exchange, S/MIME uses hierarchically validated certifier for key exchange.PGP was developed to address the security issues of plain text messages. But S/MIME is designed to secure all kinds of attachments/data files. currently S/MIME dominates the modern mail system due to the complexity of implementing PGP mail system. i can not say anything about TSP before knowing further details about it.
Question for all the SSL experts out there:
We have an embedded device with a little web server on it, and we can install our own SSL self-signed certificates on it. The client is written in .NET (but that doesn't matter so much).
How can I authenticate the device in .NET? Is it enough to compare the fingerprint of the certificate against a known entry in the database?
My understanding is that the fingerprint is a hash of the whole certificate, including the public key. A device faking to be my device could of course send the same public certificate, but it couldn't know the private key, right?
Or do I have to build up my own chain of trust, create my own CA root certificate, sign the web server certificate and install that on the client?
What you propose is in principle ok. It is for example used during key signing parties. Here the participants usually just exchange their name and fingerprints of their public keys and make sure that the person at the party really is who he/she claims. Just verifying fingerprints is much easier than to verify a long public key.
Another example is the so called self certifying file system. Here again only hashes of public keys get exchanged over a secure channel. (I.e. these hashes are embedded in URLs.) In this scheme the public keys don't have to be sent securely. The receiver only has to check that the hash of the public keys matche the hashes embedded in the URLs. Of course the receiver also has to make sure that these URLs come from a trusted source.
This scheme and what you propose are simpler than using a CA. But there is a disadvantage. You have to make sure that your database with hashes is authentic. If your database is large then this will likeley be difficult. If you use CAs then you only have to ensure that the root keys are authentic. This usually simplifies the key management significantly and is of course one reason, why CA based schemes are more popular than e.g. the self certifying file system mentioned above.
In the same way you wouldn't and shouldn't consider two objects to be equal just because their hash codes matched, you shouldn't consider a certificate to be authentic just because its fingerprint appears in a list of "known certificate fingerprints".
Collisions are a fact of life with hash algorithms, even good ones, and you should guard against the possibility that a motivated attacker could craft a rogue certificate with a matching fingerprint hash. The only way to guard against that is to check the validity of the certificate itself, i.e. check the chain of trust as you're implying in your last statement.
Short:
Well in theory you then do exactly what a Certificate Authority does for you. So it should be fine.
Longer:
When a Certificate Authority signs your public-key/certificate/certificate request it doesn't sign the whole certificate data. But just the calculated hash value of the whole certificate data.
This keeps the signature small.
When you don't want to establish your own CA or use a commercial/free one -
by comparing the fingerprint with the one you trust you'll gain the second most trustworthy configuration. The most trustworthy solution would be by comparing the whole certificate, because also protects you from hash collision attacks.
As the other guys here stated you should make sure to use a secure/safe hashing algorithm. SHA-1 is no longer secure.
more detailed informations to this topic:
https://security.stackexchange.com/questions/6737
https://security.stackexchange.com/questions/14330
The man page did not clearly specify this. But looking at openssl's apps implementations, SSL_CTX_use_PrivateKey* calls are usually made after SSL_CTX_use_certificate_file succeeded. I assume this is mostly used at the server side.
I recently confused the above function with SSL_CTX_load_verify_locations wherein you could specify a CA certificate file and path. It turned out that SSL_CTX_load_verify_locations is the one I needed to verify a server certificate which is signed by a Trusted Authority.
SSL_CTX_use_certificate_file() is used to load the certificates into the CTX object either in PEM or DER format. The certificates can be chained ultimately ending at the root certificates. SSL_CTX_use_certificate_file API loads the first certificate into the CTX context;not the entire chain of certificates. If you prefer that thorough check of certificates is needed then you need to opt for SSL_CTX_use_certificate_chain_file()
http://publib.boulder.ibm.com/infocenter/tpfhelp/current/index.jsp?topic=/com.ibm.ztpf-ztpfdf.doc_put.cur/gtpc2/cpp_ssl_ctx_use_certificate_file.html