I have a Node.js server that receives a request with the Client TLS certificate supplied in the XFCC header.
I would like to perform the Mutual TLS at the Application level, i.e. validate Client TLS cert against the server's CA truststore - all of this done in application code, rather than relying on a web proxy configuration.
I am using NPM's pem dependency, which is essentially a bunch of JS wrappers around openssl. In particular, the verification needed to resemble mTLS is the verify method:
openssl verify -CAfile /my/server/ca-chain.crt client-chain.crt
This works in the simplest case:
ca-chain.crt: Root CA -> Int 1 CA
client-chain.crt Root CA -> Int 1 CA -> Leaf 1
But it fails in the more complex cases where Int CA's are different:
ca-chain.crt: Root CA -> Int 1 CA
client-chain.crt Root CA -> Int 2 CA -> Leaf 2
With the following:
openssl verify -CAfile /my/server/ca-chain.crt client-chain.crt
error 20 at 0 depth lookup:unable to get local issuer certificate
As far as I understand mTLS would be successfully performed as long as all certs are valid and lead up to the same Root CA, despite different Int CA's, which means verify doesn't work as-is for the purpose of doing mTLS equivalent at the App level.
I know about s_client and s_server capabilities, but they seem like hacks for what I need, rather than a proper solution.
I guess my question is then this:
Is it possible to use openssl to verify certificate against CA chain according to the mTLS rules?
And if not possible, then what would be the way to do it without resorting to writing it from scratch?
As dave_thompson_085 pointed out in his other answer, for the openssl verify to make it work you need to be aware that it does not read the entire certificate chain from the supplied client cert file, only the last (leaf) certificate.
So I believe that this method in pem package is not entirely correct (in fact they do have open issue on that), but that's another thing for discussion.
The supposed openssl command should've been translated to is this:
openssl verify -CAfile /my/server/ca-chain.crt -untrusted client-ca-chain.crt client-leaf.crt
Here I split the leaf client cert from the rest of the chain which is passed in the -untrusted param, while -CAfile contains a chain with different a Int CA, but which eventually leads up to the same Root CA - and this is what effectively makes the client cert chain valid.
This should be fairly trivial to implement with the Node.js' openssl or similar wrapper.
Related
I am curious as to whether invocation of a single line of openssl command line interface has the ability to perform complete OCSP verification protocol, e.g. query all the OCSP responder servers in a chain to confirm the current validity of certificates.
To see if this might be so, I specified the -CAfile option as /dev/null, hoping that would avoid any cached certificates being used in lieu of lookup: As explained in #pepo 's answer, the server certificate chain is sent a part of the basic TLS1.2 handshake specified in RFC 5246 (more details in update below)
# openssl s_client -CAfile /dev/null -connect www.equifaxsecurity2017.com:443
which gave the output:
CONNECTED(00000003)
depth=3 C = US, O = Equifax, OU = Equifax Secure Certificate Authority
verify return:1
depth=2 C = US, O = GeoTrust Inc., CN = GeoTrust Global CA
verify return:1
depth=1 C = US, O = GeoTrust Inc., OU = Domain Validated SSL, CN = GeoTrust DV SSL CA - G3
verify return:1
depth=0 CN = www.equifaxsecurity2017.com
verify return:1
---
Certificate chain
0 s:/CN=www.equifaxsecurity2017.com
i:/C=US/O=GeoTrust Inc./OU=Domain Validated SSL/CN=GeoTrust DV SSL CA - G3
1 s:/C=US/O=GeoTrust Inc./OU=Domain Validated SSL/CN=GeoTrust DV SSL CA - G3
i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
<omitted>
-----END CERTIFICATE-----
subject=/CN=www.equifaxsecurity2017.com
issuer=/C=US/O=GeoTrust Inc./OU=Domain Validated SSL/CN=GeoTrust DV SSL CA - G3
---
No client certificate CA names sent
....
It looks as though openssl has found all three links in the chain without any help from cached files, and so must have communicated over the internet with agents (1) GeoTrust DV SSL CA - G3, and (2) GeoTrust Global CA, to build the chain. Is that correct?
No! It's not correct!
Has openssl also verified the chain by making the appropriate OCSP request to each of the three OCSP responders?
(My guess is "no". I am also aware that openssl ocsp ... can be used in conjunction with manual text operations on certificates to perform OSCP verification one link at a time. However, it does seem plausible, even preferable, that openssl would have been written to perform full OCSP verification, and that is why I am asking.)
UPDATE 9-14-2017:
Thanks to #pepo 's answer "SSL server (if configured correctly) will send certificate chain (except root CA certificate)", I looked up RFC 5246 and found section "7.4.2 Server Certificate" which explains the content of the "Server Certificate" part of the TLS1.2 handshake:
This is a sequence (chain) of certificates. The sender's
certificate MUST come first in the list. Each following certificate
MUST directly certify the one preceding it. Because certificate
validation requires that root keys be distributed independently, the
self-signed certificate that specifies the root certificate
authority MAY be omitted from the chain, under the assumption that
the remote end must already possess it in order to validate it in
any case.
Furthermore, thanks to #pepo's answer about the -crl_check_all option, I tried that and got the following output:
CONNECTED(00000003)
depth=0 CN = www.equifaxsecurity2017.com
verify error:num=3:unable to get certificate CRL
verify return:1
depth=1 C = US, O = GeoTrust Inc., OU = Domain Validated SSL, CN = GeoTrust DV SSL CA - G3
verify error:num=3:unable to get certificate CRL
It failed with error unable to get certificate CRL. It turns out not be critical, because the chosen website has OCSP stapling enabled.
If instead of -crl_check_all to perform CRL checking, we instead
add the option -status
which requests OCSP stapling, then the following output is received:
CONNECTED(00000003)
<stuff omitted omitted>
OCSP response:
======================================
OCSP Response Data:
OCSP Response Status: successful (0x0)
Response Type: Basic OCSP Response
Version: 1 (0x0)
Responder Id: CE2C8B1E8BD2300FD1B15446E9B594254949321B
Produced At: Sep 10 11:12:45 2017 GMT
Responses:
Certificate ID: ...
Cert Status: good
This Update: Sep 10 11:12:45 2017 GMT
Next Update: Sep 17 11:12:45 2017 GMT
Signature Algorithm: sha1WithRSAEncryption
<stuff omitted>
This shows OCSP stapling is enabled on the server side, but it appears to only be enabled for the first (the leaf) certificate, and not for the second certificate. (The self-signed third certificate must be validated independently anyway). So, to verify the second certificate, either CRL-checking or OCSP-request-response must be used. Since CRL-checking is not enabled by this particular authorization chain, that leaves only OCSP-request-response.
Thanks to #pepo 's reply, I understand much better the relationship between openssl, the TLS1.2 protocol, and these methods for verifying authorization (listed in historical order):
CRL (certificate revocation list) checking
OSCP request and response
OCSP stapling
However, a new question has also been raised:
Regarding the OCSP-stapling response sent by the server along with the certificates in the chain during the "Server Certificate" message step - this has signature information (from the next level up) that needs to be verified. Is this signature information actually verified during the processing of openssl ... -status?
UPDATE: 9-15-2017
The safe answer to the question "Is this signature information actually verified during the processing of openssl ... -status? " seems to be NO, according to this
answer
and also #dave_thompson_085 's comment below (he has looked through the source code).
Is it confusing? Yes! Oddly,
the "OpenSSL Cookbook (feistyduck, Ivan Ristić)"
is
unusually unclear about this question,
showing no explicit way to verify the signature, while also not explicitly saying the whether or not the signature has been verified. In contrast, for both of the other types revocation checking:
CRL-checking
and
OCSP-request-response (find "submit OCSP request")
the "OpenSSL Cookbook" shows explicit methods (recipes) to go the extra distance and complete the verification manually using the information already yielded by openssl. It would be a very human mistake to infer that reason "OpenSSL Cookbok" does not include the recipe for full validation of a stapled OCSP-reponse is because it is not necessary.
IMHO, it would be very responsible of OpenSSL (or any similar library) to include top level documentation, in the following order of priority
(1) explanation that OpenSSL does NOT offer a black box solution to TLS+authorization,
(2) explanation about what limited part of that solution it DOES offer (e.g., TLS handshake without authorization checking),
(3) documentation on recipes for assembling OpenSSL components to complete
the authorization checking solution.
Misunderstanding spreads very easily. Just a few days ago I was trying to write a simple program to send notification mail from my Linux Ubuntu PC. The standard Python releases (both ver 2 and 3) include an SMTP and an SSL library "implementing" TLS1.2. In 10 minutes I could write the program and walk through it in the debugger. I could see the call from the python SSL library to OpenSSL's handshake() function and assumed that handshake() must be handling all the authorization checking, based on the assumption that the SSL library and the SMTP library would not be released without including authorization checking. Strangely, the calling code in the SSL library included a post-handshake() check to make sure the received certificate name matched that of the called server. "Why would such a primitive check be necessary if handshake() already handled all the signature verifications, etc.?", I thought. That doubt started a journey peeling back layers of the TLS security onion and I haven't been able to stop crying since.
I don't want to try to re-invent the wheel, which would probably be wobbly anyway. Yet, I can't seem to find any available wheels. Where to go from here?
SSL server (if configured correctly) will send certificate chain (except root CA certificate). You can verify it here.
Openssl did not fetch these certificate but it got them served when initiating ssl connection. You can read more about s_client behavior in openssl documentation
I don't know if it performs OCSP verification but I doubt it. IMHO (based on The s_client utility is a test tool and is designed to continue the handshake after any certificate verification errors.) it does not perform any validation by default at all but you can at least enable CRL checking by specifying argument -crl_check_all
openssl s_client -connect www.equifaxsecurity2017.com:443 -crl_check_all
I'm trying to use a free ssl certificate(s?) I got from wosign in node.js, but I'm having some issues.
I'm creating my server with:
var server = https.createServer({key: serviceKey, cert: certificate}, httpHandler).listen(port)
which works fine with my self-signed certificate. To allow wosign to generate a certificate, I gave them the CSR that was generated alongside my self-signed certificate. They gave me the following files:
root.crt
3_user_my.domain.crt
2_issuer_Intermediate.crt
1_cross_Intermediate.crt
I've also read through the following couple sources to help me along:
https://www.ohling.org/blog/2015/02/wosign-free-2y-ssl-certificate.html
http://www.lowendtalk.com/discussion/41289/free-chinese-2-year-ssl-certificate-dv-kuaissl-by-wosign-com
Both mention that the "order" of the certificates is important, but I don't know what they mean there. I assumed that there would be a single certificate I could replace my self-signed certificate with. If there's an order, I assume it goes 1->2->3 as those numbers are in the filenames.
I replaced my certificate with the 3_user_my.domain.crt contents, and it works like my self-signed one - chrome says it should only work on localhost.
So how do I "order" these certs and ultimately how do I use the files given to me so that browsers will recognize it correctly?
Ok, I got it working.
A. I learned that the "certificate" used actually can be multiple certificates (crt files that only have one -----BEGIN CERTIFICATE----- and one -----END CERTIFICATE-----. So in order to make it work right, I needed to create one certificate file/string that contained 3 of the 4 certificates I got in the right order (apparently the root.crt isn't useful)
B1. The English set of certificate wosign gave me didn't match my private key, which I verified using the following:
openssl x509 -noout -modulus -in yourcertificate.crt | openssl md5
openssl rsa -noout -modulus -in private.key | openssl md5
# If both outputs match, the cert matches the key, otherwise they don't
B2. So I chose the chinese certificate option, and there was a much different set of certs in the archive I got from that. I used the bundled cert for NGINX. And that works!
A few similar threads exist but none has a checked answer or much discussion. I'm trying to setup an https server on express js but I'm getting
crypto.js:100
c.context.setKey(options.key);
^
Error: error:0906D06C:PEM routines:PEM_read_bio:no start line
I generated my .csr and .key files with
openssl req -nodes -newkey rsa:2048 -keyout myserver.key -out myserver.csr
One suggestion was to convert the .csr to a .pem by following these instructions: http://silas.sewell.org/blog/2010/06/03/node-js-https-ssl-server-example/
That didn't work.
The express.js docs (http://nodejs.org/api/https.html) show both of these files as .pem, however. If that's the issue, how would you convert a .key file to a .pem? This threat is partially helpful How to get .pem file from .key and .crt files? but if anyone knows what expressjs requires, I feel that's the missing component.
How would I check that the files are properly in ANSI, or convert them if not?
There is also some discussion on whether the file should begin with -----BEGIN ENCRYPTED PRIVATE KEY----- or -----BEGIN RSA PRIVATE KEY-----
Any help is greatly appreciated.
So i think there's at least a little bit of terminological confusion, and the node.js example you have there doesn't help by renaming everything to .pem.
Here's a general overview for how SSL works:
You generate a pair of public and private keys. For our purposes the former is your "certificate signing request" (CSR for short) and the latter is your private signing key (just "your key").
If you wanted to generate a self-signed certificate (this is useful for local testing purposes) you can turn around and use your key and your CSR to generate a certificate. This link http://www.akadia.com/services/ssh_test_certificate.html has a pretty clear run down of how to do that on a *nix based system.
For the purposes of web browsers, SSL certificates need to be co-signed by a trusted authority, e.g. a Certificate Authority (CA). You pay a CA to co-sign your cert, and vouch for your authenticity with browser vendors (who will in turn display a green padlock for your site when your website presents its certificate to browsers).
The co-signing process starts with you uploading your CSR to your CA. They will then take that CSR and generate your certificate. They will then provide you with a couple of certificates, your certificate, their root certificate, and possibly some intermediate certificates.
You then need to form a combined certificate that proves a chain of authenticity back to browsers. You do this literally just by concatenating your certificate, followed by the intermediate certificates (in whatever order was specified) ending with the root certificate. This combined certificate is what you hand to your web server.
In order to enable your web server to serve over SSL, you need to hand it your (combined) certificate as its public encryption key (which it provides to web browsers upon request), and your private encryption key, so that it can decrypt the traffic sent to it by web browsers.
So. Now with all of that in mind, you should take that CSR that you have and provide it to your CA, and get the various certificates back, concatenate them, and then use that w/ your private key in your express server.
I would like to be able to determine if a remote domain's TLS/SSL certificate is 'trusted' from the command line.
Here is an openssl example I was playing with a few weeks back, here I use openssl to acquire the certificate and then pipe it to openssl's 'verify' command. I assumed that the 'verify' command would verify the certificate, however, how I understand it now is that the 'verify' command just verifies the certificate chain (I think). (cdn.pubnub.com is just a domain I found from a quick Twitter search as an example to use)
echo "GET /" | openssl s_client -connect cdn.pubnub.com:443 | openssl x509 -text | openssl verify
As you can see from the cdn.pubnub.com domain (at the time of writing), the browser (Chrome at least) does not trust the certificate (because the certificate domain doesn't match), however, the openssl 'verify' command does not output 'trusted' or 'not trusted' or something else we can deduct that information from.
Another way I thought of doing this, is by using a headless browser (such as PhantomJS) and parsing any errors they return. It turns out that PhantomJS just errors but does not give any details, so this can not be used as the error could have been caused by something else.
I didn't think it would be this hard to find out that a certificate was trusted or not from the command line, without having to parse and check all the data that makes a certificate trusted myself which I don't think would be wise.
Is there a library or some other way I can tell if a remote domain's certificate is trusted from the command line?
curl (and libcurl) uses OpenSSL for https URLs, and checks certificate validity unless -k, --insecure option is enabled.
zsh 29354 % curl https://cdn.pubnub.com/
curl: (51) SSL peer certificate or SSH remote key was not OK
As you see, it doesn't give much details on why the certificate is invalid, but otherwise it should be as good as a headless browser, and much lighter.
It depends on what you consider "trusted". Beside the core cryptographic checks (e.g. checking the digital signature) the client usually does the following:
Check that the certificate chains to a trusted root
Verify that the current time is between the notValidBefore and not validAfter attributes.
The certificate is not revoked.
keyUsage and other certificate constraints match.
The entity we are communicating is somehow found in the subject of the certificate (for servers this usually means the hostname is listed as CN or subjectAlternativeName).
In your case the information to verify step 5 (namely the hostname) is missing, so it cannot be checked. You would have to do this step yourself.
Please note that different clients perform different checks to see if a certificate is trusted, so one answer may not apply to all possible clients. If you want to check your installation deeply, consider using the check from ssl labs https://www.ssllabs.com/ssltest /
The man page did not clearly specify this. But looking at openssl's apps implementations, SSL_CTX_use_PrivateKey* calls are usually made after SSL_CTX_use_certificate_file succeeded. I assume this is mostly used at the server side.
I recently confused the above function with SSL_CTX_load_verify_locations wherein you could specify a CA certificate file and path. It turned out that SSL_CTX_load_verify_locations is the one I needed to verify a server certificate which is signed by a Trusted Authority.
SSL_CTX_use_certificate_file() is used to load the certificates into the CTX object either in PEM or DER format. The certificates can be chained ultimately ending at the root certificates. SSL_CTX_use_certificate_file API loads the first certificate into the CTX context;not the entire chain of certificates. If you prefer that thorough check of certificates is needed then you need to opt for SSL_CTX_use_certificate_chain_file()
http://publib.boulder.ibm.com/infocenter/tpfhelp/current/index.jsp?topic=/com.ibm.ztpf-ztpfdf.doc_put.cur/gtpc2/cpp_ssl_ctx_use_certificate_file.html