I am trying to implement the OPC UA verification of certificates with OpenSSL in C. Therefore I have multiple certificates which are stored in different directories.
The directories are:
CertificateTrustListDir: The folder where certificates of trusted CAs are stored.
CertificateRevocationListDir: The folder where the revocation lists for the trusted CAs are stored.
IssuersCertificatesDir: The folder where issuer certificates are stored. Issuer certificates are CA certificates necessary for the verification of the full trust chain of CA certificates in the trust list.
IssuersRevocationListDir: The folder where revocation lists for issuer CAs should be stored.
I was looking into this example and discovered the function SSL_CTX_load_verify_locations. Unfortunately I can only set the location of the trusted CAs with this method.
Is there a way to tell OpenSSL to use the directories in the supposed way?
Regards,
ckmk14
I am not entirely sure but this is how I have done:
With the X509_STORE_CTX_init function I can pass a X509_STORE with the trusted certificates (CertificateTrustListDir) and the CRLS (CertificateRevocationListDir + IssuersRevocationListDir).
The certificates of the IssuersCertificatesDir can be passed as stack with the chain-parameter of the X509_STORE_CTX_init function.
Related
I am implementing mutual authentication between a single client hosted app (CLIENT) and my spring boot 2 application (SERVER). I understand the steps to be as follows:
The server generates a keystore and truststore. The keystore being used for storing the server's certificates and private key. The truststore used for storing other credentials (certificates from certificate authority (CA) or trusted client certificates).
A CSR is raised for the server which is then passed to a CA. The CA generates a signed certificate from the CSR. This is the installed in the server keystore.
The client (which has it's own keystore and truststore) provides their public key to the server. This is then installed in the server truststore.
When a https request is made from client to server:
The client makes a request to access a protected resource.
Server responds with their public certificate.
Client verifies that certificate (looks in truststore and checks if it signed by a trusted CA).
Client presents their public certificate to server.
Server then verifies certificate against their truststore.
Assuming verification success client is granted access to the protected resource.
So I have a few things which I'm a bit confused about...
Are the steps outlined above broadly correct?
How does the server verify the client certificate? (I think it looks at the truststore for that certificate but not sure what actually happens after that).
I've seen examples of the CA certificate being installed in the server truststore instead of the actual client's public certificate ~ is there a use case when this should or should not be done? For my use case I have been provided with a signed certificate from the client (third party). The CA who signed that is different from the CA who signed the server certificate.
Does this process actually authenticate the client i.e. this client can now have access to the servers protected resources but another client who might present a different certificate will not have access? (like a more secure method of providing a username and password)
Where does common name (CN) checking come into all of this? I note in Spring Boot X.509 you can derive a username from the CN and then use this to lookup the appropriate user details from the user details service.
If the client certificate gets compromised for whatever reasons is this managed by just removing it from the server's truststore?
Is there an advantage, in my scenario of using a trusted CA e.g. verisign to produce a client certificate over a self-signed one? i.e. the certificate is passed to me directly from the trusted third party, and then installed.
In respect to your first question, yes your outlined steps are correct! Here is the general mutualSSL flow with a graphical overview: (source)
A client requests access to a protected resource.
The server presents its certificate to the client.
The client verifies the server’s certificate.
If successful, the client sends its certificate to the server.
The server verifies the client’s credentials.
If successful, the server grants access to the protected resource requested by the client.
Your second question (How does the server verify the clients certificate?):
The server verifies the clients certificate with the help of the signature. The signature is usually a hash-value, build of the complete certificate. The hash-value is signed with the private key of a corresponding CA (certificate authority). The server verifies the signature of the client certificate with the help of the CA's public certificate.
Your third question (Servers truststore containing the clients public key/certificate or the corresponding CA certificate?):
If you use for example self-signed certificates, you probably have to import the clients public key/certificate directly into the servers truststore. If your client uses an CA signed certificate, it is appropriate for you server to store the CA public key/certificate only, because it is used to verify the clients certificate.
Your fourth question (Does this process actually authenticate the client): Yes! As you can see in the answer to your second question, the certificate is verified by checking the signature. The signature is a hash over the complete certificate. A standard X.509 contains information to identify the subject. By checking the signature the subject is authenticated. A standard X.509 certificate contains amongst other things e.g. this information:
Subject name, Subject Public Key Info, Public Key Algorithm, Issuer Unique Identifier (optional), ...
Your fifth question (Where comes CN checking?): The CN (common name) verification is executed during the certificate check. The CN identifies the valid hostname for the current certificate. It is limited to one entry. As an extension the SAN (subject alternative name) was introduced. A certificate can contain more than one SAN. The CN (and the SAN) entry is part of the certificate and is verified with the help of the certificates signature check.
Your sixth question (If the client certificate gets compromised for whatever reasons is this managed by just removing it from the server's truststore?): Therefore the CAs use so called revocation lists. If you are using for example self-signed certificates it would also be okay to just remove the compromised certificate entry from the servers truststore.
Your seventh question (Is there an advantage, in my scenario of using a trusted CA e.g. verisign to produce a client certificate over a self-signed one?): There exist a few advantages of using a CA signed certificate instead of self-signed ones.
The certificate and eventually the revocation is managed by the CA
The certificate is valid to every relying party of the public CA, e.g. Verisign
Most of the public CAs offer standardized ways of creating a certificate
I would like to know what are the exact steps of certificate authentication over HTTP.
I know this has been asked before, but from what I've read it's still unclear to me how it works.
When first contacting the secure site client will send its certificate
This will be his public key ( let's say a public_key.pem file begginging with ----BEGIN PUBLIC KEY---- )
Server will look whether this certificate has been signed by a trusted CA.
The server only has a list of certificates which it trusts (you configure this keystore when configuring SSL). That is where all private keys are stored. Is this equivalent to all the trusted CAs of the server?
The next step now is to take the public_key.pem and check whether any of the certificates in the keystore have signed it.
If the above process is accurate:
First question: A 'certificate' is a private key signed by another certificate ( or self-signed )
Second question: How exactly does a server validate that a public key was signed with a specific private key ( certificate ) ?
Third question: 'certificate' A can be used to sign another 'certificate' B, consequently 'certificate' B can be used to sign certificate C and so on. If my server has certificate A in its truststore, it means it will also trust public keys coming from certificate B and C ?
Edit based on answer below
My server has cert.pem and privkey.pem. The cert.pem ( x509 certificate ) has been signed by a trusted CA using its private key ( the 'signing' process involves the CA to do 'something' with its private key and sign my Certificate signing request ).
When SSL is negotiatied my server will send the cert.pem to the client ( in some form ). How does the client determine that my public cert was signed by a trusted CA. My truststore pb only contains other public certificates of trusted CAs, so it ends up checking whether my cert.pem was signed using only public certificates of trusted CAs.
This is is the part which is unclear - and I may be missunderstanding the whole process - can a client check if my x509 certificate is valid by just having an x509 list of certificates of trusted CAs?
When first contacting the secure site client will send its certificate
A client doesn't have to send a certificate. Judging from your question, it doesn't sound like you're asking specifically about client-cerificate authentication. In many sites, like if you go to google, stackoverflow, facebook, etc, the client / browser will not send a certificate.
A 'certificate' is a private key signed by another certificate ( or self-signed )
Not quite. A certificate contains a public key that is signed by the private key that corresponds to another certificate.
I think this is worth clearing up. A certificate itself will only ever contain the public key, or the "Subject Public Key Info" in x509 parlance. There is a corresponding private key to that public key that is stored separately from the x509 certificate.
There are some formats that "combine" the certificate and the private key in to a single file, but at this point it isn't a certificate anymore, it's a file containing a certificate - an example of this is PKCS#12. This is a format that can store as many certificates and private keys as needed.
A private key may not even necessarily be a file on disk - it could be in a Hardware Security Module, SmartCard, etc.
'certificate' A can be used to sign another 'certificate' B, consequently 'certificate' B can be used to sign certificate C and so on. If my server has certificate A in its truststore, it means it will also trust public keys coming from certificate B and C ?
Yes. This is called a certificate chain. A is the "root" certificate, B is an "intermediate", and C would be an "end-entity" or "leaf" certificate.
This is very common among HTTPS certificates by CAs. Certificates are never issued directly off the root, they're issued off the intermediate.
How exactly does a server validate that a public key was signed with a specific private key ( certificate ) ?
Well this makes the assumption that a certificate is a private key, which it isn't.
The server, as in the thing serving HTTPS, doesn't really care about the validity of the certificate. It's up to the client to decide if it should trust the certificate the server gives.
The server has the certificate and the corresponding private key. The server can validate that the public key and the private key belong together. How this is done depends on the key type, but the gist of it is, if you know the private key, you can fully reconstruct the public key from it. The server will validate that the private key's "public parts" match the certificate's public key.
The server will present the certificate to the client, like a browser. The browser will check many things, but at minimum it will check two important things:
Can the browser build a chain back to a certificate in the trust store. So the browser will look at the signer of the certificate, called the Issuer, and check if the issuer is in it's trust store. If yes, then the certificate is trusted. If no, it will see if that certificate has an Issuer, and loop all the way down the chain until it reaches the end of the chain, or when it finds something in the trust store. If it reaches the end, then the certificate was not issued by anyone it trusts.
That the certificate is valid for that domain. The certificate contains a Subject Alternative Name (SAN) which indicates what domains the certificate is valid for.
There are lots of other things that get checked, like expiration, revocation, Certificate Transparency, "strength" of the certificate - too many to list.
How does the client determine that my public cert was signed by a trusted CA.
Every client has its own trust store. Internet Explorer on Windows uses the Windows trust store, Google Chrome on macOS and Windows use the operating system trust store (Keychain, Windows Trust Store, etc).
The browser / client needs to build a trust path, as described above. How it does this is a bit complex, but it works out to the Authority Key ID attribute - if it exists, and the Issuer property of the certificate. It uses these values to find the certificate that issued it.
Once it finds the issuing certificate, it checks the signature on the certificate with the issuer's certificate public key.
That issuing certificate could be in the "root" trust store, and uses the public key in the issuing certificate to verify the signed certificate.
Your web server might (probably) send intermediate certificates along with the "main" (leaf) certificate. The client might use these intermediates to build a path back to a certificate that is in the root trust store. Intermediates can only be used as intermediates - you can't send an additional certificate saying "here's the root certificate, trust it".
can a client check if my x509 certificate is valid by just having an x509 list of certificates of trusted CAs?
Yes. That is the trust store. Every browser has / uses one. Firefox has their own trust store called NSS. Operating systems typically have their own trust store, too.
I've read about SSL protocol and now, I know how it encrypts data. But there is something I couldn't understand. With SSL , you're sure you're sending data to and getting data from correct server. But how?
I mean if I create a fake certificate and send it for requests of special website, how do browsers ( or other programs) detect the fake certificate?
Edit: I didn't mean to create a self-signed certificate. I meant how can someone validate my certificate if I create a certificate that its issuer and subject ,etc are something to real certificate! (the only things that are not real is Public key & signature)
TL;DR summary:
Validity of a server certificate is established by:
Host name verification
Verifying the signatures of the entire certificate chain
Performing additional checks on meta data for each certificate
Checking the revocation status of each of the certificates involved
Checking whether the self-signed root certificate of the chain is among the certificates that one trusts by default
Explanation
Let's assume you want to connect to https://mail.google.com (you can try this out in your browser!).
The (real) server will respond with a certificate that is issued to mail.google.com, i.e. in the 'Subject' field of the certificate you will find the Common Name (CN) 'mail.google.com' - cf. RFC 5280 for details on the fields of certificates. The fact that the subject is linked to the site URL is very important for the security of the whole model, and it is actively checked by your TLS implementation ("host name verification"), because otherwise there would be room for Man-In-The-Middle attacks. I.e. somebody could acquire an otherwise valid certificate and impersonate mail.google.com without you taking any notice of it.
In addition to the host name verification, your TLS implementation will also check the "validity" of the certificate. The whole procedure is rather complex and does include checking the trustworthiness of the certificate, but additionally a lot of other things will be checked, more on that in a minute.
If you view Google Mail's certificate in your browser, you will notice that there are actually three certificates shown:
mail.google.com
Thawte SGC CA
Class 3 Public Primary Certification Authority (VeriSign)
The model is that there are a few (well, unfortunately not so few anymore) trusted root certificate authorities ("root CAs") that either you could choose on your own or (more likely) that come preconfigued with your software (e.g. browser) that are blindly trusted. These trusted authorities form the anchors of the entire trust model of "PKI" (Public Key Infrastructure). The basic idea is that the trusted entities may issue certificates to other authorities and grant them permission to again issue certificates (these authorities are called intermediate certificate authorities). The intermediate CAs may again recursively apply this procedure up to a certain point, the number of intermediate CAs between an actual end entity certificate and a root CA certificate is generally limited.
At one point, an intermediate CA will issue certificates to an "end entity" ("mail.google.com" in our example). Now the process of issuing a certificate actually means that the party requesting a certificate will create a public/private key pair first, and use them to authenticate a certificate request that is sent to the certificate authority. The issuing authority creates a certificate for the subordinate entity (either intermediate CA or end entity) by "signing" that certificate using its own private key using an asymmetric algorithm such as RSA and by additionally including the public key of the requesting party within the newly generated certificate. The root CA possesses a so called self-signed certificate, i.e. the root CA is the only authority that may sign their own certificate and include their own public key. The private key remains hidden at all times, of course.
The recursive nature of the certificate issuing process implies that for each end entity certificate there is a unique way of establishing a "chain" of certificates that leads up to a root certificate authority. Now when you are presented with an end entity certificate while trying to connect to a TLS-secured site, the following procedure will be applied recursively until you end up with a root CA certificate:
Find the certificate of the authority that issued the certificate to be validated (see RFC 5280 for details). If none is found: exit with error.
Take the public key of the issuing certificate and verify the signature of the to-be-validated certificate using this public key.
Check a lot of additional things such as whether the certificate has neither expired nor is it not valid yet, "policy constraints", "key usages", "extended key usages"... (again, the gory details are in the RFC).
Certificate revocation status (more on that later)
If all checks were positive, you will ultimately end up with a certificate being self-signed, i.e. where the subject is also the issuer (such as the VeriSign certificate in our example). Now the last thing you have to verify is whether this certificate is among those that you blindly trust: if it is, all is well and the connection will succeed, if it is not, the connection attempt will be rejected.
As if this were not complicated enough already, the checks described so far do not handle cases where once valid certificates suddenly become rogue, examples being cases where a certificate is stolen or private keys are compromised (think of Comodo and DigiNotar). In these cases, the normal procedure is to "revoke" those certificates gone bad, that is you want to mark them as being invalid starting from a distinct point in time (they will expire at some point anyway, but for the remainder of that period they shall already be marked as invalid). For these cases, CAs have the possibility to issue CRLs (a catalog of certificates declared as invalid) or OCSP responses (information for one or in rare cases a set of certificates) that provides clients with information whether a given certificate has been marked as invalid or not. The revocation status needs to be checked for all certificates in a chain, should one of them be marked as invalid then the end entity certificate cannot be trusted and the connection must be rejected as well.
SSL certificates are signed by a certificate authority (CA), which is someone the user already trusts (or more likely, the people who designed their operating system trusts).
The CA digitally signs the certificate using public key encryption. The basic explanation is that the CA has a "private key", and a "public key" that everyone knows. Via some math I don't understand, the CA can create a signature using its private key which can easily be verified with its public key (but the public key can't be used to create a new signature).
When you get an SSL certificate from a server, you get the server's public key, and a signature from a CA saying that it's valid (along with some other info). If you know and trust that CA, you can check the signature and determine if it's valid. You can also use a certificate revocation list to make sure it wasn't revoked.
So basically, you can recognize a bad SSL certificate because it isn't signed by a certificate authority that you trust.
Any fake certificate you create will be a self-signed certificate.
The browser will display big scary warnings when connecting to a site with a self-signed certificate which the user will promptly ignore.
In order to avoid warnings, you need a certificate signed by a certificate authority that the browser trusts, such as VeriSign.
These companies will hopefully make sure that you actually own the domain for the certificate they're signing.
Re: Edit: You can only create a non-self-signed certificate if you get it signed from a trusted CA.
They will refuse to sign a certificate for a different subject.
Process from my understanding:
server sends servers public key
server sends certificate (all information encrypted by trusted CA with their private key)
Your PC decrypts certificate with public key (built into OS from trusted CA)
Your PC hashes (with sha1 and sha256) the servers public key
Your PC compares the hashes of servers public key with certificate stored hash, if not same browser will block site
Your PC compares allowed domains from certificate and the domain, if not allowed, if not same browser will block site
Your PC compares valid date from certificate and your date, if not valid browser will block site.
To fake this you would either need to:
obtain a CA private key (extremely hard to get),
be a CA,
be part of the 5 eyes (Government intelligence agency alliance) and ask a CA for their private key
So if you see a padlock in the address bar you are almost always safe.
Certificates work because they follow a chain of trust. Certificates have a chain of one or more issuers that are trusted; this chain is the backbone of why it works at all. Browsers and nearly all SSL certificate libraries do this chain check, or at least provide the option to.
Self-signed certificates (or those issued by chains that end in a self-signed certificate) would fail this check.
I have a question regarding validation of digital signatures using a self-signed certificate:
The following tutorial works for me:
http://www.oracle.com/technetwork/articles/javase/dig-signature-api-140772.html
However, when a X.509 certificate is self-signed, how can a receiver trust certificate data attached to an XML message? Any one can generate a self-signed cert and claim to be the same sender. The validation in the above tutorial always returns true. Sender’s cert must be loaded to receiver’s truststore, so receiver can use whatever in the truststore to validate signed doc. I cannot find any reference for such a scenario.
Your understanding is correct - with self-signed certificates anyone can create a certificate and signature validation will be ok. The reason is that signature validation performs first of all cryptographic operation, which is completed successfully. The second step is to validate the certificate itself AND also it's origins. When the CA-signed certificate is used, the certificate is validated using CA certificate(s) up to trusted CA (or known root CA). With self-signed certificate validation is not possible. In the above tutorial the procedure of certificate validation was skipped for simplicity as it's quite complex and beyond the scope of tutorial.
The problem you're describing is usually addressed by Public Key Infrastructures (PKI).
This is the traditional model for verifying certificates for HTTPS sites, for example. It starts with a set of trusted Certification Authorities (CAs) from which you import the CA certificates as "trusted". The entity certificates that you get are then verified against this set of trusted anchors by building a certification path between the certificate to verify and a CA certificate you know (linking the certificate to a trusted issuer, perhaps via intermediate CA certificates).
The various rules to do this are described in RFC 5280. The PKI system doesn't apply only to web servers, but to any entity (there are additional rules for web servers to verify that they're the one you want to talk to, on top of having a valid certificate).
(In particular because the choice of which CA certificates to trust is often done on behalf of the user, at least by default, by the OS or browser vendor, this model isn't perfect, but it's the most common in use.)
Alternatively, there's nothing wrong with establishing a list of self-signed certificates you would trust in advance.
Either way, you need to pre-set what you trust by mechanisms out of bands (e.g. by meeting someone you trust and using the certificate they give you in person).
This PKI model goes hand-in-hand with the X.509 format thanks to the notion of Issuer DN and Subject DN. You could have other models, for example relying on PGP certificates, where you would build a web of trust; you would still need an initial set of trusted anchors.
For XML-DSig in Java, you should implement a X509KeySelector that only returns a key that you trust. In a simple scenario, where you have a pre-defined set of self-signed certificates you trust, you can iterate over a keystore containing those trusted certificates. Otherwise, use the Java PKI Programmer Guide (as linked from the tutorial you've used).
I'm attempting to enable SSL communication from a web service client (Axis2) using the certificate on the user's CAC card. Works like a charm....UNTIL the web server is CAC enabled. At that point the SSL connection is rejected with the error message that the other certificates in the chain were not included.
I have ensured that the provider is available, either by adding it to the security.properties file or creating it programatically.
My current approach is to simply set the system properties:
System.setProperty("javax.net.ssl.keyStore", "NONE");
System.setProperty("javax.net.ssl.keyStoreType", "PKCS11");
I understand from this question/answer that this approach only sends the "end entity" certificate. Apparently I need to implement my own X509KeyManager. This is new ground for me, can anyone suggest a good reference or provide samples of how to do so?
Appreciate the assistance.
The best key manager implementation depends on the issuer of the certificates you expect to be using.
If the certificate on the user's CAC will always be issued by a specific CA, simply store that issuer's certificate and any intermediate certificates further up the chain in a PKCS #7 file. In the getCertificateChain() method, this collection can be appended blindly to the user's certificate and returned.
If things aren't quite that simple, but a complete list of possible issuers can be enumerated, obtain all of their certificates, and their issuer's certificates, and so on, up to the root certificates.
Add all of the root certificates to a key store as trusted entries. Bundle the intermediate certificates in a PKCS-#7–format file.
Implement X509KeyManager (or extend X509ExtendedKeyManager if you're working with SSLEngine). Specifically, in the getCertificateChain() method, you'll use a CertPathBuilder to create a valid chain from the user's certificate to a trusted root. The target is the certificate that you load from the user's CAC with the alias parameter. The trusted roots are the certificates in trust store that you created; the intermediate certificates can be loaded from the PKCS #7 file and added to the builder parameters. Once the chain is built, get the certificate path and convert it to an array. This is the result of the getCertificateChain() method.
If you can't predict who will be issuing the user's certificate, you might be able to obtain the intermediate certificates at runtime from an LDAP directory or other repository. That's a whole new level of difficulty.