I'm using softHSM (FWIW with a go library https://github.com/ThalesIgnite/crypto11, documentation here https://pkg.go.dev/github.com/ThalesIgnite/crypto11?tab=doc).
My goal is to store a 'master key' (AES256) for encrypting objects similarly to how AWS S3 does into the HSM device (because it's more secure). From that key, just derive any other key that I need to encrypt my objects (or decrypt them).
I'm failing at understanding how a generated secret key in a HSM can later be retrieved by the same software program. I see that the API mentions of a context..
rephrased: when I generate a secret key in the HSM like this:
func TestFindingAllKeys(t *testing.T) {
withContext(t, func(ctx *Context) {
for i := 0; i < 10; i++ {
id := randomBytes()
key, err := ctx.GenerateSecretKey(id, 128, CipherAES)
require.NoError(t, err)
defer func(k *SecretKey) { _ = k.Delete() }(key)
}
keys, err := ctx.FindAllKeys()
require.NoError(t, err)
require.NotNil(t, keys)
require.Len(t, keys, 10)
})
}
how do I 'associate' one of those secret keys with my program data (e.g. a S3 bucket or customer)?
How do I retrieve that same secret key again (even if I can't dump it out of the HSM) to decrypt the data at a later time?
I'm missing this apparently stupid, but crucial connection: how does one retrieve a previously generated secret key again at a later time?
You can use pkcs#11 token labels, or equivalent to tag symmetric keys.
You could also use the slot concept, keeping a local database mapping users/customers to keys.
For asymmetric primitives you can export the public key and map this object to a customer/user.
I recently shifted from using Composer to writing ChainCode in Go. In Composer, using ACL, I could restrict certain transactions to specific participant types.
I'm attempting to build a multi-org network where user 'types' are defined as structs in Go -- Client Agent and Manager
I want the Client to have access to certain transactions, Agent to have access to a different set of transactions, and Manager to have access to all of Agent's and some more transactions.
How do I implement this using Fabric Go chaincode? Appreciate any help!
Thank you
OK so lets assume Alice is an Agent. There is a function onlyAgent() that you want to ensure can be called only by Alice. It will be something like this
func (t *SimpleChaincode) createParticipant (stub shim.ChaincodeStubInterface, args []string) pb.Response {
username := args[0]; // This would be Alice
type := args[1]; // This should be Agent
user := &marble{type , username }
userAsJsonBues, err := json.Marshal(user )
err = stub.PutState(marbleName, userAsJsonBues);
return shim.Success(nil);
}
func (t *SimpleChaincode) onlyAgent(stub shim.ChaincodeStubInterface, args []string) pb.Response {
username := args[0]; // Expects to be Alice
aliceAsBytes, err := stub.GetState(username)
alice:= User{}
err = json.Unmarshal(aliceAsBytes, &alice)
// alice.user should return Agent. Perform whatever checks you want
}
This should give you a rough idea on how to proceed, couple of things to remember
This example requires the name Alice to be passed as parameter in onlyAgent. I did it for demonstrative purposes, technically you would want to extract Alice's certificate and then query Alice directly from that (I can do this in nodejs chaincode but can't seem to find the exact API call in go)
I am creating an app using Go and I am trying to start a https server using the ListenAndServeTLS function. Here is my code:
func StartServer() {
defer config.CapturePanic()
c := config.GetInstance()
serverAddress := fmt.Sprintf(":%s", c.GetConfig().ServerPort)
server := http.Server{Addr: serverAddress}
log.Info("Starting local server")
http.HandleFunc("/", login.Handler)
http.HandleFunc("/login", login.Handler)
http.HandleFunc("/settings", settings.Handler)
//cert, _ := data.Asset("my-cert.pem")
//key, _ := data.Asset("my-key.pem")
err := server.ListenAndServeTLS("my-cert.crt", "my-cert.key")
if err != nil {
log.WithError(err).Fatal("Error stopping local server")
}
}
The thing is that I would like to embed my certificate and its key inside my executable file and then pass them to the the server.ListeAndServeTLS function as a string or a byte array. However this function does not take these types of arguments. Is there another way to do this?
Note: I am aware that it is a bad practice to embed a private key inside a client application, however what I am trying to do here is just to create a config webpage that will be hosted as https://localhost:8080.
You can build your own server object and still call ListenAndServeTLS. Since your tls config has certificates, it will ignore the passed-in filenames.
I'm omitting the return on error for conciseness, please do not:
// Generate a key pair from your pem-encoded cert and key ([]byte).
cert, err := tls.X509KeyPair(<cert contents>, <key contents>)
// Construct a tls.config
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{cert}
// Other options
}
// Build a server:
server := http.Server{
// Other options
TLSConfig: tlsConfig,
}
// Finally: serve.
err = server.ListenAndServeTLS("", "")
In my case, I was unable to load files from disk, but did have the certificates passed in to the environment as variables ([]byte).
Adding on to Marc's answer, I had to only change the top line.
// Append signed leaf certificate and intermediate to a single
certChain := append(leaf.PublicBytes, intermediate.PublicBytes...)
// Generate a key pair from your pem-encoded cert and key ([]byte).
cert, err := tls.X509KeyPair(certChain, leaf.PrivateBytes)
...
I have some experience using Go, but now I don't really understand the complexity in security of what I am doing, so I need to ask.
I am creating an RSA private key, converting to PEM and then encryping it with a passphrase.
So, how secure is to store it in a public place?
I'm not looking for answers like "it's ok, just change the passphrase over time", I really want to know which mechanism of cypher Golang is using to do it and if is safe to leave the encrypted PEM in, for example, a public blockchain and why I can do it or why I cannot.
I'm leaving here the code I am using right now:
func New(passphrase string)(*pem.Block, error){
pk, err := createPrivateKey(2048)
if err != nil {
return false, err
}
pem := getPemFromPK(pk)
block, err := EncryptPEMBlock(pem,passphrase)
if err != nil {
return false, err
}
return block,nil
}
func createPrivateKey(bits int) (*rsa.PrivateKey, error){
pk, err := rsa.GenerateKey(rand.Reader, bits)
if err != nil {
return nil, err
}
return pk,nil
}
func getPemFromPK(pk *rsa.PrivateKey) (*pem.Block){
block := &pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(pk),
}
return block
}
func EncryptPEMBlock(block *pem.Block, passphrase string) (*pem.Block, error){
block, err := x509.EncryptPEMBlock(rand.Reader, block.Type, block.Bytes, []byte(passphrase), x509.PEMCipherAES256)
if err != nil {
return nil, err
}
return block,nil
}
Thank you very much.
Edit:
As an answer here and other forums, it is not recommended to publish in public any type of private key, even if encrypted.
This topic is answered.
You are making a mistake in your thinking about what a private key is and what a passphrase is. The passphrase is used to encrypt and unencrypt your private key - if you are storing a key file which needs a passphrase to be used, then that file contains your encrypted key.
If you store the "private key" as you say, it sounds like you wish to publicly store the unencrypted key. However, even if you publish an encrypted private key on a public online repository, there's many ways to crack a passphrase. If the passphrase is short or unsecure in other ways, the attacker now has your private key. If they target you and gain access to a machine of yours that has used this key in an application (i.e. bash), then they can just access bash history log to find the passphrase.
Sometimes actually, it's trivial to keylog someone in a targeted attack.
There are many many things that can go wrong if you store an unencrypted private key online.
In my build process, I want to include a timestamp from an RFC-3161-compliant TSA. At run time, the code will verify this timestamp, preferably without the assistance of a third-party library. (This is a .NET application, so I have standard hash and asymmetric cryptography functionality readily at my disposal.)
RFC 3161, with its reliance on ASN.1 and X.690 and whatnot, is not simple to implement, so for now at least, I'm using Bouncy Castle to generate the TimeStampReq (request) and parse the TimeStampResp (response). I just can't quite figure out how to validate the response.
So far, I've figured out how to extract the signature itself, the public cert, the time the timestamp was created, and the message imprint digest and nonce that I sent (for build-time validation). What I can't figure out is how to put this data together to generate the data that was hashed and signed.
Here's a rough idea of what I'm doing and what I'm trying to do. This is test code, so I've taken some shortcuts. I'll have to clean a couple of things up and do them the right way once I get something that works.
Timestamp generation at build time:
// a lot of fully-qualified type names here to make sure it's clear what I'm using
static void WriteTimestampToBuild(){
var dataToTimestamp = Encoding.UTF8.GetBytes("The rain in Spain falls mainly on the plain");
var hashToTimestamp = new System.Security.Cryptography.SHA1Cng().ComputeHash(dataToTimestamp);
var nonce = GetRandomNonce();
var tsr = GetTimestamp(hashToTimestamp, nonce, "http://some.rfc3161-compliant.server");
var tst = tsr.TimeStampToken;
var tsi = tst.TimeStampInfo;
ValidateNonceAndHash(tsi, hashToTimestamp, nonce);
var cms = tst.ToCmsSignedData();
var signer =
cms.GetSignerInfos().GetSigners()
.Cast<Org.BouncyCastle.Cms.SignerInformation>().First();
// TODO: handle multiple signers?
var signature = signer.GetSignature();
var cert =
tst.GetCertificates("Collection").GetMatches(signer.SignerID)
.Cast<Org.BouncyCastle.X509.X509Certificate>().First();
// TODO: handle multiple certs (for one or multiple signers)?
ValidateCert(cert);
var timeString = tsi.TstInfo.GenTime.TimeString;
var time = tsi.GenTime; // not sure which is more useful
// TODO: Do I care about tsi.TstInfo.Accuracy or tsi.GenTimeAccuracy?
var serialNumber = tsi.SerialNumber.ToByteArray(); // do I care?
WriteToBuild(cert.GetEncoded(), signature, timeString/*or time*/, serialNumber);
// TODO: Do I need to store any more values?
}
static Org.BouncyCastle.Math.BigInteger GetRandomNonce(){
var rng = System.Security.Cryptography.RandomNumberGenerator.Create();
var bytes = new byte[10]; // TODO: make it a random length within a range
rng.GetBytes(bytes);
return new Org.BouncyCastle.Math.BigInteger(bytes);
}
static Org.BouncyCastle.Tsp.TimeStampResponse GetTimestamp(byte[] hash, Org.BouncyCastle.Math.BigInteger nonce, string url){
var reqgen = new Org.BouncyCastle.Tsp.TimeStampRequestGenerator();
reqgen.SetCertReq(true);
var tsrequest = reqgen.Generate(Org.BouncyCastle.Tsp.TspAlgorithms.Sha1, hash, nonce);
var data = tsrequest.GetEncoded();
var webreq = WebRequest.CreateHttp(url);
webreq.Method = "POST";
webreq.ContentType = "application/timestamp-query";
webreq.ContentLength = data.Length;
using(var reqStream = webreq.GetRequestStream())
reqStream.Write(data, 0, data.Length);
using(var respStream = webreq.GetResponse().GetResponseStream())
return new Org.BouncyCastle.Tsp.TimeStampResponse(respStream);
}
static void ValidateNonceAndHash(Org.BouncyCastle.Tsp.TimeStampTokenInfo tsi, byte[] hashToTimestamp, Org.BouncyCastle.Math.BigInteger nonce){
if(tsi.Nonce != nonce)
throw new Exception("Nonce doesn't match. Man-in-the-middle attack?");
var messageImprintDigest = tsi.GetMessageImprintDigest();
var hashMismatch =
messageImprintDigest.Length != hashToTimestamp.Length ||
Enumerable.Range(0, messageImprintDigest.Length).Any(i=>
messageImprintDigest[i] != hashToTimestamp[i]
);
if(hashMismatch)
throw new Exception("Message imprint doesn't match. Man-in-the-middle attack?");
}
static void ValidateCert(Org.BouncyCastle.X509.X509Certificate cert){
// not shown, but basic X509Chain validation; throw exception on failure
// TODO: Validate certificate subject and policy
}
static void WriteToBuild(byte[] cert, byte[] signature, string time/*or DateTime time*/, byte[] serialNumber){
// not shown
}
Timestamp verification at run time (client site):
// a lot of fully-qualified type names here to make sure it's clear what I'm using
static void VerifyTimestamp(){
var timestampedData = Encoding.UTF8.GetBytes("The rain in Spain falls mainly on the plain");
var timestampedHash = new System.Security.Cryptography.SHA1Cng().ComputeHash(timestampedData);
byte[] certContents;
byte[] signature;
string time; // or DateTime time
byte[] serialNumber;
GetDataStoredDuringBuild(out certContents, out signature, out time, out serialNumber);
var cert = new System.Security.Cryptography.X509Certificates.X509Certificate2(certContents);
ValidateCert(cert);
var signedData = MagicallyCombineThisStuff(timestampedHash, time, serialNumber);
// TODO: What other stuff do I need to magically combine?
VerifySignature(signedData, signature, cert);
// not shown: Use time from timestamp to validate cert for other signed data
}
static void GetDataStoredDuringBuild(out byte[] certContents, out byte[] signature, out string/*or DateTime*/ time, out byte[] serialNumber){
// not shown
}
static void ValidateCert(System.Security.Cryptography.X509Certificates.X509Certificate2 cert){
// not shown, but basic X509Chain validation; throw exception on failure
}
static byte[] MagicallyCombineThisStuff(byte[] timestampedhash, string/*or DateTime*/ time, byte[] serialNumber){
// HELP!
}
static void VerifySignature(byte[] signedData, byte[] signature, System.Security.Cryptography.X509Certificates.X509Certificate2 cert){
var key = (RSACryptoServiceProvider)cert.PublicKey.Key;
// TODO: Handle DSA keys, too
var okay = key.VerifyData(signedData, CryptoConfig.MapNameToOID("SHA1"), signature);
// TODO: Make sure to use the same hash algorithm as the TSA
if(!okay)
throw new Exception("Timestamp doesn't match! Don't trust this!");
}
As you might guess, where I think I'm stuck is the MagicallyCombineThisStuff function.
I finally figured it out myself. It should come as no surprise, but the answer is nauseatingly complex and indirect.
The missing pieces to the puzzle were in RFC 5652. I didn't really understand the TimeStampResp structure until I read (well, skimmed through) that document.
Let me describe in brief the TimeStampReq and TimeStampResp structures. The interesting fields of the request are:
a "message imprint", which is the hash of the data to be timestamped
the OID of the hash algorithm used to create the message imprint
an optional "nonce", which is a client-chosen identifier used to verify that the response is generated specifically for this request. This is effectively just a salt, used to avoid replay attacks and to detect errors.
The meat of the response is a CMS SignedData structure. Among the fields in this structure are:
the certificate(s) used to sign the response
an EncapsulatedContentInfo member containing a TSTInfo structure. This structure, importantly, contains:
the message imprint that was sent in the request
the nonce that was sent in the request
the time certified by the TSA
a set of SignerInfo structures, with typically just one structure in the set. For each SignerInfo, the interesting fields within the structure are:
a sequence of "signed attributes". The DER-encoded BLOB of this sequence is what is actually signed. Among these attributes are:
the time certified by the TSA (again)
a hash of the DER-encoded BLOB of the TSTInfo structure
an issuer and serial number or subject key identifier that identifies the signer's certificate from the set of certificates found in the SignedData structure
the signature itself
The basic process of validating the timestamp is as follows:
Read the data that was timestamped, and recompute the message imprint using the same hashing algorithm used in the timestamp request.
Read the nonce used in the timestamp request, which must be stored along with the timestamp for this purpose.
Read and parse the TimeStampResp structure.
Verify that the TSTInfo structure contains the correct message imprint and nonce.
From the TimeStampResp, read the certificate(s).
For each SignerInfo:
Find the certificate for that signer (there should be exactly one).
Verify the certificate.
Using that certificate, verify the signer's signature.
Verify that the signed attributes contain the correct hash of the TSTInfo structure
If everything is okay, then we know that all signed attributes are valid, since they're signed, and since those attributes contain a hash of the TSTInfo structure, then we know that's okay, too. We have therefore validated that the timestamped data is unchanged since the time given by the TSA.
Because the signed data is a DER-encoded BLOB (which contains a hash of the different DER-encoded BLOB containing the information the verifier actually cares about), there's no getting around having some sort of library on the client (verifier) that understands X.690 encoding and ASN.1 types. Therefore, I conceded to including Bouncy Castle in the client as well as in the build process, since there's no way I have time to implement those standards myself.
My code to add and verify timestamps is similar to the following:
Timestamp generation at build time:
// a lot of fully-qualified type names here to make sure it's clear what I'm using
static void WriteTimestampToBuild(){
var dataToTimestamp = ... // see OP
var hashToTimestamp = ... // see OP
var nonce = ... // see OP
var tsq = GetTimestampRequest(hashToTimestamp, nonce);
var tsr = GetTimestampResponse(tsq, "http://some.rfc3161-compliant.server");
ValidateTimestamp(tsq, tsr);
WriteToBuild("tsq-hashalg", Encoding.UTF8.GetBytes("SHA1"));
WriteToBuild("nonce", nonce.ToByteArray());
WriteToBuild("timestamp", tsr.GetEncoded());
}
static Org.BouncyCastle.Tsp.TimeStampRequest GetTimestampRequest(byte[] hash, Org.BouncyCastle.Math.BigInteger nonce){
var reqgen = new TimeStampRequestGenerator();
reqgen.SetCertReq(true);
return reqgen.Generate(TspAlgorithms.Sha1/*assumption*/, hash, nonce);
}
static void GetTimestampResponse(Org.BouncyCastle.Tsp.TimeStampRequest tsq, string url){
// similar to OP
}
static void ValidateTimestamp(Org.BouncyCastle.Tsp.TimeStampRequest tsq, Org.BouncyCastle.Tsp.TimeStampResponse tsr){
// same as client code, see below
}
static void WriteToBuild(string key, byte[] value){
// not shown
}
Timestamp verification at run time (client site):
/* Just like in the OP, I've used fully-qualified names here to avoid confusion.
* In my real code, I'm not doing that, for readability's sake.
*/
static DateTime GetTimestamp(){
var timestampedData = ReadFromBuild("timestamped-data");
var hashAlg = Encoding.UTF8.GetString(ReadFromBuild("tsq-hashalg"));
var timestampedHash = System.Security.Cryptography.HashAlgorithm.Create(hashAlg).ComputeHash(timestampedData);
var nonce = new Org.BouncyCastle.Math.BigInteger(ReadFromBuild("nonce"));
var tsq = new Org.BouncyCastle.Tsp.TimeStampRequestGenerator().Generate(System.Security.Cryptography.CryptoConfig.MapNameToOID(hashAlg), timestampedHash, nonce);
var tsr = new Org.BouncyCastle.Tsp.TimeStampResponse(ReadFromBuild("timestamp"));
ValidateTimestamp(tsq, tsr);
// if we got here, the timestamp is okay, so we can trust the time it alleges
return tsr.TimeStampToken.TimeStampInfo.GenTime;
}
static void ValidateTimestamp(Org.BouncyCastle.Tsp.TimeStampRequest tsq, Org.BouncyCastle.Tsp.TimeStampResponse tsr){
/* This compares the nonce and message imprint and whatnot in the TSTInfo.
* It throws an exception if they don't match. This doesn't validate the
* certs or signatures, though. We still have to do that in order to trust
* this data.
*/
tsr.Validate(tsq);
var tst = tsr.TimeStampToken;
var timestamp = tst.TimeStampInfo.GenTime;
var signers = tst.ToCmsSignedData().GetSignerInfos().GetSigners().Cast<Org.BouncyCastle.Cms.SignerInformation>();
var certs = tst.GetCertificates("Collection");
foreach(var signer in signers){
var signerCerts = certs.GetMatches(signer.SignerID).Cast<Org.BouncyCastle.X509.X509Certificate>().ToList();
if(signerCerts.Count != 1)
throw new Exception("Expected exactly one certificate for each signer in the timestamp");
if(!signerCerts[0].IsValid(timestamp)){
/* IsValid only checks whether the given time is within the certificate's
* validity period. It doesn't verify that it's a valid certificate or
* that it hasn't been revoked. It would probably be better to do that
* kind of thing, just like I'm doing for the signing certificate itself.
* What's more, I'm not sure it's a good idea to trust the timestamp given
* by the TSA to verify the validity of the TSA's certificate. If the
* TSA's certificate is compromised, then an unauthorized third party could
* generate a TimeStampResp with any timestamp they wanted. But this is a
* chicken-and-egg scenario that my brain is now too tired to keep thinking
* about.
*/
throw new Exception("The timestamp authority's certificate is expired or not yet valid.");
}
if(!signer.Verify(signerCerts[0])){ // might throw an exception, might not ... depends on what's wrong
/* I'm pretty sure that signer.Verify verifies the signature and that the
* signed attributes contains a hash of the TSTInfo. It also does some
* stuff that I didn't identify in my list above.
* Some verification errors cause it to throw an exception, some just
* cause it to return false. If it throws an exception, that's great,
* because that's what I'm counting on. If it returns false, let's
* throw an exception of our own.
*/
throw new Exception("Invalid signature");
}
}
}
static byte[] ReadFromBuild(string key){
// not shown
}
I am not sure to understand why you want to rebuild the data structure signed in the response. Actually if you want to extract the signed data from the time-stamp server response you can do this:
var tsr = GetTimestamp(hashToTimestamp, nonce, "http://some.rfc3161-compliant.server");
var tst = tsr.TimeStampToken;
var tsi = tst.TimeStampInfo;
var signature = // Get the signature
var certificate = // Get the signer certificate
var signedData = tsi.GetEncoded(); // Similar to tsi.TstInfo.GetEncoded();
VerifySignature(signedData, signature, certificate)
If you want to rebuild the data structure, you need to create a new Org.BouncyCastle.Asn1.Tsp.TstInfo instance (tsi.TstInfo is a Org.BouncyCastle.Asn1.Tsp.TstInfo object) with all elements contained in the response.
In RFC 3161 the signed data structure is defined as this ASN.1 sequence:
TSTInfo ::= SEQUENCE {
version INTEGER { v1(1) },
policy TSAPolicyId,
messageImprint MessageImprint,
-- MUST have the same value as the similar field in
-- TimeStampReq
serialNumber INTEGER,
-- Time-Stamping users MUST be ready to accommodate integers
-- up to 160 bits.
genTime GeneralizedTime,
accuracy Accuracy OPTIONAL,
ordering BOOLEAN DEFAULT FALSE,
nonce INTEGER OPTIONAL,
-- MUST be present if the similar field was present
-- in TimeStampReq. In that case it MUST have the same value.
tsa [0] GeneralName OPTIONAL,
extensions [1] IMPLICIT Extensions OPTIONAL }
Congratulations on getting that tricky protocol work done!
See also a Python client implementation at rfc3161ng 2.0.4.
Note that with the RFC 3161 TSP protocol, as discussed at Web Science and Digital Libraries Research Group: 2017-04-20: Trusted Timestamping of Mementos and other publications, you and your relying parties must trust that the Time-Stamping Authority (TSA) is operated properly and securely. It is of course very difficult, if not impossible, to really secure online servers like those run by most TSAs.
As also discussed in that paper, with comparisons to TSP, now that the world has a variety of public blockchains in which trust is distributed and (sometimes) carefully monitored, there are new trusted timestamping options (providing "proof of existence" for documents). For example see
OriginStamp - Trusted Timestamping with Bitcoin. The protocol is much much simpler, and they provide client code for a large variety of languages. While their online server could also be compromised, the client can check whether their hashes were properly embedded in the Bitcoin blockchain and thus bypass the need to trust the OriginStamp service itself.
One downside is that timestamps are only posted once a day, unless an extra payment is made. Bitcoin transactions have become rather expensive, so the service is looking at supporting other blockchains also to drive costs back down and make it cheaper to get more timely postings.
Update: check out Stellar and Keybase
For free, efficient, lightning-fast, widely-vetted timestamps, check out the Stellar blockchain protocol, and the STELLARAPI.IO service.