I have some experience using Go, but now I don't really understand the complexity in security of what I am doing, so I need to ask.
I am creating an RSA private key, converting to PEM and then encryping it with a passphrase.
So, how secure is to store it in a public place?
I'm not looking for answers like "it's ok, just change the passphrase over time", I really want to know which mechanism of cypher Golang is using to do it and if is safe to leave the encrypted PEM in, for example, a public blockchain and why I can do it or why I cannot.
I'm leaving here the code I am using right now:
func New(passphrase string)(*pem.Block, error){
pk, err := createPrivateKey(2048)
if err != nil {
return false, err
}
pem := getPemFromPK(pk)
block, err := EncryptPEMBlock(pem,passphrase)
if err != nil {
return false, err
}
return block,nil
}
func createPrivateKey(bits int) (*rsa.PrivateKey, error){
pk, err := rsa.GenerateKey(rand.Reader, bits)
if err != nil {
return nil, err
}
return pk,nil
}
func getPemFromPK(pk *rsa.PrivateKey) (*pem.Block){
block := &pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(pk),
}
return block
}
func EncryptPEMBlock(block *pem.Block, passphrase string) (*pem.Block, error){
block, err := x509.EncryptPEMBlock(rand.Reader, block.Type, block.Bytes, []byte(passphrase), x509.PEMCipherAES256)
if err != nil {
return nil, err
}
return block,nil
}
Thank you very much.
Edit:
As an answer here and other forums, it is not recommended to publish in public any type of private key, even if encrypted.
This topic is answered.
You are making a mistake in your thinking about what a private key is and what a passphrase is. The passphrase is used to encrypt and unencrypt your private key - if you are storing a key file which needs a passphrase to be used, then that file contains your encrypted key.
If you store the "private key" as you say, it sounds like you wish to publicly store the unencrypted key. However, even if you publish an encrypted private key on a public online repository, there's many ways to crack a passphrase. If the passphrase is short or unsecure in other ways, the attacker now has your private key. If they target you and gain access to a machine of yours that has used this key in an application (i.e. bash), then they can just access bash history log to find the passphrase.
Sometimes actually, it's trivial to keylog someone in a targeted attack.
There are many many things that can go wrong if you store an unencrypted private key online.
Related
I could use the Azure Key Vault provider for Secrets Store CSI driver (https://learn.microsoft.com/en-us/azure/aks/csi-secrets-store-driver)
but I need to have a bit more control over the certificates via a custom k8 management API am developing for my use case. I had some challenges with this.
I think I have a good enough solution but want to put it out in the world for comment from others that may have done this in the past.
here is what I did...
When one creates the certificates (or uploads them) in Azure vault the values are stored in secrets. I used the Azure sdk for Go to get them
I use base64 package to decode what Azure sends me (a PFX containing private and public keys
I use pkcs12 package pkcs12.toPEM function to break it apart into an array of pem.Block.
Each block has a value that contains headers in the value
I need to strip the headers out before I use them in the kubernetes secret
package secretcreator
import (
"context"
"encoding/base64"
"encoding/pem"
"log"
"time"
"github.com/Azure/azure-sdk-for-go/sdk/keyvault/azcertificates"
"github.com/Azure/azure-sdk-for-go/sdk/keyvault/azsecrets"
"golang.org/x/crypto/pkcs12"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
)
type MyStruct struct {
kubeClientSet *kubernetes.Clientset
azureVaultSecrets *azsecrets.Client
azureVaultCertificates *azcertificates.Client
beNamespace *v1.Namespace
identitySecrests *v1.Secret
}
func (t *MyStruct) createSecrets() error {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
//tlsSecret.value is a base64 encoded PEM/PFX containing PRIVATE KEY and CERTIFICATES blocks
tlsSecret, err := t.azureVaultSecrets.GetSecret(ctx, `myCertName`, ``, nil)
if err != nil {
log.Printf(`Failed to retrieve cert :%s`, err.Error())
return err
}
//pfxBytes is the Byte representation of the string stored in tlsSecret.Value
pfxBytes, err := base64.StdEncoding.Strict().DecodeString(*tlsSecret.Value)
if err != nil {
log.Printf(`Failed to decode cert PFX:%s`, err.Error())
return err
}
// pemBlocks is a array of *pem.Block - in my case I have 1 PRIVATE KEY Block and three CERTIFICATE Blocks
//the blocks have headers too that look something like this on the PRIVATE KEY, after encoding the encoded string looks like this
//
// -----BEGIN PRIVATE KEY-----
// Microsoft CSP Name: Microsoft Enhanced Cryptographic Provider v1.0
// friendlyName: {D06B3D5A-98D1-45F1-A8C2-DC091232D6A7}
// localKeyId: 01000000
//
// MIIEAI.......ymfjsQ==
// -----END PRIVATE KEY-----
pemBlocks, err := pkcs12.ToPEM(pfxBytes, "")
if err != nil {
log.Printf(`Failed to decode key or cert from PEM Blocks:%s`, err.Error())
return err
}
// k8 needs to store TLS in a secret with key value pairs in the data - one for tls.key and the other for tls.crt
// this code iterates through the blocks, looks at the type of data stored and breaks it up into tls.key and tls.crt
// k8.nginx does not like the headers for some reason - so i strip them out using base64.StdEncoding.Encode to string and
// then adding "-----BEGIN..." and "-----END..." strings"
var tlsKey []byte
var tlsCert []byte
for _, b := range pemBlocks {
if b.Type == "PRIVATE KEY" {
block, _ := pem.Decode(pem.EncodeToMemory(b))
tlsKey = append(tlsKey, "-----BEGIN PRIVATE KEY-----\n"...)
tlsKey = append(tlsKey, base64.StdEncoding.EncodeToString(block.Bytes)...)
tlsKey = append(tlsKey, "-----END PRIVATE KEY-----\n"...)
}
if b.Type == "CERTIFICATE" {
block, _ := pem.Decode(pem.EncodeToMemory(b))
tlsCert = append(tlsCert, "-----BEGIN CERTIFICATE-----\n"...)
tlsCert = append(tlsCert, base64.StdEncoding.EncodeToString(block.Bytes)...)
tlsCert = append(tlsCert, "-----END CERTIFICATE-----\n"...)
}
}
// this following code uses client-go package to create a TLS secret in kubernetes
tlsData := make(map[string][]byte)
tlsData[`tls.key`] = tlsKey
tlsData[`tls.crt`] = tlsCert
beTls := &v1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: `mycertname`,
Namespace: t.beNamespace.Name,
},
Data: tlsData,
Type: v1.SecretTypeTLS,
}
_, err = t.kubeClientSet.CoreV1().Secrets(t.beNamespace.Name).Create(ctx, beTls, metav1.CreateOptions{})
if err != nil {
log.Printf(`Failed to create TLS secret :%s`, err.Error())
return err
}
return nil
}
,,,
I'm using softHSM (FWIW with a go library https://github.com/ThalesIgnite/crypto11, documentation here https://pkg.go.dev/github.com/ThalesIgnite/crypto11?tab=doc).
My goal is to store a 'master key' (AES256) for encrypting objects similarly to how AWS S3 does into the HSM device (because it's more secure). From that key, just derive any other key that I need to encrypt my objects (or decrypt them).
I'm failing at understanding how a generated secret key in a HSM can later be retrieved by the same software program. I see that the API mentions of a context..
rephrased: when I generate a secret key in the HSM like this:
func TestFindingAllKeys(t *testing.T) {
withContext(t, func(ctx *Context) {
for i := 0; i < 10; i++ {
id := randomBytes()
key, err := ctx.GenerateSecretKey(id, 128, CipherAES)
require.NoError(t, err)
defer func(k *SecretKey) { _ = k.Delete() }(key)
}
keys, err := ctx.FindAllKeys()
require.NoError(t, err)
require.NotNil(t, keys)
require.Len(t, keys, 10)
})
}
how do I 'associate' one of those secret keys with my program data (e.g. a S3 bucket or customer)?
How do I retrieve that same secret key again (even if I can't dump it out of the HSM) to decrypt the data at a later time?
I'm missing this apparently stupid, but crucial connection: how does one retrieve a previously generated secret key again at a later time?
You can use pkcs#11 token labels, or equivalent to tag symmetric keys.
You could also use the slot concept, keeping a local database mapping users/customers to keys.
For asymmetric primitives you can export the public key and map this object to a customer/user.
For some reason, I've been asked to implement some basic encryption to secure a specific value transmitted to the client (depending on the client).
Context is: we are the one who generate the encrypted key, we pass that encrypted key to the client, and the client will never have to decrypt it (but we will have to in backend)
Example => we give the encrypted key "123ABCDE==" to the client. The client calls our API passing data + that encrypted key, like:
{
"encKey": "123ABCDE==",
"payload": "somedatahere"
}
Then we decrypt the key, if it matches a specific value in DB (again, depending on client), we continue with some other operations.
So, I decided to go with AES encryption. Following is what I have for now.
definition of the key information:
public class KeyInfo
{
public byte[] Key { get; }
public byte[] Iv { get; }
public KeyInfo()
{
using (var myAes = Aes.Create())
{
Key = myAes.Key;
Iv = myAes.IV;
}
}
public KeyInfo(string key, string iv)
{
Key = Convert.FromBase64String(key);
Iv = Convert.FromBase64String(iv);
}
}
the encryption method
private static byte[] Encrypt_AES(string plainText, byte[] key, byte[] iv)
{
if (plainText == null || plainText.Length <= 0)
throw new ArgumentNullException(nameof(plainText));
if (key == null || key.Length <= 0)
throw new ArgumentNullException(nameof(key));
if (iv == null || iv.Length <= 0)
throw new ArgumentNullException(nameof(iv));
byte[] encrypted;
using (var aesAlgo = Aes.Create())
{
aesAlgo.Key = key;
aesAlgo.IV = iv;
var encryptor = aesAlgo.CreateEncryptor(aesAlgo.Key, aesAlgo.IV);
using (var msEncrypt = new MemoryStream())
{
using (var csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
{
using (var swEncrypt = new StreamWriter(csEncrypt))
{
swEncrypt.Write(plainText);
}
encrypted = msEncrypt.ToArray();
}
}
}
return encrypted;
}
the decryption method:
private static string Decrypt_AES(byte[] cipherText, byte[] key, byte[] iv)
{
if (cipherText == null || cipherText.Length <= 0)
throw new ArgumentNullException(nameof(cipherText));
if (key == null || key.Length <= 0)
throw new ArgumentNullException(nameof(key));
if (iv == null || iv.Length <= 0)
throw new ArgumentNullException(nameof(iv));
string plaintext;
using (var aesAlgo = Aes.Create())
{
aesAlgo.Key = key;
aesAlgo.IV = iv;
var decryptor = aesAlgo.CreateDecryptor(aesAlgo.Key, aesAlgo.IV);
using (var msDecrypt = new MemoryStream(cipherText))
{
using (var csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read))
{
using (var srDecrypt = new StreamReader(csDecrypt))
{
plaintext = srDecrypt.ReadToEnd();
}
}
}
}
return plaintext;
}
}
the key and IV stored in appsettings.json:
"EncryptionKey": "myEncryptionKeyHere",
"EncryptionInitialVector": "myInitialVectorHere",
the registration in Satrtup.cs
services.AddTransient(ec => new EncryptionService(new KeyInfo(appSettings.EncryptionKey, appSettings.EncryptionInitialVector)));
I have a few question about all of this.
is AES rh right choice for my needs?
is the way it is implemented here correct?
where should I store the Key and IV ? (not sure appsettings.json is okay)
how do I generate the Key and IV ? Is there tools for that?
Thanks for reading!
EDIT:
You say "we are the one who generate the encrypted key, we pass that
encrypted key to the client" - how does that happen securely?
-> The client will have to connect to his account where he could access that encKey.
So, reading #vcsjones answer, AES may not be the right thing to implement here. Since I don't want to store the IV on database, if the client loses it, it means he would have to generate another key, with another IV, and change the encKey in all applications.
Would an asymetric encryption be better? If I understood correctly, it would mean to encrypt the value with a private key, give that encrypted value to the client + the public key (which would be the same for every client?)
is AES the right choice for my needs?
By itself, no it is not. AES is a cryptographic primitive - a building block - and such building blocks are not usually useful by themselves. For example with AES-CBC (the mode you are using), this is currently vulnerable to a padding oracle attack and lacks authentication. AES might be the right choice when combined with other primitives that provide authentication, like an HMAC.
The best way to solve this problem is to treat primitives for what they are - primitives that are insufficient for use on their own. There are other libraries, like libsodium, that are more abstracted concepts of cryptography, and provides simple APIs that "do the right thing".
You say "we are the one who generate the encrypted key, we pass that encrypted key to the client" - how does that happen securely?
Barring using something like libsodium, there are some issues to address.
There is no authentication of the ciphertext ("authentication" in cryptography has its own meaning, not like sign-on authentication).
The initialization vector should never be used more than once with the same key. You appear to be using a fixed IV. Every thing you encrypt should use its own random IV. The IV is not a secret. It should be authenticated however along with the cipher text as per point 1.
where should I store the Key and IV ?
The IV, since there should be a 1:1 of them with each cipher text (encrypted output) should be stored with the cipher text. When it's time to decrypt, the caller will need to provide the IV again.
The Key is the real secret. Storing them securely is important. If you store it in appsettings.json, then the security of your key is the same as the security of that JSON file. A more common approach in a cloud environment is to use a managed service. Azure has Azure Key Vault, AWS has KMS and Secret Manager.
how do I generate the Key and IV ? Is there tools for that?
They should be generated with a CSPRNG. For example, to do so programmatically:
byte[] iv = new byte[128 / 8];
RandomNumberGenerator.Fill(iv);
//iv now contains random bytes
You can do the same for generating a key.
I am creating an app using Go and I am trying to start a https server using the ListenAndServeTLS function. Here is my code:
func StartServer() {
defer config.CapturePanic()
c := config.GetInstance()
serverAddress := fmt.Sprintf(":%s", c.GetConfig().ServerPort)
server := http.Server{Addr: serverAddress}
log.Info("Starting local server")
http.HandleFunc("/", login.Handler)
http.HandleFunc("/login", login.Handler)
http.HandleFunc("/settings", settings.Handler)
//cert, _ := data.Asset("my-cert.pem")
//key, _ := data.Asset("my-key.pem")
err := server.ListenAndServeTLS("my-cert.crt", "my-cert.key")
if err != nil {
log.WithError(err).Fatal("Error stopping local server")
}
}
The thing is that I would like to embed my certificate and its key inside my executable file and then pass them to the the server.ListeAndServeTLS function as a string or a byte array. However this function does not take these types of arguments. Is there another way to do this?
Note: I am aware that it is a bad practice to embed a private key inside a client application, however what I am trying to do here is just to create a config webpage that will be hosted as https://localhost:8080.
You can build your own server object and still call ListenAndServeTLS. Since your tls config has certificates, it will ignore the passed-in filenames.
I'm omitting the return on error for conciseness, please do not:
// Generate a key pair from your pem-encoded cert and key ([]byte).
cert, err := tls.X509KeyPair(<cert contents>, <key contents>)
// Construct a tls.config
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{cert}
// Other options
}
// Build a server:
server := http.Server{
// Other options
TLSConfig: tlsConfig,
}
// Finally: serve.
err = server.ListenAndServeTLS("", "")
In my case, I was unable to load files from disk, but did have the certificates passed in to the environment as variables ([]byte).
Adding on to Marc's answer, I had to only change the top line.
// Append signed leaf certificate and intermediate to a single
certChain := append(leaf.PublicBytes, intermediate.PublicBytes...)
// Generate a key pair from your pem-encoded cert and key ([]byte).
cert, err := tls.X509KeyPair(certChain, leaf.PrivateBytes)
...
I'm doing some tests with HMAC by using a time-window mechanism based on UTC+0 synced time. The server has a special public API call http://myserver.com/api/servertime/ that will return the server's exact UTC+0 time. This way the API users can sync their requesting client so it will be able to match the time window my API allows for secure calls. I built in a 30 minute timeslot (-15min - +15min).
My code looks like this:
func GenerateHmac512(message []byte, key []byte) []byte {
h := hmac.New(sha512.New, key)
h.Write(message)
return []byte(base64.StdEncoding.EncodeToString(h.Sum(nil)))
}
func ValidateHmac512(message, messageMAC, key []byte) bool {
var err error
decryptedMessageMAC, err := base64.StdEncoding.DecodeString(string(messageMAC))
if err != nil {
log.Fatalln(err.Error())
return false
}
mac := hmac.New(sha512.New, key)
mac.Write(message)
expectedMAC := mac.Sum(nil)
return hmac.Equal(decryptedMessageMAC, expectedMAC)
}
func main() {
timestamp := time.Now().Unix()
key := []byte("afad9411468602782fb62d904f623d87")
message := []byte(fmt.Sprintf("SecretHash,Value1,Value2,Value3,TimeStamp:%d", time.Now().Unix()))
hmacHash := GenerateHmac512(message, key)
hmacValid := ValidateHmac512(message, hmacHash, key)
log.Println(timestamp)
log.Println(string(hmacHash))
log.Println(hmacValid)
requestValid := false
if timestamp > time.Now().Unix()-(60*15) && timestamp < time.Now().Unix()+(60+15) {
requestValid = true
}
log.Println(requestValid)
}
I'm hashing the timestamp that will be publicly provided in the call in my HMAC hash, combined with the secret hash. I'm wondering if this is fool-proof enough, or it would need more work to make it totally solid? The call would be something like this:
POST http://myserver.com/api/users/
Value1 : Data1
Value2 : Data2
Value3 : Data3
Timestamp : 1420497639
Eventually when this is all OK I'm gonna send this data over SSL/TLS. I know SSL is more than enough and HMAC wouldn't be needed, but I like to have these 3 layers of security. And I want to benchmark variations of these layers to see what the performance impact is and how I can tweak it to have a good balance between performance and security.
There's not much to answer here, an HMAC authenticates a message and verifies integrity, and that seems to be what you want. Also, TLS is only "more than enough" if you're authenticating the client. If this is an unauthenticated call, then HMAC is still reasonable to prove the knowledge of a shared secret.
Note that SecretHash is superfluous. You already have a secret shared key.