AWS KMS python3.6 and boto3 example with AES CBC - python-3.x

I have tried to do an ecnryption demo using python 3.6 and boto3 with AWS KMS but it lacks the operational mode of AES. I wonder if you can point me in the direction of how to do this.
I have tried to define AES.CBC_MODE within the calling of the KeySpec but it only takes AES_256 or AES_128.
Here is the code:
import base64
import boto3
from Crypto.Cipher import AES
PAD = lambda s: s + (32 - len(s) % 32) * ' '
def get_arn(aws_data):
return 'arn:aws:kms:{region}:{account_number}:key/{key_id}'.format(**aws_data)
def encrypt_data(aws_data, plaintext_message):
kms_client = boto3.client(
'kms',
region_name=aws_data['region'],
aws_access_key_id='your_key_id',
aws_secret_access_key='your_secred_key_id')
data_key = kms_client.generate_data_key(
KeyId=aws_data['key_id'],
KeySpec='AES_256')
cipher_text_blob = data_key.get('CiphertextBlob')
plaintext_key = data_key.get('Plaintext')
# Note, does not use IV or specify mode... for demo purposes only.
cypher = AES.new(plaintext_key, AES.MODE_CBC)
encrypted_data = base64.b64encode(cypher.encrypt(PAD(plaintext_message)))
# Need to preserve both of these data elements
return encrypted_data, cipher_text_blob
def decrypt_data(aws_data, encrypted_data, cipher_text_blob):
kms_client = boto3.client(
'kms',
region_name=aws_data['region'])
decrypted_key = kms_client.decrypt(CiphertextBlob=cipher_text_blob).get('Plaintext')
cypher = AES.new(decrypted_key)
return cypher.decrypt(base64.b64decode(encrypted_data)).rstrip()
def main():
# Add your account number / region / KMS Key ID here.
aws_data = {
'region': 'us-east-1',
'account_number': 'your_account',
'key_id': 'your_key_id',
}
# And your super secret message to envelope encrypt...
plaintext = 'Superduper and the mighty Scoop!'
# Store encrypted_data & cipher_text_blob in your persistent storage. You will need them both later.
encrypted_data, cipher_text_blob = encrypt_data(aws_data, plaintext)
print(encrypted_data)
# # Later on when you need to decrypt, get these from your persistent storage.
decrypted_data = decrypt_data(aws_data, encrypted_data, cipher_text_blob)
print(decrypted_data)
if __name__ == '__main__':
main()

Rather than implementing your own envelope encryption, have you considered using the AWS Encryption SDK?[1][2] It integrates closely with AWS KMS and makes it simple to do secure envelope encryption, protecting your data keys with a KMS CMK. It also makes it simple to keep track of all the pieces you need for decryption (IV, encrypted data key, encryption context, etc) by giving you back a single ciphertext message that contains everything that the client needs to know in order to decrypt the message.
You can find an example of how to implement something similar to what you show in your question here[3].
[1] https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/introduction.html
[2] https://aws-encryption-sdk-python.readthedocs.io/en/latest/
[3] https://github.com/aws/aws-encryption-sdk-python/blob/master/examples/src/basic_encryption.py

Related

Troubleshooting type "PGPKeyring" is not callable

Working with PGPy and using PGPKeyring.
My code is pretty straigtforward.
I first have a function to fetch the keys considering that
encryption requires a public key, while decryption requires a private or secret key.
We have a GoAnyWhere server that is being demoted and all this encryption has to be moved to Python.
import sys
import pgpy
from pgpy import PGPKey
from loguru import logger
def fetch_keys(self.action):
#Select the correct key
if self.action == "encrypt":
#Encryption requires a public key...go get it.
KR = self.PGPKeyServer + self.publicKeyPath + self.publicKeyRing
KeyRing = pgpy.PGPKeyring(KR)
with KeyRing(self.keyID) as key:
#Unload a loaded key and its subkeys.
unloadKR = KeyRing.unload(key)
logger.info(f"Pub Key Ring has unloaded - {self.keyID}")
PublicKey, _ = PGPKey.from_file(unloadKR)
It says "Object of type "PGPKeyring" is not callable"
It's also saying "Type of "unload" is "(key: Unknown) -> None"
I'm assuming that I need to extract the key from the ring before I load it into PublicKey
After this part is working I'll do the following..
elif self.action == "decrypt":
#Decryption requires a secret key...go get it.
KeyRing = self.PGPKeyServer + self.secretKeyPath + self.secretKeyRing
KeyRing = pgpy.PGPKeyring(KeyRing)
with KeyRing(self.keyID) as key:
#Unload a loaded key and its subkeys.
unloaded = KeyRing.unload(key)
logger.info(f"Secret Key Ring has unloaded - {self.keyID}")
SecKey, _ = PGPKey.from_file(unloadKR)
And the secret key will be locked with a passphrase where I'll do something like...
with SecKey.unlock(self.passphrase):
# Secretkey is now unlocked
assert Seckey.is_unlocked
If anybody has worked with PGPy before and can give me some advice on the best way forward it would be appreciated.
For now I just want to get line 1 through 12 working so I can run the script passing "encrypt" as the parameter and get some logged result. Is "with KeyRing(self.keyID) as key:" not the proper syntax?
I've tried to work with other python libraries for encryption and decryption and think PGPy has the best examples to follow. I haven't been able to run the script yet with any success.
My run parameters will be passed like so.
def run(self) -> None:
if self.action == "encrypt":
PubKey = fetch_keys(self.action)
# self.do_encrypt(PubKey)
elif self.action == "decrypt":
SecKey = fetch_keys(self.action)
# self.do_decrypt(SecKey)
else:
logger.error(
f"Encrytion for: {self.file_name} - Invalid action '{self.action}'"
)

Python 3 write encrypted string of bytes to file

I encrypt string:
def encrypt(self, message):
obj = AES.new('This is a key123'.encode("utf8"), AES.MODE_CFB, 'This is an IV456'.encode("utf8"))
encrypted = obj.encrypt(message.encode("utf8"))
return encrypted
How can I store encrypted in a file and read to decrypt using:
def decrypt(self, encrypted):
obj = AES.new('This is a key123'.encode("utf8"), AES.MODE_CFB, 'This is an IV456'.encode("utf8"))
decrypted=obj.decrypt(encrypted)
return decrypted
THe library "pycryptodome" has a full running example for AES encryption to a file and vice versa.
I know that the example runs another mode and stores additional data but that might be helpfull to you as the usage of a static IV is UNSECURE: https://pycryptodome.readthedocs.io/en/latest/src/examples.html
The following code generates a new AES128 key and encrypts a piece of data into a file. We use the EAX mode because it allows the receiver to detect any unauthorized modification (similarly, we could have used other authenticated encryption modes like GCM, CCM or SIV).
from Crypto.Cipher import AES
from Crypto.Random import get_random_bytes
key = get_random_bytes(16)
cipher = AES.new(key, AES.MODE_EAX)
ciphertext, tag = cipher.encrypt_and_digest(data)
file_out = open("encrypted.bin", "wb")
[ file_out.write(x) for x in (cipher.nonce, tag, ciphertext) ]
file_out.close()
At the other end, the receiver can securely load the piece of data back (if they know the key!). Note that the code generates a ValueError exception when tampering is detected.
from Crypto.Cipher import AES
file_in = open("encrypted.bin", "rb")
nonce, tag, ciphertext = [ file_in.read(x) for x in (16, 16, -1) ]
# let's assume that the key is somehow available again
cipher = AES.new(key, AES.MODE_EAX, nonce)
data = cipher.decrypt_and_verify(ciphertext, tag)

Encrypting strings with a predetermined key

I'm trying to make a program that fetches someone's MAC Address from their machine, encrypts it, and then copies it to their clipboard. However, all of the encryption methods I see generate a fresh key and thus can't be deciphered without knowing the specific key that was used to encrypt the address. Is there a way to use one key to encrypt everything so all addresses can be decrypted with a single key, and a fresh key is not generated every single time?
you can try it, using Fernet Lib:
from cryptography.fernet import Fernet
# IMPORTANT: The encryption key must be binary, so the prefix 'b' before the string
# To create a random binary key, use 'generate_key' method as below:
# new_key = Fernet.generate_key()
crypto_key = b'dTlQeWw2u5oMoFPHXQ7vQHPaQUEiD71SYzWeJJAQQUk='
mac = '00:33:A4:D9:F1:E1'
fernet = Fernet(crypto_key)
enc_mac = fernet.encrypt(mac.encode())
dec_mac = fernet.decrypt(enc_mac).decode()
print(f'Fixed encryption key: {crypto_key}')
print('Original MAC string: ', mac)
print('Encrypted MAC string: ', enc_mac)
print('Decrypted MAC string: ', dec_mac)
You are describing asymmetric encryption here.
That exists and is a thing, yes. It works by by having a public key for encryption, and a private key for decryption.
There are multiple algorithms that implement that, like RSA.
RSA is supported by the python library cryptography.
A tutorial on how to use it can be found for example here:
https://nitratine.net/blog/post/asymmetric-encryption-and-decryption-in-python/
Working example
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.asymmetric import rsa, padding
from cryptography.hazmat.primitives import serialization, hashes
# Generate keys. This only has to be done once.
# Store the keys somewhere and distribute them with the program.
def generate_keys():
private_key = rsa.generate_private_key(
public_exponent=65537,
key_size=2048,
backend=default_backend()
)
public_key = private_key.public_key()
private_key_string = private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.TraditionalOpenSSL,
encryption_algorithm=serialization.NoEncryption()
)
public_key_string = public_key.public_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PublicFormat.SubjectPublicKeyInfo
)
return (public_key_string, private_key_string)
# This is just for demonstration.
# In practice, don't generate them every time.
# Only generate them once and store them in a string or a file.
(public_key_string, private_key_string) = generate_keys()
# REMOTE COMPUTER
# Only use the public key here, the private key has to stay private.
public_key = serialization.load_pem_public_key(public_key_string, backend=default_backend())
mac_address = "01:23:45:67:89:AB"
mac_address_encrypted = public_key.encrypt(
mac_address.encode(),
padding.OAEP(
mgf=padding.MGF1(algorithm=hashes.SHA256()),
algorithm=hashes.SHA256(),
label=None
)
)
# LOCAL SERVER
# Use private keys here to decrypt the MAC address
private_key = serialization.load_pem_private_key(private_key_string, password=None, backend=default_backend())
mac_address_decrypted = private_key.decrypt(
mac_address_encrypted,
padding.OAEP(
mgf=padding.MGF1(algorithm=hashes.SHA256()),
algorithm=hashes.SHA256(),
label=None
)
).decode()
print(mac_address_decrypted)
https://ideone.com/0eEyU6
You can use Import RSA library rsa
installing :
pip install rsa
Then encrypt the byte string with the public key.
Then the encrypted string can be decrypted with the private key.
The public key can only be used for encryption and the private can only be used for decryption
for examle:
import rsa
publicKey, privateKey = rsa.newkeys(512)
message = "Salio" #this is MAC Address
encMessage = rsa.encrypt(message.encode(), publicKey)
print("encrypted: ", encMessage)
decMessage = rsa.decrypt(encMessage, privateKey).decode()
print("decrypted : ", decMessage)

Decrypt data using cryptography package giving error "ValueError: Encryption/decryption failed."

import base64
import os.path
from shutil import copyfile
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization, hashes
from cryptography.hazmat.primitives.asymmetric import padding, rsa
from cryptography.hazmat.backends.openssl.rsa import _RSAPublicKey, _RSAPrivateKey
from asym_crypto_yaml import (decrypt_value, encrypt_value, Encrypted,
load_private_key_from_file, load_public_key_from_file,
generate_new_private_key, generate_new_public_key,
load, dump, NUMBER_OF_BYTES_PER_ENCRYPTED_CHUNK, KEY_CHUNK_SIZE,
SUPPORTED_KEY_SIZES, generate_private_key_to_file, generate_private_key_to_file, generate_public_key_to_file,
encrypt_value_and_print ,add_secret_to_yaml_file, decrypt_yaml_file_and_write_encrypted_file_to_disk,
reencrypt_secrets_and_write_to_yaml_file)
from functools import reduce
def test_add_secret_to_yaml_file():
private_key_output_filename = "/home/asy/private_key.private"
public_key_output_filename = "/home/asy/public_key.public"
private_key = generate_private_key_to_file(private_key_output_filename)
public_key = generate_public_key_to_file(private_key_output_filename, public_key_output_filename)
yaml_file_fixture = "/home/asy/saml.yml"
yaml_file_to_append_to = "/home/asy/saml_du.yml"
test_key_to_encrypt = ['FACEBOOK_APP_ID', 'FACEBOOK_APP_SECRET', 'AWS_S3_BUCKET', 'SECRET_TOKEN', 'TWITTER_CONSUMER_KEY', 'TWITTER_CONSUMER_SECRET',
'TWITTER_OAUTH_TOKEN', 'TWITTER_OAUTH_TOKEN_SECRET', 'LINKEDIN_API_KEY', 'LINKEDIN_SECRET_KEY']
print ("################################ENCRYPT YAML########################################")
before_dict = None
with open(yaml_file_to_append_to, "r") as f:
before_dict = load(f)
# Encrypt data in yml file
for test_key in test_key_to_encrypt:
print ('Encrypted key is:', test_key)
print ('Encrypted value is:', before_dict[test_key])
add_secret_to_yaml_file(test_key, before_dict[test_key], public_key_output_filename, yaml_file_to_append_to)
print ("################################DECRYPT YAML########################################")
before_dict = None
with open(yaml_file_to_append_to, "r") as f:
before_dict = load(f)
# Decrypt data from yml file (Using same function)
for test_key_value in test_key_to_encrypt:
print ('key is', before_dict[test_key_value])
test_encrypted_key_value = decrypt_value(before_dict[test_key_value], private_key)
print ("decrypt data", test_encrypted_key_value)
#
def decrypt_data():
private_key_output_filename = "/home/asy/private_key.private"
public_key_output_filename = "/home/asy/public_key.public"
private_key = generate_private_key_to_file(private_key_output_filename)
public_key = generate_public_key_to_file(private_key_output_filename, public_key_output_filename)
yaml_file_to_append_to = "/home/asy/saml_du.yml"
test_key_to_encrypt = ['FACEBOOK_APP_ID', 'FACEBOOK_APP_SECRET', 'AWS_S3_BUCKET', 'SECRET_TOKEN', 'TWITTER_CONSUMER_KEY', 'TWITTER_CONSUMER_SECRET',
'TWITTER_OAUTH_TOKEN', 'TWITTER_OAUTH_TOKEN_SECRET', 'LINKEDIN_API_KEY', 'LINKEDIN_SECRET_KEY']
print ("################################DECRYPT YAML########################################")
before_dict = None
with open(yaml_file_to_append_to, "r") as f:
before_dict = load(f)
for test_key_value in test_key_to_encrypt:
print ('key is', test_key_value)
print ('value is', before_dict[test_key_value])
test_encrypted_key_value = decrypt_value(before_dict[test_key_value], private_key)
print ("decrypt data", test_encrypted_key_value)
if __name__ == "__main__":
test_add_secret_to_yaml_file()
# decrypt_data()
sample yml file:
SECRET_TOKEN: "d4e5783de1c74c7a4e3a27578df6gdgf6g786g8df7g6g87d6fgb709"
FACEBOOK_APP_ID: "35864341"
FACEBOOK_APP_SECRET: "759a1e7sd7fvyfsd473"
TWITTER_CONSUMER_KEY: "1UrRKJDF8SD7FSDF3S"
TWITTER_CONSUMER_SECRET: "5W7TE8KJJk787bnG0s"
TWITTER_OAUTH_TOKEN: "716397744-3rHXFkFkjKjkjK78PQ5"
TWITTER_OAUTH_TOKEN_SECRET: "DuDJKFSD89SDFD"
LINKEDIN_API_KEY: "2vjkJKjk4"
LINKEDIN_SECRET_KEY: "5KLSJDFsE"
GMAIL_USERNAME: "username#gmail.com"
GMAIL_PASSWORD: "PASSWORD"
AWS_ACCESS_KEY_ID: "ASDKLSDJFIA"
AWS_SECRET_ACCESS_KEY: "7ASDFJksdfjskdlf87sdfKb"
AWS_S3_BUCKET: "bucket"
development:
MAILER_HOST: "localhost:3000"
test:
MAILER_HOST: "localhost:3000"
production:
MAILER_HOST: "domain.com"
I am using "asym_crypto_yaml" yaml package to write encrypted value in .yml file.
I am not able to decrypt value from different decrypt function (decrypt_data()).
Above code only decrypt value if I execute code first time. But from second time its giving "encryption/decryption error".
My objective is to decrypt data from yml file. Little help will be appreciated.
The error is triggered because the private key used in decrypt_data() for decryption does not belong to the public key used in test_add_secret_to_yaml_file() to perform the encryption. Therefore, decryption with this private key fails.
The problem can be solved by using in decrypt_data() the private key of the key pair generated in test_add_secret_to_yaml_file(). To do this, remove the generate_private_key_to_file() and generate_public_key_to_file() calls (to generate and store a key pair) in decrypt_data(). The required private key can be loaded with load_private_key_from_file() from the file where it was stored in test_add_secret_to_yaml_file().
This ValueError: Encryption/decryption failed. Error is also thrown by the hazmat class when the data you want to encrypt is too large for the size of key that you generated. Try again by using larger key size like this.
private_key = rsa.generate_private_key(public_exponent=65537, key_size = 4096)
but keep in mind it will increase time to generate key, that is one of the biggest disadvantage of RSA encryption algorithm.

Python Diffie-Hellman exchange cryptography library. Shared key not the same

Following the tutorial listed on the cryptography library documentation I have successfully created a function that demonstrates a Diffie-Hellman exchange. I am now trying to create a proof of concept socket server and socket client.
An undocumented requirement of this application is the sending of the public key to the client. This requires the DHPublicKey object to be serialised and serialised in order for it to be sent over the socket.
By doing this however the shared keys are not the same! I have tried to use different encoding formats (PEM for example) to see if this made a difference. Unfortunatly it has not. I get a different shared key on both sides. Here is an example of my code.
Client
parameters = dh.generate_parameters(generator=2, key_size=1024, backend=default_backend())
client_private_key = parameters.generate_private_key()
client_public_key = client_private_key.public_key().public_bytes(Encoding.DER, PublicFormat.SubjectPublicKeyInfo)
# Recv Server Pub key
length = s.recv(2) # Prepend the length of the message
server_public_key = s.recv(int.from_bytes(length, "big"))
print("Got server public key: " + str(server_public_key))
server_public_key = load_der_public_key(server_public_key, default_backend())
# Send Pub key
s.send(len(client_public_key).to_bytes(2, "big") + client_public_key)
print("Generating shared key...")
shared_key = client_private_key.exchange(server_public_key)
print("Our shared key!: " + str(shared_key))
Server
parameters = dh.generate_parameters(generator=2, key_size=1024, backend=default_backend())
server_private_key = parameters.generate_private_key()
server_public_key = server_private_key.public_key().public_bytes(Encoding.DER, PublicFormat.SubjectPublicKeyInfo)
newsocket.send(len(server_public_key).to_bytes(2, "big") + server_public_key)
print("Sent server public key: " + str(server_public_key))
length = newsocket.recv(2) # Prepend the length of the message
client_public_key = newsocket.recv(int.from_bytes(length, "big"))
client_public_key = load_der_public_key(client_public_key, default_backend())
# Send the public key to the client
shared_key = server_private_key.exchange(client_public_key)
print("Our shared key is: " + str(shared_key))
As stated, I'm using the Python 3 Library Cryptography and use the following imports
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.asymmetric import dh
from cryptography.hazmat.primitives.serialization import PublicFormat, Encoding, load_der_public_key
Also, putting the code into one file (Without network serialization) it works! the key is the same! Here is an example of the working code
print("SERVER: Performing DH exchange. DH 2048-bit key size")
parameters = dh.generate_parameters(generator=2, key_size=2048, backend=default_backend()) # Generate a 256-byte key
print("SERVER: Generating server private and public keys")
server_private_key = parameters.generate_private_key()
server_peer_public_key = server_private_key.public_key()
print("CLIENT: Generating private and public keys")
client_private_key = parameters.generate_private_key()
client_peer_public_key = client_private_key.public_key()
print("SERVER: Sending public key to client")
print("CLIENT: Sending public key to server")
server_shared_key = server_private_key.exchange(client_peer_public_key)
client_shared_key = client_private_key.exchange(server_peer_public_key)
print("server key is: " + str(server_shared_key))
print("client key is: " + str(client_shared_key))
What am I doing wrong when serialising or deserialising the key?
Your problem isn’t to do with serializing and deserializing the key, it is because you are generating different DH parameters on the server and the client. They need to be the same for Diffie Hellman to work.
You could generate the parameters on the server and send them to the client, but a better option is to use a set of predefined parameters, for example group 14 defined in RFC 3526.
To do that, change the line
parameters = dh.generate_parameters(generator=2, key_size=1024, backend=default_backend())
in both client and server to:
p = 0xFFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD129024E088A67CC74020BBEA63B139B22514A08798E3404DDEF9519B3CD3A431B302B0A6DF25F14374FE1356D6D51C245E485B576625E7EC6F44C42E9A637ED6B0BFF5CB6F406B7EDEE386BFB5A899FA5AE9F24117C4B1FE649286651ECE45B3DC2007CB8A163BF0598DA48361C55D39A69163FA8FD24CF5F83655D23DCA3AD961C62F356208552BB9ED529077096966D670C354E4ABC9804F1746C08CA18217C32905E462E36CE3BE39E772C180E86039B2783A2EC07A28FB5C55DF06F4C52C9DE2BCBF6955817183995497CEA956AE515D2261898FA051015728E5A8AACAA68FFFFFFFFFFFFFFFF
g = 2
params_numbers = dh.DHParameterNumbers(p,g)
parameters = params_numbers.parameters(default_backend())
Now both client and server will be using the same set of parameters and the key agreement will work. It will also be much faster, parameter generation is a costly process.
Your code in a single file works because you only generate one set of parameters that is used by both sides of the exchange.

Resources