I need to encrypt data stored in web2py, more precisely passwords.
This is not about authentication, but more something in the line of a KeePass-like application.
I've seen that is included in web2py, but and M2Secret could easily do that. With M2Secret I can use this:
import m2secret
# Encrypt
secret = m2secret.Secret()
secret.encrypt('my data', 'my master password')
serialized = secret.serialize()
# Decrypt
secret = m2secret.Secret()
secret.deserialize(serialized)
data = secret.decrypt('my master password')
But I would have to include the M2Crypto library in my appliance.
Is there a way to do this with PyMe which is already included with web2py?
By default web2py stores passwords hashed using HMAC+SHA512 so there is nothing for you to do. It is better than the mechanism that you suggest because encryption is reversible while hashing is not. You can change this and do what you ask above but it would not be any more secure than using plaintext (since you would have to expose the encryption key in the app).
Anyway. Let's say you have a
db.define_table('mytable',Field('myfield'.'password'))
and you want to use m2secret. You would do:
class MyValidator:
def __init__(self,key): self.key=key
def __call__(self,value):
secret = m2secret.Secret()
secret.encrypt(value, self.key)
return secret.serialize()
def formatter(self,value):
secret = m2secret.Secret()
secret.deserialize(value)
return (secret.decrypt(self.key),None)
db.mytable.myfield.requires=MyValidator("master password")
In web2py validators are also two way filters.
Related
I need to encrypt some message before sending it as request body in an API. I have to do this in Python and the library I used is Pycryptodome. From hereon, when I use the term API, I am referring to the API endpoint provided by the vendor and not the Pycryptodome API. The receiver(API vendor) shared the certificate (RSA 2048 bits) which I need to use to encrypt this message. This certificate is in binary format (DER) and has .cer as the extension. I need to import this using Pycryptodome and from the docs (https://pycryptodome.readthedocs.io/en/latest/src/public_key/rsa.html), I find that the import_key method accepts X.509 certificates in either binary or PEM encoding. The API documentation says I need to use RSA/NONE/OAEPWithSHA1AndMGF1Padding for encryption. This is my code.
def encrypt_message(message):
with open('/home/krishna/work/.certificates/random_certificate.cer', 'rb') as _file:
key_contents = _file.read()
recipient_key = RSA.import_key(key_contents)
cipher_rsa = PKCS1_OAEP.new(recipient_key)
return base64.b64encode(cipher_rsa.encrypt(message.encode())).decode()
I read the PKCS1_OAEP.py file, the default mask generation function is MGF1, default hash algorithm is SHA1. Hence, I did not specify these parameters in the new function in the above code. Also, I need to send this encrypted message as a base64 string (UTF-8). After reading the API documentation and the Pycryptodome documentation, I felt this code is sufficient for encryption. Also, this code is somewhat standard and is almost the same as the one in the Pycryptodome documentation. However, I get a decryption error and the vendor tells me this is due to "BAD PADDING EXCEPTION". This should not happen since I used OAEP for padding and the vendor confirmed they are using OAEP for padding as well. This is very hard for me to debug as I do not possess the vendor's private key.
So I tried generating my own pair of private and public keys (RSA-2048) and checked if I am able to decrypt my message. And duh, I am able to decrypt my message. Here is the code for encryption and decryption using my own keys.
def encrypt_message(message):
with open('my_pub.pem', 'rb') as _file:
pub = _file.read()
recipient_key = RSA.importKey(pub)
cipher_rsa = PKCS1_OAEP.new(recipient_key)
return base64.b64encode(cipher_rsa.encrypt(message.encode())).decode()
def decrypt_message(gibberish):
with open('my_priv.pem', 'rb') as _file:
priv = _file.read()
pvt_key = RSA.importKey(priv)
cipher_rsa = PKCS1_OAEP.new(pvt_key)
return cipher_rsa.decrypt(base64.b64decode(gibberish.encode())).decode()
So where I am going wrong? They key difference between this test and the actual API call is that my keys are generated in PEM format and have .pem as the extension. In the case of the API call, I have to use a .cer certificate in DER format. But I am assuming Pycryptodome is handling this for me (from the docs).
I have a few more questions regarding this.
What is NONE in RSA/NONE/OAEPWithSHA1AndMGF1Padding?
What is the difference between import_key() and importKey() in Pycryptodome? There are 2 methods and it does not look like importKey() has some documentation to it.
Can padding exceptions occur while using OAEP? From what I read from the internet, OAEP is the best method out there, not only in terms of adding randomness during encryption, but I also read that padding exceptions are very rare when this is used.
I really need to know what is going wrong here as I am dead clueless. Any suggestions or help will be appreciated.
I can successfully create an Hmac via NodeJS using the following code:
(slightly altered example from : https://nodejs.org/api/crypto.html#cryptocreatehmacalgorithm-key-options)
Crypto.createHmac('sha256', Crypto.randomBytes(16))
.update('I love cupcakes')
.digest('hex');
That results in a value like the following (hex-based string Hmac signature):
fb2937ca821264812d511d68ae06a643915931375633173ba64af9425f2ffd53
How do I use that signature to verify that the data was not altered? (using NodeJS, of course).
My Assumption
I'm assuming there is a method call where you supply the data and the signature and you get a boolean that tells you if the data was altered or not -- or something similar.
Another Solution?
Oh, wait, as I was writing that I started thinking...
Do I need to store the original random bytes I generated (Crypto.randomBytes(16)) and pass them to the receiver so they can just generate the HMac again and verify that the result is the same (fb2937ca821264812d511d68ae06a643915931375633173ba64af9425f2ffd53)?
If that is true that would be odd, because the parameter for Crypto.randomBytes(16) is named secret (in the official example)*. Seems like that needs to be kept secret??
Please let me know if there is a way to verify the signature on the receiving side & how I do that.
Official Documentation : A Bit Confusing
Here's the function as it is defined in the official docs:
crypto.createHmac(algorithm, key[, options])
In the function definition, you can see the second param is named key.
However, in the example they refer to it as secret
const secret = 'abcdefg';
const hash = crypto.createHmac('sha256', secret)
.update('I love cupcakes')
.digest('hex');
console.log(hash);
Just posting the answer so if anyone in future sees this they will be able to have the definitive answer.
As the commentor (Topaco) pointed out, the simple answer is that:
The receiver who want wants to validate the Hmac simply needs to use the same key value & data and apply it to the method and retrieve the hash value.
const secret = 'abcdefg';
const hash = crypto.createHmac('sha256', secret)
.update('I love cupcakes')
.digest('hex');
console.log(hash);
The original Hmac-creating party must provide three things for the verifying party:
data : (could be encrypted data from AES256, for example)
key : original key passed into the createHmac() method -- note: this item is called secret in the sample code by NodeJS (above).
hash :the (clearText) hash which the original creator generated when calling the createHmac() method.
With those three things the verifying party can now call the createHmac() method and determine if the hash they get matches the hash that the original hmac-creating party generated.
Doing this validates that the Data which was sent has not been corrupted or altered.
Additional Note On Key (secret)
I've come back after thinking about the Hmac a bit more.
It is required that both parties know the key (aka secret) but it does not mean that it should be exposed to others.
This must be kept secret (as the code implies) because if a nefarious type knew the value and could alter it, then they could also alter the data and generate a new key (secret) and pass it along as if the original creator sent it along (MITM - man in the middle attack).
So, the point here is that yes, both parties have to know the key (secret) value, but it should not be shared where it might be discovered by nefarious types.
Instead, it will have to be agreed upon or based upon a secret password, etc.
I understand I can encrypt and store the contents of files (csv mainly) using the techniques explained here and here.
However, I am looking for a way to prevent anyone from accessing these files, even users with sudo access to the server. The only one (or group of people) who should be able to access the encrypted files would be those who have a password or encryption key chosen by me. Is this possible?
By default the file will show encrypted data, hence even if the file anyone get cant seen data. however you can also put key protectection using class from Encrypt directly or using spatie crypt
Here LOOK Spatie link
You can also use the default crypt like this
use Illuminate\Encryption\Encrypter;
//Keys and cipher used by encrypter(s)
$fromKey = base64_decode("from_key_as_a_base_64_encoded_string");
$toKey = base64_decode("to_key_as_a_base_64_encoded_string");
$cipher = "AES-256-CBC"; //or AES-128-CBC if you prefer
//Create two encrypters using different keys for each
$encrypterFrom = new Encrypter($fromKey, $cipher);
$encrypterTo = new Encrypter($toKey, $cipher);
//Decrypt a string that was encrypted using the "from" key
$decryptedFromString = $encrypterFrom->decryptString("gobbledygook=that=is=a=from=key=encrypted=string==");
//Now encrypt the decrypted string using the "to" key
$encryptedToString = $encrypterTo->encryptString($decryptedFromString);
I am trying to encrypt my postgres connection string variable.
db_host = os.environ.get('db_host')
db_port = os.environ.get('db_port')
db_name =os.environ.get('db_name')
db_user =os.environ.get('db_user')
db_pass =os.environ.get('db_pass')
as you can see I have used environment variable, but I want to encrypt it.
I have figured out a way to encrypt it was also checking a way to decrypt it.
Lambda function suggested a below code.
import boto3
import os
from base64 import b64decode
ENCRYPTED = os.environ['db_host']
# Decrypt code should run once and variables stored outside of the function
# handler so that these are decrypted once per container
DECRYPTED = boto3.client('kms').decrypt(
CiphertextBlob=b64decode(ENCRYPTED),
EncryptionContext={'LambdaFunctionName': os.environ['AWS_LAMBDA_FUNCTION_NAME']}
)['Plaintext'].decode('utf-8')
def lambda_handler(event, context):
# TODO handle the event here
pass
Questions:
Can I include this in the same .py file which contents my lambda function?
This shows the decryption of a single variable. I have multiple variables how do I implement for that so that all are decrypted in one go?
EncryptionContext={'LambdaFunctionName': os.environ['AWS_LAMBDA_FUNCTION_NAME']}
)['Plaintext'].decode('utf-8')
What is {'LambdaFunctionName': os.environ['AWS_LAMBDA_FUNCTION_NAME']}
It depends on the size of your lambda script, really. If it's a really small script with very few decrypt calls, you can have them inside your Lambda function. Or else, if you want to separate decryption calls for better readability and manageability you can either create a function or have decryption in a different .py file and import it in the lambda function.
You can not decrypt multiple ciphers using the decrypt call. You might've to call decrypt multiple times to accomplish this. Otherwise, you can pack the entire content as a comma-separated list or JSON encoded array and store the encrypted value in the environment variable. This is not a recommended way to go. Highly recommended alternative solution will be to use the SSM parameter store to encrypt and store the sensitive strings. SSM get_parameters allows to get multiple parameters in a single API call. These links might help Parameter store - SecureString Parameters - boto3 ssm get_parameters
EncryptionContext is non-secret data to support additional authentication, only applicable for symmetric encryptions. If you encrypt some data using a key-value pair of EncryptionContext, you must provide the exact EncryptionContext to decrypt the same. In your example, it simply means that you've specified EncryptionContext as {'LambdaFunctionName': '<your-function-name>'}, so in order to decrypt, you must provide the exact EcryptionContext while calling decrypt.
EncryptionContext={'LambdaFunctionName': os.environ['AWS_LAMBDA_FUNCTION_NAME']} )['Plaintext'].decode('utf-8')
This is a boto3 kms decryption option.
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kms.html#KMS.Client.decrypt
so. If you want to use this option, first add the "Encryption Context" option when encrypting.
For example, for the AWS CLI
aws kms encrypt --key-id alias/SlackKey --plaintext "hooks.slack.com/services/xxxxxxxxx" --region ap-northeast-2 --encryption-context "LambdaFunctionName=[you function name]"
As above, you can attach options when encrypting.
The key is
--encryption-context "LambdaFunctionName=[you function name]"
A web application contains sensitive data of the users. Neither the operator of the web application nor the hosting provider should be able to see this data. Therefore I wanted to store these data in the DB encrypted with the entrance password of the users.
dataInDB = encrypt (rawData, user password)
With this strategy it is however not possible to implement the usual use case for password recovery: Since usually only the hash value of the password is stored by the web app, the application cannot send the old, forgotten password to the user. And with the assignment of a new, coincidental password the encrypted data in the DB are no longer readable.
Is there any other solution ?
A possible solution (I am not responsible for any destruction):
When encrypting sensitive data, don't use the user's password as the key. Rather, derive the key from the user's password (preferably using a standard algorithm such as PBKDF2). Just in case the user forgets their password, you can keep a copy of this derived key (encrypted using a different key derived from the user's answer). If the user forgets their password, they can answer their security question. Only the correct answer will decrypt the original password key (not the original password). This affords you the opportunity to re-encrypt the sensitive information.
I will demonstrate using (Python-esque) pseudo code, but first let's look at a possible table for the users. Don't get caught up in the columns just yet, they will become clear soon...
CREATE TABLE USERS
(
user_name VARCHAR,
-- ... lots of other, useful columns ...
password_key_iterations NUMBER,
password_key_salt BINARY,
password_key_iv BINARY,
encrypted_password_key BINARY,
question VARCHAR,
answer_key_iterations NUMBER,
answer_key_salt BINARY
)
When it comes time to register a user, they must provide a question and answer:
def register_user(user_name, password, question, answer):
user = User()
# The question is simply stored for later use
user.question = question
# The password secret key is derived from the user's password
user.password_key_iterations = generate_random_number(from=1000, to=2000)
user.password_key_salt = generate_random_salt()
password_key = derive_key(password, iterations=user.password_key_iterations, salt=user.password_key_salt)
# The answer secret key is derived from the answer to the user's security question
user.answer_key_iterations = generate_random_number(from=1000, to=2000)
user.answer_key_salt = generate_random_salt()
answer_key = derive_key(answer, iterations=user.answer_key_iterations, salt=user.answer_key_salt)
# The password secret key is encrypted using the key derived from the answer
user.password_key_iv = generate_random_iv()
user.encrypted_password_key = encrypt(password_key, key=answer_key, iv=user.password_key_iv)
database.insert_user(user)
Should the user forget their password, the system will still have to ask the user to answer their security question. Their password cannot be recovered, but the key derived from the password can be. This allows the system to re-encrypt the sensitive information using the new password:
def reset_password(user_name, answer, new_password):
user = database.rerieve_user(user_name)
answer_key = derive_key(answer, iterations=user.answer_key_iterations, salt=user.answer_key_salt)
# The answer key decrypts the old password key
old_password_key = decrypt(user.encrypted_password_key, key=answer_key, iv=user.password_key_iv)
# TODO: Decrypt sensitive data using the old password key
new_password_key = derive_key(new_password, iterations=user.password_key_iterations, salt=user.password_key_salt)
# TODO: Re-encrypt sensitive data using the new password key
user.encrypted_password_key = encrypt(new_password_key, key=user.answer_key, iv=user.password_key_iv)
database.update_user(user)
Of course, there are some general cryptographic principles not explicitly highlighted here (cipher modes, etc...) that are the responsibility of the implementer to familiarize themselves with.
Hope this helps a little! :)
Update courtesy of Eadwacer's comment
As Eadwacer commented:
I would avoid deriving the key directly from the password (limited entropy and changing the password will require re-encrypting all of the data). Instead, create a random key for each user and use the password to encrypt the key. You would also encrypt the key using a key derived from the security questions.
Here is a modified version of my solution taking his excellent advice into consideration:
CREATE TABLE USERS
(
user_name VARCHAR,
-- ... lots of other, useful columns ...
password_key_iterations NUMBER,
password_key_salt BINARY,
password_encrypted_data_key BINARY,
password_encrypted_data_key_iv BINARY,
question VARCHAR,
answer_key_iterations NUMBER,
answer_key_salt BINARY,
answer_encrypted_data_key BINARY,
answer_encrypted_data_key_iv BINARY,
)
You would then register the user as follows:
def register_user(user_name, password, question, answer):
user = User()
# The question is simply stored for later use
user.question = question
# The randomly-generated data key will ultimately encrypt our sensitive data
data_key = generate_random_key()
# The password key is derived from the password
user.password_key_iterations = generate_random_number(from=1000, to=2000)
user.password_key_salt = generate_random_salt()
password_key = derive_key(password, iterations=user.password_key_iterations, salt=user.password_key_salt)
# The answer key is derived from the answer
user.answer_key_iterations = generate_random_number(from=1000, to=2000)
user.answer_key_salt = generate_random_salt()
answer_key = derive_key(answer, iterations=user.answer_key_iterations, salt=user.answer_key_salt)
# The data key is encrypted using the password key
user.password_encrypted_data_key_iv = generate_random_iv()
user.password_encrypted_data_key = encrypt(data_key, key=password_key, iv=user.password_encrypted_data_key_iv)
# The data key is encrypted using the answer key
user.answer_encrypted_data_key_iv = generate_random_iv()
user.answer_encrypted_data_key = encrypt(data_key, key=answer_key, iv=user.answer_encrypted_data_key_iv)
database.insert_user(user)
Now, resetting a user's password looks like this:
def reset_password(user_name, answer, new_password):
user = database.rerieve_user(user_name)
answer_key = derive_key(answer, iterations=user.answer_key_iterations, salt=user.answer_key_salt)
# The answer key decrypts the data key
data_key = decrypt(user.answer_encrypted_data_key, key=answer_key, iv=user.answer_encrypted_data_key_iv)
# Instead of re-encrypting all the sensitive data, we simply re-encrypt the password key
new_password_key = derive_key(new_password, iterations=user.password_key_iterations, salt=user.password_key_salt)
user.password_encrypted_data_key = encrypt(data_key, key=new_password_key, iv=user.password_encrypted_data_key_iv)
database.update_user(user)
Hopefully my head is still functioning clearly tonight...