I tried decrypting a file from s3 which was uploaded by unload command from redshift with AES symmetric key encryption.
If we use the AWS java sdk to download with aes key given to the s3 client it works fine.But if we try to manually decrypt it after downloading the file it gives javax.crypto.BadPaddingException: Given final block not properly padded error.
The reason for manually decrypting the file is i want to decrypt the file using node.js and as far as i know there is no sdk in node that can do this directly.
Node.js code that i tried:
var AWS = require('aws-sdk');
var fs = require('fs');
var crypto = require('crypto');
var CryptoJS = require("crypto-js");
var algorithm = 'aes256';
var inputEncoding = 'hex';
var outputEncoding = 'utf-8';
var key = "symmetric key base 64"; //prod
var data = fs.readFileSync('/tmp/files/myfile');
console.log(data);
var decipher = crypto.createDecipher(algorithm,key);
var deciphered = decipher.update(data, inputEncoding, outputEncoding);
console.log(deciphered);
deciphered += decipher.final(outputEncoding);
console.log(deciphered);
When i try this i get this error: Error: error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt
So Redshift uses the envelope encryption the same way as the AWS SDK uses envelope encryption to store files on S3. So in order to decrypt the file you should:
Get the encrypted data key and the iv from the S3 object metadata (x-amz-meta-x-amz-key and x-amz-meta-x-amz-iv respectively)
Decrypt that x-amz-meta-x-amz-key value using your symmetric key using AES256 ECB mode => var decipher = crypto.createDecipheriv('AES-128-ECB',key,'');
Then decrypt '0000_part_00' using AES256 CBC mode with iv set to iv from step1 and the key set to the result of step 2. => crypto.createDecipheriv('aes-128-cbc', key, iv)
Remove padding (should be able to use cipher.setAutoPadding(true) if Node.js Crypo, what's the default padding for AES? is correct)
I haven't coded it in nodejs but I have successfully used these steps in Python
step2 step 3+4
Related
I am working out a custom hybrid encryption system. I've got symmetric encryption & asymmetric encryption & decryption all handled server-side. All I need to work out now is symmetric decryption.
I got some trouble because my client is sending symmetric key, iv & data all in string format (after asymmetric decryption), but CryptoJS is very touchy with it's encoding. It's also very confusing and vague as far as documentation goes- at least for a relatively new developer. I just can't figure out what encoding CryptoJS wants for each argument. I figure I should have guessed right by now, but no.
Docs
Some help I've gotten previously
I'm requesting help getting the encoding right so that I can decrypt with the following. And thanks a lot for any assistance.
Example of data after asymmetric decryption as per below (throw away keys):
symmetricKey: bDKJVr5wFtQZaPrs4ZoMkP2RjtaYpXo5HHKbzrNELs8=,
symmetricNonce: Z8q66bFkbEqQiVbrUrts+A==,
dataToReceive: "hX/BFO7b+6eYV1zt3+hu3o5g61PFB4V3myyU8tI3W7I="
exports.transportSecurityDecryption = async function mmTransportSecurityDecryption(dataToReceive, keys) {
const JSEncrypt = require('node-jsencrypt');
const CryptoJS = require("crypto-js");
// Asymmetrically decrypt symmetric cypher data with server private key
const privateKeyQuery = new Parse.Query("ServerPrivateKey");
const keyQueryResult = await privateKeyQuery.find({useMasterKey: true});
const object = keyQueryResult[0];
const serverPrivateKey = object.get("key");
const crypt = new JSEncrypt();
crypt.setPrivateKey(serverPrivateKey);
let decryptedDataString = crypt.decrypt(keys);
let decryptedData = JSON.parse(decryptedDataString);
// Symmetrically decrypt transit data
let symmetricKey = decryptedData.symmetricKey;
let symmetricNonce = decryptedData.symmetricNonce;
// Works perfectly till here <---
var decrypted = CryptoJS.AES.decrypt(
CryptoJS.enc.Hex.parse(dataToReceive),
CryptoJS.enc.Utf8.parse(symmetricKey),
{iv: CryptoJS.enc.Hex.parse(symmetricNonce)}
);
return decrypted.toString(CryptoJS.enc.Utf8);
}
You are using the wrong encoders for data, key and IV. All three are Base64 encoded (and not hex or Utf8). So apply the Base64 encoder.
The ciphertext must be passed to CryptoJS.AES.decrypt() as a CipherParams object or alternatively Base64 encoded, which is implicitly converted to a CipherParams object.
When both are fixed, the plain text is: "[\"001\",\"001\"]".
var symmetricKey = "bDKJVr5wFtQZaPrs4ZoMkP2RjtaYpXo5HHKbzrNELs8="
var symmetricNonce = "Z8q66bFkbEqQiVbrUrts+A=="
var dataToReceive = "hX/BFO7b+6eYV1zt3+hu3o5g61PFB4V3myyU8tI3W7I="
var decrypted = CryptoJS.AES.decrypt(
dataToReceive, // pass Base64 encoded
//{ciphertext: CryptoJS.enc.Base64.parse(dataToReceive)}, // pass as CipherParams object, works also
CryptoJS.enc.Base64.parse(symmetricKey),
{iv: CryptoJS.enc.Base64.parse(symmetricNonce)}
);
console.log(decrypted.toString(CryptoJS.enc.Utf8));
<script src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/4.1.1/crypto-js.min.js"></script>
I have some encrypted data in my database
I did it few years ago using crypto.createCipher
const cipher = crypto.createCipher('aes192', password);
As createCipher and createDecipher is deprecated, I would like to change to createCipheriv and createDecipheriv. The problem is that the data I have in my database are encoded without iv.
Is it possible to decode with createDecipheriv data encoded with createDecipher and to generate the same secret with createCipher and createCipheriv.
I tried setting the iv to null but not working
Thanks, because the database migration is an heavy work !
I tried setting the iv to null but not working
This is because this method didn’t allow for passing an initialization vector (IV), and instead derived the IV from the key using the OpenSSL EVP_BytesToKey derivation function, using a null salt meaning that the IV would be deterministic for a given key which is an issue for ciphers with counter mode like CTR, GCM and CCM.
Looking at your code:
const cipher = crypto.createCipher('aes192', password);
If you want to make this code backwards compatible, you need to call OpenSSL’s EVP_BytesToKey function yourself, typically through evp_bytestokey module which makes it available in JS userland.
Is it possible to decode with createDecipheriv data encoded with createDecipher and to generate the same secret with createCipher and createCipheriv.
Yes, you can. check out my example code here:
const crypto = require('crypto');
const EVP_BytesToKey = require('evp_bytestokey')
const ALGO = 'aes192';
const password = 'Your_Password_Here';
const KEY_SIZE = 24;
function decrypt_legacy_using_IV(text) {
const result = EVP_BytesToKey(
password,
null,
KEY_SIZE * 8, // byte to bit size
16
)
let decipher = crypto.createDecipheriv(ALGO, result.key, result.iv);
let decrypted = decipher.update(text, 'hex','utf8') + decipher.final('utf8');
return decrypted.toString();
}
function encrypt_legacy_using_IV(text) {
const result = EVP_BytesToKey(
password,
null,
KEY_SIZE * 8, // byte to bit size
16
)
var cipher = crypto.createCipheriv(ALGO, result.key, result.iv);
var encrypted = cipher.update(text, 'utf8', 'hex') + cipher.final('hex');
return encrypted.toString();
}
For complete running example, clone node-snippets and run node apogee-legacy-crypto-cipheriv.js.
However the reason this function is deprecated in the first place is because you shouldn’t use it, and instead use a random unpredictable IV, which requires you to change your code to something like this:
const iv = crypto.randomBytes(16)
const cipher = crypto.createCipheriv('aes192', password, iv)
Here, for AES-192 in CBC mode (aes192 being aliased to AES-192-CBC by OpenSSL), the IV size is expected to be the same as the block size, which is always 16 bytes.
In order to decrypt the message, you will need the IV as well. Typically you’d store the IV together with the message, as the important part is for the IV to not be predictable ahead of time.
I'm having trouble finding some information. Does anyone know if the value returned from cipher.getAuthTag() (--> returns MAC) can be publicly visible?
TL;DR
Can a message authentication code be publicly visible, or does this need to be kept secret like a password?
Some background, I am trying to encrypt a file. I found this stackoverflow question and answer that helped me get started. https://stackoverflow.com/a/27345933/11070228
After doing some research in the nodejs documentation, I found that the answer uses a deprecated function. createCipher. The new function to use should be createCipheriv.
So, to use the new createCipheriv, I used the documentation to write a new encryption and decryption function, similar to the one in the post using the new createCipheriv function. After writing the decryption function, I got an error that was
Error: Unsupported state or unable to authenticate data
After googling that issue, it led me here to this github post. In a nutshell, it said that the authTag generated with the cipher is needed to decrypt the file.
I did not know what this authTag was, and neither did anyone I knew. So I started googling that and it let me to this blogpost. It states
The authTag is the message authentication code (MAC) calculated during the encryption.
And here is a wikipedia article on what a message authentication code is.
So. Here is my question. Can a message authentication code be publicly visible, or does this need to be kept secret like a password?
My code, not as relevant, but might help someone create the encryption and decryption using createCipheriv and createDecipheriv.
Encryption
const crypto = require('crypto');
const fs = require('fs');
// const iv = crypto.randomBytes(32).toString('hex');
// EDIT - based on #President James K. Polk. The initialization vector should be 12 bytes long
// const iv = crypto.randomBytes(6).toString('hex');
// EDIT - based on #dsprenkels. I misunderstood #President James K. Polk
const iv = crypto.randomBytes(12).toString('hex');
const privateKey = 'private key that is 32 byte long';
const cipher = crypto.createCipheriv('aes-256-gcm', privateKey, iv);
const filename = 'somefile.txt';
const encFilename = 'somefile.txt.enc';
const unencryptedInput = fs.createReadStream(filename);
const encryptedOutput = fs.createWriteStream(encFilename);
unencryptedInput.pipe(cipher).pipe(encryptedOutput);
encryptedOutput.on('finish', () => {
const authTagAsHex = cipher.getAuthTag().toString('hex'); // <-- can this be public
console.log(authTagAsHex);
});
Decryption
const crypto = require('crypto');
const fs = require('fs');
// const publicIV = 'same iv generated during encryption crypto.randomBytes(32).toString("hex")';
// EDIT - based on #President James K. Polk. The initialization vector should be 12 bytes long
// const publicIV = 'same iv generated during encryption crypto.randomBytes(6).toString("hex")';
// EDIT - based on #dsprenkels. I misunderstood #President James K. Polk
const publicIV = 'same iv generated during encryption crypto.randomBytes(12).toString("hex")';
const authTag = 'same authKey generated from cipher.getAuthTag().toString("hex")';
const privateKey = 'private key that is 32 byte long';
const decipher = crypto.createDecipheriv('aes-256-gcm', privateKey, publicIV);
decipher.setAuthTag(Buffer.from(authTag, 'hex'));
const filename = 'somefile.txt';
const encFilename = 'somefile.txt.enc';
const readStream = fs.createReadStream(encFilename);
const writeStream = fs.createWriteStream(filename);
readStream.pipe(decipher).pipe(writeStream);
Yes. The MAC is considered public.
In general, message authentication codes are considered public. A message authentication code authenticates the (encrypted) message under the key that you provided. On other words, it is used by the receiver to check if the ciphertext did not change during transmission. In your situation, as long as the key remains secret, the attacker does not have any use for the MAC.
The MAC is normally put next to the ciphertext when the ciphertext is stored (just as the IV).
By the way, in your case you are were randomly generating the IV. That is fine, but beware that the amount of messages that can be safely encrypted under the same key is quite small. If an IV is used for multiple message (even once!) the complete security of this scheme breaks down. Really, you probably want this:
const iv = crypto.randomBytes(12);
I am encrypting some text with the following encrypt function on Heroku:
const crypto = require('crypto');
// function to encrypt data ....
function encrypt(KEY, text){
const cipher = crypto.createCipher('aes192', KEY);
var encrypted = cipher.update(text,'utf8', 'hex');
encrypted += cipher.final('hex');
return encrypted;
}
// function to decryt data..............
function decrypt(KEY, text){
const decipher = crypto.createDecipher('aes192', KEY)
var decrypted = decipher.update(text,'hex','utf8')
decrypted += decipher.final('utf8');
return decrypted;
}
It then saves the text I encrypted to MongoDb server. I read the encrypted value and try to decrypt it on the local machine, but get a digital envelope routines:EVP_DecryptFinal_ex:bad decrypt. I have spent so much time trying to figure out what is the issue.
Both in Heroku and locally I am using the same key. If I try this code locally (i.e. encrypt and decrypt locally), then everything works as expected.
Do you have an idea of what might be going wrong?
I have noticed that heroku server is in United States, while I am in UK. Does the timezone play any role in here?
I am facing up a bit of a wall trying to send encrypted data over to a remote server using the NodeJs crypto module.
According to the API docs, the payload needs to be encrypted using the AES-256 algorithm with randomly generated KEY and IV.
The randomly generated KEY and IV [above] are then encrypted with a shared private key using the RSAES-PKCS1-V1_5 standard.
Finally, the encrypted payload is signed with the private key, using the RSASSA-PKCS1-V1_5 signature scheme and then SHA1 hashed.
Once that is done, I compose an HTTP request and pass the encrypted KEY, IV, encrypted payload and the signature to the remove server.
I am not an expert when it comes to cryptography, so I am convinced that I am doing something wrong somewhere.
The server is able to verify the signature, which gives me confidence that there is no problem with the shared private key file.
However, the server fails to decrypt the encrypted KEY and IV which are needed to decrypt the payload.
I am using the code below for testing:
const crypto = require('crypto');
const fs = require('fs');
//Generate random KEY and IV
const randomKey = crypto.randomBytes(32);
const randomIV = crypto.randomBytes(16);
//Load private key from disk
const privateKey = fs.readFileSync(__dirname + '/private.key');
//Get data payload that should be encrypted with AES-256
const payload = 'Payload to be sent';
//Encrypt payload with AES-256
const cipher = crypto.createCipheriv('aes-256-cbc', randomKey, randomIV);
const encryptedPayload = Buffer.concat([cipher.update(payload), cipher.final()]);
//Sign the encrypted payload using the RSASSA-PKCS1-V1_5 algorithm
const signer = crypto.createSign('RSA-SHA1');
signer.update(encryptedPayload);
signer.end();
const signature = signer.sign(privateKey); //Sign with the private key
//Encrypt both KEY and IV
const encryptOptions = {
key: privateKey,
padding: constants.RSA_PKCS1_PADDING
}
const encryptedKey = crypto.publicEncrypt(encryptOptions, randomKey);
const encryptedIV = crypto.publicEncrypt(encryptOptions, randomIV);
//A function that encodes Buffer type data to base64
const encode = buffer => buffer.toString('base64');
const request = {
encryptedKey: encode(encryptedKey),
encryptedIV: encode(encryptedIV),
encryptedPayload: encode(encryptedPayload),
signature: encode(signature)
};
const endPoint = require('./end-point');
endPoint.call(request);
//-> Server successfully verifies signature but fails to decrypt encrypted KEY and IV
Could someone point me to where I am doing it wrong?