I have data that was encrypted in PHP as follows:
mcrypt_encrypt(MCRYPT_RIJNDAEL_256, SECRET, $data, MCRYPT_MODE_CBC, $iv)
I need to decrypt this data in a Python 3 application. I am trying to use PyCrypto but I am open to other libraries. I expect the following to work:
decryptor = AES.new(key, mode, IV=IV)
plain = decryptor.decrypt(ciphertext)
My initialization vector is 32 bytes, and the following exception is thrown:
ValueError: IV must be 16 bytes long
How can I set PyCrypto to use a 32 byte initialization vector and 32 byte block size?
Alternatively, is there a different library that I can use to decrypt the data?
Thanks to the comments I implemented a suitable solution. I modified rijndael.py in the linked duplicate question to accept bytes rather than strings. I then use it as follows to decrypt 32-byte blocks with the 32-byte initialization vectors.
from rijndael import rijndael
iv = b'myInitializationVectorfoobarfoob'
key = b'myKeyfoobarfoobarfoobarfoobarfoo'
text = b'myCipherTextFoobarfoobarfoobarfo'
r = rijndael(key, block_size=32)
plaintext = r.decrypt(text)
l = ''.join([chr(a ^ b) for a, b in zip(plaintext.encode('latin-1'), iv)])
print(l)
Note that using this rather than PyCrypto is only necessary because libmcrypt incorrectly sets the data block sizes, and thus the initialization vector sizes, to be equal to the key sizes. As far as I understand, data block sizes should always be 128 bits for AES-Rijndael.
Related
I know this topic have a couple answer, but I have problem about details key length.
I would like to encryption data in AES algorithm with CTR. The encryption will have length of 256 bits.
If I do encryption data by key length 256 and IV length 16, I get a error Invalid key length. I thought IV must be the same key length. I change length of IV to 256, but I get an error Invalid iv length. I found out IV must be 16 bits long.
My code works only if my key is 16 bits and my IV is 16 bits long. So my code didn't encrypt with length of 256 bits.
My code
const crypto = require('crypto');
let key = crypto.randomBytes(16).toString('hex'); // Key is static
let iv = crypto.randomBytes(16);
let cipher = crypto.createCipheriv('aes-256-ctr', process.env.KEY, iv);
let encrypte = cipher.update("Example_data", 'utf-8', 'hex');
encrypte += cipher.final('hex');
What should I do to encrypt data with a key length of 256 bits?
First of all: AES uses a 16 bytes IV (corresponding to the AES block size) and a 16, 24 or 32 bytes key. In the posted code aes-256-ctr is specified, so a 256 bits = 32 bytes key is needed.
And this is exactly the size of the key applied, not 16 bytes as you assume, which is why the code is executed in the first place (at least if the undefined process.env.KEY is replaced by key): crypto.randomBytes(16) creates a 16 bytes buffer and toString('hex') converts it to a hex string, doubling the size to 32 bytes (since 1 byte in the buffer is represented by 2 characters (and thus 2 bytes) in the string). So technically, with UTF-8 encoding (the default for crypto.createCipheriv()), this results in a 32 bytes key which can be used for aes-256-ctr. However, each byte in this key can only take 16 values (corresponding to the digits 0-9 and a-f), which corresponds to 1632 = 25616 = 2128 possible values and thus not 256, but only 128 bit security strength.
You prevent the reduction of the security strength by using directly the binary data for the key, so crypto.randomBytes(32). Here each byte can take 256 different values, which corresponds for a 32 bytes key to 25632 = 2256 possible values and thus a 256 bit security strength. If the key is hex encoded, it must not be UTF8 encoded, but hex decoded.
If the key is derived e.g. using a key derivation function, the hex encoding can cause another problem across platforms. If one platform applies uppercase letters for hex encoding and the other lowercase letters, different keys will result.
Long story short: Avoid using the UTF8 encoded data of a hex string for a key (or at least be aware of the implications).
I'm attempting to replicate some python encryption code in node.js using the built in crypto library. To test, I'm encrypting the data using the existing python script and then attempting to decrypt using node.js.
I have everything working except for one problem, doing the decryption results in a truncated initial decrypted result unless I grab extra data, which then results in a truncated final result.
I'm very new to the security side of things, so apologize in advance if my vernacular is off.
Python encryption logic:
encryptor = AES.new(key, AES.MODE_CBC, IV)
<# Header logic, like including digest, salt, and IV #>
for rec in vect:
chunk = rec.pack() # Just adds disparate pieces of data into a contiguous bytearray of length 176
encChunk = encryptor.encrypt(chunk)
outfile.write(encChunk)
Node decryption logic:
let offset = 0;
let derivedKey = crypto.pbkdf2Sync(secret, salt, iterations, 32, 'sha256');
let decryptor = crypto.createDecipheriv('aes-256-cbc', derivedKey, iv);
let chunk = data.slice(offset, (offset + RECORD_LEN))
while(chunk.length > 0) {
let clearChunk = decryptor.update(chunk);
// unpack clearChunk and do something with that data
offset += RECORD_LEN;
chunk = data.slice(offset, offset + RECORD_LEN);
}
I would expect my initial result to print something like this to hex:
54722e34d8b2bf158db6b533e315764f87a07bbfbcf1bd6df0529e56b6a6ae0f123412341234123400000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
And it gets close, expect it cuts off the final 16 bytes (in example above the final 32 "0's" would be missing). This shifts all following decryptions by those 16 bytes, meaning those 32 "0's" are added to the front of the next decrypted chunk.
If I add 16 bytes to the initial chunk size (meaning actually grab more data, not just shift the offset) then this solves everything on the front end, but results in the final chunk losing it's last 16 bytes of data.
One thing that seems weird to me: The initial chunk has a length of 176, but the decrypted results has a length of 160. All other chunks have lengths 176 before and after decryption. I'm assuming I'm doing something wrong with how I'm initializing the decryptor which is causing it to expect an extra 16 bytes of data at the beginning, but can't for the life of me figure out what.
I must be close since the decrypted data is correct, minus the mystery shifting, even when reading in large amounts of data. Just need to figure out this final step.
Short version based on your updated code: if you are absolutely certain that every block will be 176 bytes (i.e. a multiple of 16), then you can add cipher.setAutoPadding(false) to your Node code. If that's not true, or for more about why, read on.
At the end of your decryption, you need to call decryptor.final to get the final block.
If you have all the data together, you can decrypt it in one call:
let clearChunk = decryptor.update(chunk) + decryptor.final()
update() exists so that you can pass data to the decryptor in chunks. For example, if you had a very large file, you may not want a full copy of the encrypted data plus a full copy of the decrypted data in memory at the same time. You can therefore read encrypted data from the file in chunks, pass it to update(), and write out the decrypted data in chunks.
The input data using CBC mode must be a multiple of 16 bytes long. To ensure this, we typically use PKCS7 padding. That will pad out your input data to a multiple of 16. If it's already a multiple of 16 it will add an extra block of 16 bytes. The padding value is the number of padding values. So if your block is 12 bytes long, it will be padded with 04040404. If it's a multiple of 16, then the padding is 16 bytes of 0x10. This padding system lets the decryptor validate that it's removing the right amount of padding. This is likely what's causing your 176/160 issue.
This padding issue is why there's a final() call. The system needs to know which block is the last block so it can remove the padding. So the first call to update() will always return one fewer blocks than you pass in, since it's holding onto it until it knows whether it's the last block.
Looking at your Python code, I think it's not padding at all (most Python libraries I'm familiar with don't pad automatically). As long as the input is certain to be a multiple of 16, that's ok. But the default for Node is to expect padding. If you know that your size will always be a multiple of 16, then you can change the Node side with cipher.setAutoPadding(false). If you don't know for certain that the input size will always be a multiple of 16, then you need to add a pad() call on the Python side for the final block.
My server will be sending many IDs to the browser/user, and, for that session, the user might operate using those IDs. Between users, and between multiple sessions of a single user, I need IDs to be encrypted, so they cannot be traced except within the context of a single session. Due to the number of IDs that will be used in each session, it is not reasonable to dereference or hash them and store lookup tables for each session.
The IDs are effectively UUIDs, unique 128 bit values. The server will encrypt them in the context of a session, and when the user queries using them, and the server can decrypt those values within the context of that same session. I would like for the encrypted output to also be 128 bits in length (for example, so they could be rendered as UUIDs even in their encrypted state). What is the best way for me to achieve this?
This is my sample code, demonstrating that I can encrypt 16 bytes (the size of a block), but the cipher extends it to 2 blocks, doubling the size to 32 bytes, when I finalize it. I think because it is OK for a value to be encrypted the same way twice in the context of a single session, it is acceptable to reuse the same IV for each item; so the server stores a key and IV for the session, and can encrypt and decrypt all of the IDs with those.
async function sampleCrypt() {
const algorithm = 'aes-128-cbc';
crypto.scrypt("samplePassword", "salty", 16, (err, key) => {
const iv: Buffer = crypto.randomBytes(16);
const cipher: crypto.Cipher = crypto.createCipheriv(algorithm, key, iv);
const inbuffer = Buffer.allocUnsafe(16);
inbuffer.writeUInt32BE(1960); // just some sample data
console.log(cipher.update(inbuffer, undefined, 'hex')); // loads the whole buffer in
console.log(cipher.final('hex'));
});
}
sampleCrypt();
/* Sample output:
83134f7dc2f9b175bd70a7dd0512eaf7
9495e0cfceab0439fddc92f3fffa48c2
*/
Please advise if I have made any incorrect assumptions here as well. Thanks!
Block ciphers such as AES require plaintexts whose lengths are an integer multiple of the blocksize (16 bytes for AES). If this isn't the case, padding must be used. NodeJS applies PKCS7 padding by default. Here a complete padding block is appended if the plaintext length is already an integer multiple of the blocksize. This is the reason why in your case a 16 bytes plaintext results in a 32 bytes ciphertext. But since the plaintext is always exactly one block long, there is actually no need for padding. In NodeJS the padding can be disabled with cipher.setAutoPadding(false), so in your case plaintext and ciphertext are both 16 bytes long.
A block cipher only encrypts one block. To encrypt longer plaintexts, an operation mode must be used, e.g. CBC as in the posted code. Generally, these operation modes use an initialization vector (IV) whose size is equal to the blocksize (16 bytes for AES). The IV must meet certain conditions, e.g. a key/IV pair may only be used once. Since the IV isn't secret it's usually placed before the ciphertext. In your case, this would result in a 32 bytes result (IV + ciphertext). The condition mentioned also means that the concept you use (one key/IV pair for all encryptions) is inherently insecure.
An operating mode that doesn't require an IV is ECB. ECB generates the same ciphertext for the same plaintext, which generallay allows conclusions from the ciphertext to the plaintext. This problem doesn't exist for a mode with an IV. Therefore ECB is more insecure compared to a mode with an IV. However, the severity of this insecurity ultimately depends on the characteristics of the plaintext and the particular application, and the respective requirements determine whether this disadvantage is tolerable or not. 1-block plaintexts containing a GUID are less vulnerable in this respect than multi-block plaintexts with some message content, so ECB may be an option here.
With disabled padding and ECB mode a 16 bytes plaintext results in a 16 bytes ciphertext, as the following TypeScript code demonstrates:
import * as crypto from "crypto";
const algorithm:string = 'aes-128-ecb';
const key:Buffer = crypto.randomBytes(16);
// Encryption
const plaintextEnc:Buffer = Buffer.from('0123456789012345');
const cipherEnc:crypto.Cipher = crypto.createCipheriv(algorithm, key, null);
cipherEnc.setAutoPadding(false);
const ciphertext:Buffer = Buffer.concat([cipherEnc.update(plaintextEnc), cipherEnc.final()]);
console.log(ciphertext.toString('hex'));
// Decryption
const cipherDec:crypto.Decipher = crypto.createDecipheriv(algorithm, key, null);
cipherDec.setAutoPadding(false);
const plaintextDec:Buffer = Buffer.concat([cipherDec.update(ciphertext), cipherDec.final()]);
console.log(plaintextDec.toString('hex'));
If the limitation to 16 bytes is dropped, GCM would be a recommendable mode that provides besides confidentiality also authenticity and integrity. GCM uses a 12 bytes IV (nonce) and generates a tag (typically 16 bytes) that is used for authentication. In your case, the result (IV + ciphertext + tag) would have a length of 44 bytes. Note that if a key/IV pair is used more than once for GCM, security is lost.
I'm attempting to perform encryption in Node.js 7.5.0 using the 'des-cbc' algorithm. According to RFC 1423, this algorithm requires a 64-bit cryptographic key, and a 64-bit initialization vector.
I'm trying to use a key and iv composed of 8 Latin-1 characters; however, Node is saying, "Error: Invalid IV length". Here's some example code:
let crypto = require('crypto');
let key = '\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8',
iv = '\xb8\xb7\xb6\xb5\xb4\xb3\xb2\xb1';
let cipher = crypto.createCipheriv('des-cbc', Buffer.from(key), Buffer.from(iv));
If I change the iv to 8 ASCII characters, then Node is saying, "Error: Invalid key length":
let crypto = require('crypto');
let key = '\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8',
iv = 'abcdefgh';
let cipher = crypto.createCipheriv('des-cbc', Buffer.from(key), Buffer.from(iv));
But if both the key and iv are 8 ASCII characters, it works:
let crypto = require('crypto');
let key = 'hgfedcba',
iv = 'abcdefgh';
let cipher = crypto.createCipheriv('des-cbc', Buffer.from(key), Buffer.from(iv));
Why can't the Latin-1 characters be used for key and iv?
Solution
You should either use
Buffer.from('\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8', 'binary')
or even more cleaner as Maarten Bodewes points out
Buffer.from([0xb1, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7, 0xb8'])
The same goes for your IV.
Reason
Buffer.from('\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8').length results in 16 which is not a valid length for a DES key. DES expects exactly 64 bits or 8 bytes as a key. That is also why DES is very insecure. The key size is simply too small.
The reason the above Buffer has a size of 16 bytes instead of 8 bytes is that the default encoding is UTF-8. If a code point is larger than 127 (decimal) or 0x7F (hexadecimal) it will encode into at least two bytes instead of one. Each and every code point (character) of your key is larger than 0x7F. So, each of them is encoded into two bytes.
Things to think about
Don't use DES nowadays. It only provides 56 bit of security. AES would be a much better, because it's more secure with the lowest key size of 128 bit. There is also a practical limit on the maximum ciphertext size with DES. See Security comparison of 3DES and AES.
The IV must be unpredictable (read: random). Don't use a static IV, because that makes the cipher deterministic and therefore not semantically secure. An attacker who observes ciphertexts can determine when the same message prefix was sent before. The IV is not secret, so you can send it along with the ciphertext. Usually, it is simply prepended to the ciphertext and sliced off before decryption.
The key is supposed to be indistinguishable from random noise. It is best to just generate it randomly and use it in your code in encoded form.
It is better to authenticate your ciphertexts so that attacks like a padding oracle attack are not possible. This can be done with authenticated modes like GCM or EAX, or with an encrypt-then-MAC scheme.
I am trying to do some experimenting in encrypting and decrypting using PyCrypto.AES when I try to decrypt it gives me TypeError: 'str' does not support the buffer interface
I found some solutions where I have to encode or use string, but I couldn't figure how to use it.
AESModule.py
from Crypto.Cipher import AES
#base64 is used for encoding. dont confuse encoding with encryption#
#encryption is used for disguising data
#encoding is used for putting data in a specific format
import base64
# os is for urandom, which is an accepted producer of randomness that
# is suitable for cryptology.
import os
def encryption(privateInfo,secret,BLOCK_SIZE):
#32 bytes = 256 bits
#16 = 128 bits
# the block size for cipher obj, can be 16 24 or 32. 16 matches 128 bit.
# the character used for padding
# used to ensure that your value is always a multiple of BLOCK_SIZE
PADDING = '{'
# function to pad the functions. Lambda
# is used for abstraction of functions.
# basically, its a function, and you define it, followed by the param
# followed by a colon,
# ex = lambda x: x+5
pad = lambda s: s + (BLOCK_SIZE - len(s) % BLOCK_SIZE) * PADDING
# encrypt with AES, encode with base64
EncodeAES = lambda c, s: base64.b64encode(c.encrypt(pad(s)))
# generate a randomized secret key with urandom
#secret = os.urandom(BLOCK_SIZE)
print('Encryption key:',secret)
# creates the cipher obj using the key
cipher = AES.new(secret)
# encodes you private info!
encoded = EncodeAES(cipher, privateInfo)
print('Encrypted string:', encoded)
return(encoded)
def decryption(encryptedString,secret):
PADDING = '{'
DecodeAES = lambda c, e: c.decrypt(base64.b64decode(e)).rstrip(PADDING)
#Key is FROM the printout of 'secret' in encryption
#below is the encryption.
encryption = encryptedString
cipher = AES.new(secret)
decoded = DecodeAES(cipher, encryption)
print(decoded)
test.py
import AESModule
import base64
import os
BLOCK_SIZE = 16
key = os.urandom(BLOCK_SIZE)
c = AESRun2.encryption('password',key,BLOCK_SIZE)
AESRun2.decryption(c,key)
Strings (str) are text. Encryption does not deal in text, it deals in bytes (bytes).
In practice insert .encode and .decode calls where necessary to convert between the two. I recommend UTF-8 encoding.
In your case since you are already encoding and decoding the ciphertext as base-64 which is another bytes/text conversion, you only need to encode and decode your plaintext. Encode your string with .encode("utf-8") when passing it into the encryption function, and decode the final result with .decode("utf-8") when getting it out of the decryption function.
If you're reading examples or tutorials make sure they are for Python 3. In Python 2 str was a byte string and it was commonplace to use it for both text and bytes, which was very confusing. In Python 3 they fixed it.