I am beginnging to wonder if the implementation of AES is different across libraries..
Currently i have a plaintext encrypted with PyCrypto.
Im trying to decrypt the ciphertext with Node.js's Crypto Library..
Basically with PyCrypto..
im using AES-128-CBC with a random generated IV. (which decrypts perfectly in PyCrypto)
However..
On Node.js im doing this
var buf = new Buffer(ciphertext)
var decipher = crypto.createDecipher('aes-128-cbc',aeskey)
buf = decipher.update(buf,'binary', 'binary')
buf += decipher.final('binary')
Which spits out a bunch of Garbage.... ( changing 'binary' to hex/utf8 doesnt help)
As i am using CBC (Cipher Block Chaining)...
i am prepending the IV to the beginning of the ciphertext (16 blocks)..
In PyCrypto this works perfectly, similarly to the specification of PGP, CFB usage..
Does anyone know for what reason this is not working???
Am i expecting too much of Node.js's standard libraries?
Documentation does not mention this, but aeskey you're passing to crypto.createDecipher is not the key, but a password, handled to OpenSSL's EVP_BytesToKey function.
To pass the actual raw key data one should use (presently undocumented) crypto.createDecipheriv(cipher, key, iv) function. This applies to ECB mode too, even though there's no IV in ECB.
If this fails, I think, the first step in debugging would be to try with AES KATs to see whenever the decryption code is correct.
I've tripped on a similar issue here: https://github.com/joyent/node/issues/1318
AES is a rijndael standard. It shouldn't be different. You should look into data types and default settings that are hidden. Something must be set different between the two. The key sizes might be different as 128 bit "hello" is padded with zeros I think and a smaller key would start with "hello" but have a smaller padding, therefore different.
The short answer to your question is: Yes, AES is the same in PyCrypto and Node.js' crypto module. Node's crypto is just a wrapper around openssl on your system, and PyCrypto is interoperable with OpenSSL (see http://lists.dlitz.net/pipermail/pycrypto/2010q4/000301.html).
Having said that, there are definitely bugs in the Node crypto module (though I've only experienced problems with base64 encoding, myself). So whether it's a bug or not, the problems you're experiencing are almost certainly happening in the data encoding/decoding stages.
What does your ciphertext look like? Is it a hexadecimal string? If so, then you need to do
buf = decipher.update(buf, 'hex', 'binary')
That's not how IV works in Node, you have to use crypto.createDecipheriv(cipher, key, iv) instead, otherwise you get a default baked-in one. Even in PyCrypto you should be using the third argument to AES.new as the IV, not stuffing it into the bytestream.
Make sure you use the same key and IV in both pycrypto and node.js!! Not only that, but make sure you have the same encoding in both ends:
cipher = AES.new(key.decode('hex'), AES.MODE_CBC, iv.decode('hex'))
text = json.dumps(payload)
pad = lambda s: s + (16 - len(s) % 16) * '\x07'
encryptedText = base64.b64encode(cipher.encrypt(pad(text)))
Then in node.js (sorry, no easy access to that code now), also make sure you decode your key and iv to hex
Related
I'm looking for a way to encrypt main password using short key/pin and decrypt it by this pin every time.
I tried to generate hash (sha256) from short key given by user and cut off the hash to desired length, decode it to bytecode and use as Fernet
Piece of code:
pin = self.pin_ent.get()
key: str = hashlib.sha256(pin.encode()).hexdigest()[10:-10]
f = Fernet(key.decode())
but python rise the error
ValueError: Fernet key must be 32 url-safe base64-encoded bytes.
It is unclear what you mean with "desired length". SHA-256 creates a 32 byte hash value (without the hex encoding that you added). You just need to base64url encode it, as the error description suggests.
A PIN is not suitable for encryption purposes as it is too easy to try all possible PIN values, and try to decrypt the stored password. If you'd use a normal-strength password to encrypt the other password (which seems counter-productive, but hey) then SHA-256 is not secure either, you'd have to use a PBKDF such as PBKDF2 to strengthen the password.
I have a file that has been encrypted using openssl using the following command:
openssl enc -in data -out encrypted -e -aes256 -k myverystrongpassword
Where data is the original file and encrypted is the encrypted file.
I tried various ways using crypto library but nothing seems to work. I understand that the password needs to be converted into a key so maybe I am doing something wrong there. Looked all over for a solution but nothing seems to work.
The posted OpenSSL statement uses a key derivation function EVP_BytesToKey() to derive a 32 bytes key and a 16 bytes IV from the password in combination with a random 8 bytes salt.
The ciphertext corresponds to the concatenation of the ASCII encoding of Salted__, followed by the salt and finally by the actual ciphertext.
As you already know according to your comment, EVP_BytesToKey() uses a digest for which OpenSSL applied MD5 by default in earlier versions and SHA-256 as of version v1.1.0 (the default value can be overridden in the OpenSSL statement with the -md option).
Decryption is possible e.g. with CryptoJS: Due to its OpenSSL compatibility (s. sec. Interoperability) CryptoJS has a built-in implementation of an accessible EVP_BytesToKey() function and additionally allows to explicitly set the digest in the internal EVP_BytesToKey() call during key derivation. This makes it possible to decrypt encryptions that used SHA-256 or MD5 in key derivation.
The following data is the Base64 encoding of a ciphertext generated with the posted OpenSSL statement. The plaintext used was The quick brown fox jumps over the lazy dog. The OpenSSL version applied is v1.1.1i (i.e. SHA-256 is implicitly used in the key derivation):
U2FsdGVkX19W4wmC9dD35X4J66cSvaRaIQpvjDKHrLF9+qYg5VTo5urvExHLXhwf/bE8FXJTQZmKN8ITMJVdqQ==
This ciphertext can be successfully decrypted using the following CryptoJS implementation:
const password = 'myverystrongpassword';
const saltCiphertextB64 = 'U2FsdGVkX19W4wmC9dD35X4J66cSvaRaIQpvjDKHrLF9+qYg5VTo5urvExHLXhwf/bE8FXJTQZmKN8ITMJVdqQ==';
CryptoJS.algo.EvpKDF.cfg.hasher = CryptoJS.algo.SHA256.create(); // default: MD5
const decryptedData = CryptoJS.AES.decrypt(saltCiphertextB64, password);
console.log(decryptedData.toString(CryptoJS.enc.Utf8)); // The quick brown fox jumps over the lazy dog
<script src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/4.1.1/crypto-js.min.js"></script>
Note that the digest in the code must be explicitly specified as SHA-256 since OpenSSL v1.1.1i was used for encryption.
If the encryption was done with an OpenSSL version that uses MD5, the digest in the code must be modified accordingly.
Edit: As noted in the comment, the crypto functions createCipher()/createDecipher() also use EVP_BytesToKey() as key derivation.
However, the following should be noted:
Unlike CryptoJS, it is not possible to specify the digest, i.e. MD5 is used unchangeably. Thus, encryptions that applied SHA-256 for key derivation cannot be decrypted (what applies to the encryptions here).
In contrast to CryptoJS, no salt is used by default. Therefore, salt creation and concatenation (Salted__|<salt>|<cipherext>) during encryption and separation during decryption would have to be implemented additionally. createCipher()/createDecipher() then has to be passed the concatenation of passphrase and salt.
Both functions are deprecated since version 10.0.0 and should actually not be used.
A more robust approach to decrypt encryptions (with arbitrary digests in key derivation) using the crypto module is to apply createCipheriv()/createDecipheriv() and a port of the required functionality of EVP_BytesToKey() to derive key and IV (various implementations can be found on the net).
Security: EVP_BytesToKey() is deemed to be a vulnerability these days. This is worsened by a low iteration count (like 1, which is used by OpenSSL), a broken digest (like MD5) or a missing salt (as is the default for crypto). Ultimately, this is why createCipher()/createDecipher() are deprecated. Instead of EVP_BytesToKey(), a more reliable key derivation function such as PBKDF2 or the more modern scrypt or Argon2 should be used.
I am using Node.JS and the crypto module. I have a SHA-256 hash in hex string and would like to create a crypto.Hash instance out of it. I only found ways to hash the input string itself, but not to update or create a new hash. Am I missing something from the documentation?
I am looking for something like (for UUID though):
crypto.Hash.from("sha256", "hex", "d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592")
Generally there are not many libraries that do what you ask of them. There are certainly libraries where the internal state can be retrieved and restored such as Bouncy Castle, but as yet I haven't seen it in any JavaScript library. It would be very easy to create though.
Indeed, the 256 bit (total) intermediate values after each block of 512 bits will be used as final output after the last block is hashed. So if you can "restore" those values (i.e. put them in the state) then you could continue hashing after that.
This might not be that useful though, as those values already contain the padding and message size encoded into a 64 bit representation at the end of the block. So if you continue hashing after that, that padding and length will likely be included again, but now with different values.
One trick sometimes used in smart cards is to upload the intermediate values (including number of bits hashed) before the last data to be hashed, and let the smart card perform the padding, the length encoding and the final hash block operation. This is usually performed during signature calculation over large amounts of data (because you really don't want to send a whole document to smart card).
Pretty dumb if you ask me, just directly signing using a pre-calculated hash value is the way forward. Or making sure that the large swath of data is pre-hashed, and the hash is then signed (including another pass of the hash) - that way the entire problem can be avoided without special tricks.
The following small example code hashes a string to a base64- and hex string encoding.
This is the output:
buf: The quick brown fox jumps over the lazy dog
sha256 (Base64): 16j7swfXgJRpypq8sAguT41WUeRtPNt2LQLQvzfJ5ZI=
sha256 (hex): d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592
code:
var crypto = require('crypto');
const plaintext = 'The quick brown fox jumps over the lazy dog';
const buf = Buffer.from(plaintext, 'utf8');
console.log('buf: ' + buf);
const sha256Base64 = crypto.createHash('sha256').update(buf).digest('base64');
console.log('sha256 (Base64): ' + sha256Base64);
const sha256Hex = crypto.createHash('sha256').update(buf).digest('hex');
console.log('sha256 (hex): ' + sha256Hex);
Edit: I misunderstood the question, the task is to run several SHA-256 update calls (e.g. for different strings) and receive a (intermediate) result. Later this result should be used as "input" and the hashing should be proceeded with more update calls, finished with by "final"/"digest" to get the final sha-256 value about all parts.
Unfortunately there seems to be no way as the intermediate value is a final value and there (again) seems to be no way back because the final/digest calls make so additional computations (has to do with the underlying Merkle-Damgård constructions). The only way would be use an own written SHA-256 function and save the state of all internal registers, reset the registers when continuing and get the final value in the end.
An advice could be to grab the source code of a sha-256 implementation and save the internal states of all used variables. When proceeding you need to restore these variables and run the next update calls. I viewed some (Java) imps, and it does not look very difficult for me.
I need to have a limit in the output length of my cipher. I know it depends on which algorithm you use, but I didnt find how do do this.
const crypto = require('crypto');
const options = {secret : 'SECRET', algorithm : 'CAST-cbc'};
exports.encrypt = (url) => {
const encrypt = crypto.createCipher(options.algorithm, options.secret);
let cipher = encrypt.update(url, 'utf8', 'hex');
cipher += encrypt.final('hex');
return cipher;
};
This is how im generating the cipher. Thanks.
limit in the output length of my cipher
...
'm trying to build a url shortener without having to have a database behind it
Encryption will be always longer than the original plaintext. You will need to have some IV (initialization vector), encrypted information, optionally an authentication code and then all encoded into an url safe format.
In theory you may sacrifice some level of security and have some sort of format-preserving encryption, but it will be at least as long as the original source.
final hash example: 1d6ca5252ff0e6fe1fa8477f6e8359b4
You won't be able to reconstruct original value from a hash. That's why for serious url shortening service you will need a database with key-value pair, where the key can be id or hash
You may still properly encrypt and encode the original data, just the output won't be shorter.
The output depends on input, always. What you could do is pad the input or output to the desired length to generate a consistent final hash. This requires there be a maximum length to the input, however.
Beyond that, there is no way to force a length and still have something decryptable. If you simply need a hash for validation, that's a bit different. If you clarify the purpose we can help more.
Given the updates that you need a hash, not encryption, then here is how this works.
You hash the original URL and save a file to disk (or wherever, S3). That file has the name that corresponds to the hash. So if the URL is 'http://google.com' the hash may be 'C7B920F57E553DF2BB68272F61570210' (MD5 hash), so you would save 'C7B920F57E553DF2BB68272F61570210'.txt or something on the server with 'http://google.com' as the file contents.
Now when someone goes to http://yourURLShortnener.com/C7B920F57E553DF2BB68272F61570210 you just look on disk for that file and load the contents, then issue a redirect for that URL.
Obviously you'd prefer a shorter hash, but you can just take a 10 digit substring from the original hash. It increases chances of hash collisions, but that's just the risk you take.
server code:
cipher = createCipher('aes128', 'password');
str = cipher.update('message', 'utf8', 'base64');
str += cipher.final('base64')
I want the client code (the browser) have the same algorithm as above, given the same message and password, produce the same output as the server's.
I tried CryptoJS, SJCL and some other libraries, but they use iv and salt which makes the result entirely different. In my situation, that security isn't necessary.
(I don't know exactly what iv and salt is, I just hope the code can function without them.)
UPDATE: I find that without proper knowledge of the encryption itself, using the function in this way is a huge mistake.
Per doc:
password is used to derive key and IV, which must be a 'binary'
encoded string or a buffer.
I'm going to learn some basics first.