I'm trying to maintain FastSpring e-commerce platform Secure Payload api implementation in Python.
Their documentation has examples for encrypting (or technically signing?) the payload with private key in Java and PHP: https://developer.fastspring.com/docs/pass-a-secure-request#locally-encrypt-the-session-object
And I have been previously using a Python "cryptography" library based implementation based on this repository:
https://github.com/klokantech/flask-fastspring/blob/0462833f67727cba9755a26c88941f11f0159a48/flask_fastspring.py#L247
However, that relies on undocumented openssl "_lib.RSA_private_encrypt()" function that is no longer exposed in cryptography versions higher than 2.9.2, which is already several years old. And with latest python versions it no longer includes binary packages and PIP must compile it from source.
PyCryptodome seems to include similar RSA private key signing with PKCS #1 v1.5 padding, but it requires payload to be a Hash object, so naturally the produced output doesn't match what FastSpring expects regardless of what Hash function I use: https://pycryptodome.readthedocs.io/en/latest/src/signature/pkcs1_v1_5.html?highlight=pkcs1_15#pkcs-1-v1-5-rsa
I have been trying to firure out any alternative ways to implement this kind of "private key encryption" without success. So my question is: Is there ANY way to do this with up-to-date python libararies or am I stuck to use an outdated cryptography library until it no longer is supported at all?
The two linked codes implement low level signing using RSASSA-PKCS1-v1_5, but a modified encoding is used for the message rather than EMSA-PKCS1-v1_5, and therefore the processing differs from the standard.
The two major Python crypto libraries PyCryptodome and Cryptography only support high level signing, which encapsulates the entire process, follows the standard and thus does not allow any modification of the encoding of the message.
The most efficient way to solve the problem would be to use a Python library that also supports a low level signing, so that the encoding of the message from the linked Java or Python code can be used. However, I am not aware of such a library.
If you don't know such a library either, there is the following alternative: Since RSASSA-PKCS1-v1_5 is pretty simple and Python supports large integers and their operations natively, a custom implementation in combination with the helper functions of e.g. PyCryptodome is easily possible. At least you wouldn't have to rely on the legacy library anymore:
from Crypto.Util import number
def customizedSign(key, msg):
modBits = number.size(key.n)
k = number.ceil_div(modBits, 8)
ps = b'\xFF' * (k - len(msg) - 3)
em = b'\x00\x01' + ps + b'\x00' + msg
em_int = number.bytes_to_long(em)
m_int = key._decrypt(em_int)
signature = number.long_to_bytes(m_int, k)
return signature
Explanation:
The implementation follows the PyCryptodome implementation of the sign() method. The only functional difference is that instead of EMSA-PKCS1-v1_5 the encoding of the linked codes is used.
EMSA-PKCS1-v1_5 is defined as:
EM = 0x00 || 0x01 || PS || 0x00 || T
where T is the concatenation of the DER encoded DigestInfo value and the hashed message, see here.
The encoding of the linked codes simply uses the message MSG instead of T:
EM = 0x00 || 0x01 || PS || 0x00 || MSG
In both cases, PS is a padding with 0xFF values up to the key size (i.e. size of the modulus).
Usage and test:
Since the signature is deterministic, the same key and the same message always provide the same signature. This way it is easy to show that the above function is equivalent to the linked Java or Python code:
from Crypto.PublicKey import RSA
import base64
# For simplicity, a 512 bits key is used. Note that a 512 bits key may only be used for testing, in practice the key size has to be >= 2048 bits for security reasons.
pkcs8 = """-----BEGIN PRIVATE KEY-----
MIIBVQIBADANBgkqhkiG9w0BAQEFAASCAT8wggE7AgEAAkEA2gdsVIRmg5IH0rG3
u3w+gHCZq5o4OMQIeomC1NTeHgxbkrfznv7TgWVzrHpr3HHK8IpLlG04/aBo6U5W
2umHQQIDAQABAkEAu7wulGvZFat1Xv+19BMcgl3yhCdsB70Mi+7CH98XTwjACk4T
+IYv4N53j16gce7U5fJxmGkdq83+xAyeyw8U0QIhAPIMhbtXlRS7XpkB66l5DvN1
XrKRWeB3RtvcUSf30RyFAiEA5ph7eWXbXWpIhdWMoe50yffF7pW+C5z07tzAIH6D
Ko0CIQCyveSTr917bdIxk2V/xNHxnx7LJuMEC5DcExorNanKMQIgUxHRQU1hNgjI
sXXZoKgfaHaa1jUZbmOPlNDvYYVRyS0CIB9ZZee2zubyRla4qN8PQxCJb7DiICmH
7nWP7CIvcQwB
-----END PRIVATE KEY-----"""
key = RSA.import_key(pkcs8)
msg = 'The quick brown fox jumps over the lazy dog'.encode('utf8')
signature = customizedSign(key, msg)
print(base64.b64encode(signature).decode('utf8')) # OwpVG/nPmkIbVxONRwXHvOqLdYNnP67YtiWA+GcKBZ3rIzAJ+8izvmlqUQnzVp03Wrrzq2ogUmCMaLSPlInDNw==
The linked Java code provides the same signature for the same key and message.
Related
I have a file that has been encrypted using openssl using the following command:
openssl enc -in data -out encrypted -e -aes256 -k myverystrongpassword
Where data is the original file and encrypted is the encrypted file.
I tried various ways using crypto library but nothing seems to work. I understand that the password needs to be converted into a key so maybe I am doing something wrong there. Looked all over for a solution but nothing seems to work.
The posted OpenSSL statement uses a key derivation function EVP_BytesToKey() to derive a 32 bytes key and a 16 bytes IV from the password in combination with a random 8 bytes salt.
The ciphertext corresponds to the concatenation of the ASCII encoding of Salted__, followed by the salt and finally by the actual ciphertext.
As you already know according to your comment, EVP_BytesToKey() uses a digest for which OpenSSL applied MD5 by default in earlier versions and SHA-256 as of version v1.1.0 (the default value can be overridden in the OpenSSL statement with the -md option).
Decryption is possible e.g. with CryptoJS: Due to its OpenSSL compatibility (s. sec. Interoperability) CryptoJS has a built-in implementation of an accessible EVP_BytesToKey() function and additionally allows to explicitly set the digest in the internal EVP_BytesToKey() call during key derivation. This makes it possible to decrypt encryptions that used SHA-256 or MD5 in key derivation.
The following data is the Base64 encoding of a ciphertext generated with the posted OpenSSL statement. The plaintext used was The quick brown fox jumps over the lazy dog. The OpenSSL version applied is v1.1.1i (i.e. SHA-256 is implicitly used in the key derivation):
U2FsdGVkX19W4wmC9dD35X4J66cSvaRaIQpvjDKHrLF9+qYg5VTo5urvExHLXhwf/bE8FXJTQZmKN8ITMJVdqQ==
This ciphertext can be successfully decrypted using the following CryptoJS implementation:
const password = 'myverystrongpassword';
const saltCiphertextB64 = 'U2FsdGVkX19W4wmC9dD35X4J66cSvaRaIQpvjDKHrLF9+qYg5VTo5urvExHLXhwf/bE8FXJTQZmKN8ITMJVdqQ==';
CryptoJS.algo.EvpKDF.cfg.hasher = CryptoJS.algo.SHA256.create(); // default: MD5
const decryptedData = CryptoJS.AES.decrypt(saltCiphertextB64, password);
console.log(decryptedData.toString(CryptoJS.enc.Utf8)); // The quick brown fox jumps over the lazy dog
<script src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/4.1.1/crypto-js.min.js"></script>
Note that the digest in the code must be explicitly specified as SHA-256 since OpenSSL v1.1.1i was used for encryption.
If the encryption was done with an OpenSSL version that uses MD5, the digest in the code must be modified accordingly.
Edit: As noted in the comment, the crypto functions createCipher()/createDecipher() also use EVP_BytesToKey() as key derivation.
However, the following should be noted:
Unlike CryptoJS, it is not possible to specify the digest, i.e. MD5 is used unchangeably. Thus, encryptions that applied SHA-256 for key derivation cannot be decrypted (what applies to the encryptions here).
In contrast to CryptoJS, no salt is used by default. Therefore, salt creation and concatenation (Salted__|<salt>|<cipherext>) during encryption and separation during decryption would have to be implemented additionally. createCipher()/createDecipher() then has to be passed the concatenation of passphrase and salt.
Both functions are deprecated since version 10.0.0 and should actually not be used.
A more robust approach to decrypt encryptions (with arbitrary digests in key derivation) using the crypto module is to apply createCipheriv()/createDecipheriv() and a port of the required functionality of EVP_BytesToKey() to derive key and IV (various implementations can be found on the net).
Security: EVP_BytesToKey() is deemed to be a vulnerability these days. This is worsened by a low iteration count (like 1, which is used by OpenSSL), a broken digest (like MD5) or a missing salt (as is the default for crypto). Ultimately, this is why createCipher()/createDecipher() are deprecated. Instead of EVP_BytesToKey(), a more reliable key derivation function such as PBKDF2 or the more modern scrypt or Argon2 should be used.
I am using Node.JS and the crypto module. I have a SHA-256 hash in hex string and would like to create a crypto.Hash instance out of it. I only found ways to hash the input string itself, but not to update or create a new hash. Am I missing something from the documentation?
I am looking for something like (for UUID though):
crypto.Hash.from("sha256", "hex", "d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592")
Generally there are not many libraries that do what you ask of them. There are certainly libraries where the internal state can be retrieved and restored such as Bouncy Castle, but as yet I haven't seen it in any JavaScript library. It would be very easy to create though.
Indeed, the 256 bit (total) intermediate values after each block of 512 bits will be used as final output after the last block is hashed. So if you can "restore" those values (i.e. put them in the state) then you could continue hashing after that.
This might not be that useful though, as those values already contain the padding and message size encoded into a 64 bit representation at the end of the block. So if you continue hashing after that, that padding and length will likely be included again, but now with different values.
One trick sometimes used in smart cards is to upload the intermediate values (including number of bits hashed) before the last data to be hashed, and let the smart card perform the padding, the length encoding and the final hash block operation. This is usually performed during signature calculation over large amounts of data (because you really don't want to send a whole document to smart card).
Pretty dumb if you ask me, just directly signing using a pre-calculated hash value is the way forward. Or making sure that the large swath of data is pre-hashed, and the hash is then signed (including another pass of the hash) - that way the entire problem can be avoided without special tricks.
The following small example code hashes a string to a base64- and hex string encoding.
This is the output:
buf: The quick brown fox jumps over the lazy dog
sha256 (Base64): 16j7swfXgJRpypq8sAguT41WUeRtPNt2LQLQvzfJ5ZI=
sha256 (hex): d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592
code:
var crypto = require('crypto');
const plaintext = 'The quick brown fox jumps over the lazy dog';
const buf = Buffer.from(plaintext, 'utf8');
console.log('buf: ' + buf);
const sha256Base64 = crypto.createHash('sha256').update(buf).digest('base64');
console.log('sha256 (Base64): ' + sha256Base64);
const sha256Hex = crypto.createHash('sha256').update(buf).digest('hex');
console.log('sha256 (hex): ' + sha256Hex);
Edit: I misunderstood the question, the task is to run several SHA-256 update calls (e.g. for different strings) and receive a (intermediate) result. Later this result should be used as "input" and the hashing should be proceeded with more update calls, finished with by "final"/"digest" to get the final sha-256 value about all parts.
Unfortunately there seems to be no way as the intermediate value is a final value and there (again) seems to be no way back because the final/digest calls make so additional computations (has to do with the underlying Merkle-Damgård constructions). The only way would be use an own written SHA-256 function and save the state of all internal registers, reset the registers when continuing and get the final value in the end.
An advice could be to grab the source code of a sha-256 implementation and save the internal states of all used variables. When proceeding you need to restore these variables and run the next update calls. I viewed some (Java) imps, and it does not look very difficult for me.
I'd like to derive an elliptic curve private key from input keying material (a master key). Is this possible?
Attempts
Node's crypto function, crypto.generateKeyPair does not accept input keying material, and crypto.createPrivateKey only converts a .pem to Node's native KeyObject.
I also can't find a way to do this in OpenSSL using ecparam. The -rand flag seems promising but isn't widely available (it's not on my machine).
Why / Details
I need to create a number of secrets and want have all of them derived from a single master key. This is analogous to HKDF.
I'm using the keys for ECDSA with curve P-384 (secp384r1).
I'm surprised you think ecparam -rand file -genkey ... is rare; it's in every upstream version back at least to 0.9.8 in 2005, and it's not one of the things that can be omitted by a build (configure) option, so your machine must have one weird version. But it doesn't matter because -rand doesn't do what you want; it adds the file data to the RNG 'pool' but does not replace it, so it doesn't give you deterministic key generation.
As Woodstock commented, for all practical purposes a raw P-384 private key is just 384 bits from any good random generator, or deterministically from any uniform random function. Technically you should exclude zero and values greater than or equal to the (sub)group order n, but those exclusions are so small relative to 2^384 that there is essentially no chance a good random choice will hit them during the lifetime of the Earth, and perhaps of the universe. You might want to look at how Bitcoin 'hierarchical deterministic' key derivation aka BIP 32 works, although of course that does 256-bit keys for secp256k1.
That leaves you the problem of converting the raw key to a form usable by nodejs crypto (which is a fairly thin wrapping of openssl library) and/or openssl commandline. To do this follow the principles of How to convert an ECDSA key to PEM format which is in turn based on https://bitcoin.stackexchange.com/questions/66594/signing-transaction-with-ssl-private-key-to-pem except use the OID and size(s) for P-384 instead of secp256k1. Specifically, concatenate
the 7 bytes represented in hex by 303e0201010430
the 48 bytes (384 bits) of the raw private key
the 9 bytes represented in hex by a00706052b81040022 (for P-384 aka secp384r1)
Depending on your language(s) or tool(s) you might handle these values directly, or concatenate the hex representations and then convert to binary. The result is the 'DER' (binary) form of the algorithm-specific (SEC1) private key (only), which can be read by nodejs 11 or 12 crypto.createPrivateKey( {key:(data), format:'der', type:'sec1'} ) and also by commandline openssl ec -inform der.
If you prefer textlike things (e.g. for cut&paste), convert the DER above to base64, break into lines of 64 chars (other than the last), and add lines -----BEGIN EC PRIVATE KEY----- before and -----END EC PRIVATE KEY------ after. This is PEM format and can be read by createPrivateKey without any other options, and by openssl ec without any option.
I have signed a hash value in windows using BCryptSignHash with ECDSA algorithm. The output signature buffer is of length 64 bytes. I also generated the public and private key blobs using BCryptGenerateKeyPair function (BCRYPT_ECDSA_P256_ALGORITHM algorithm) with which i signed the hash.
I have to verify this signature with this key pair in linux. I am able to decipher the public-private key pair that got generated, using the link "http://msdn.microsoft.com/en-us/library/windows/desktop/aa375520%28v=vs.85%29.aspx" and able to use the same in linux.
The 64-byte signature generated should ideally be signature pair (r,s) (http://en.wikipedia.org/wiki/Elliptic_Curve_DSA).
Is there a way to understand the 64-bytes signature generated so that i can map the signature blob contents to (r,s) pair in linux and verify it?
Or is there a simpler way to verify the generated signature in linux?
Thanks,
F
Is there a way to understand the 64-bytes signature generated so that I can map the signature blob contents to (r,s) pair in linux and verify it?
The r and s are in P1363 format, which is simply a concatenation of r and s in a 2's compliment format. That is, the signature is simply r || s.
You need to know the hash to use this format. For example, SHA1 will create a r of 20 bytes and an s of 20 bytes. If r or s is "too short", then it is padded on the left with 0's.
Java and OpenPGP are different than P1363. Java and OpenPGP use an ASN.1 encoding:
SEQUENCE ::= {
r INTEGER,
s INTEGER
}
Depending what library you use on Linux, you may have to convert between the formats. Cryptographic Interoperability: Digital Signatures gives examples of signing and verifying using a few different libraries.
Or is there a simpler way to verify the generated signature in linux?
Try Crypto++. I believe Microsoft and Crypto++ uses the same signature format, so you won't need to convert. See Elliptic Curve Digital Signature Algorithm for details.
I am beginnging to wonder if the implementation of AES is different across libraries..
Currently i have a plaintext encrypted with PyCrypto.
Im trying to decrypt the ciphertext with Node.js's Crypto Library..
Basically with PyCrypto..
im using AES-128-CBC with a random generated IV. (which decrypts perfectly in PyCrypto)
However..
On Node.js im doing this
var buf = new Buffer(ciphertext)
var decipher = crypto.createDecipher('aes-128-cbc',aeskey)
buf = decipher.update(buf,'binary', 'binary')
buf += decipher.final('binary')
Which spits out a bunch of Garbage.... ( changing 'binary' to hex/utf8 doesnt help)
As i am using CBC (Cipher Block Chaining)...
i am prepending the IV to the beginning of the ciphertext (16 blocks)..
In PyCrypto this works perfectly, similarly to the specification of PGP, CFB usage..
Does anyone know for what reason this is not working???
Am i expecting too much of Node.js's standard libraries?
Documentation does not mention this, but aeskey you're passing to crypto.createDecipher is not the key, but a password, handled to OpenSSL's EVP_BytesToKey function.
To pass the actual raw key data one should use (presently undocumented) crypto.createDecipheriv(cipher, key, iv) function. This applies to ECB mode too, even though there's no IV in ECB.
If this fails, I think, the first step in debugging would be to try with AES KATs to see whenever the decryption code is correct.
I've tripped on a similar issue here: https://github.com/joyent/node/issues/1318
AES is a rijndael standard. It shouldn't be different. You should look into data types and default settings that are hidden. Something must be set different between the two. The key sizes might be different as 128 bit "hello" is padded with zeros I think and a smaller key would start with "hello" but have a smaller padding, therefore different.
The short answer to your question is: Yes, AES is the same in PyCrypto and Node.js' crypto module. Node's crypto is just a wrapper around openssl on your system, and PyCrypto is interoperable with OpenSSL (see http://lists.dlitz.net/pipermail/pycrypto/2010q4/000301.html).
Having said that, there are definitely bugs in the Node crypto module (though I've only experienced problems with base64 encoding, myself). So whether it's a bug or not, the problems you're experiencing are almost certainly happening in the data encoding/decoding stages.
What does your ciphertext look like? Is it a hexadecimal string? If so, then you need to do
buf = decipher.update(buf, 'hex', 'binary')
That's not how IV works in Node, you have to use crypto.createDecipheriv(cipher, key, iv) instead, otherwise you get a default baked-in one. Even in PyCrypto you should be using the third argument to AES.new as the IV, not stuffing it into the bytestream.
Make sure you use the same key and IV in both pycrypto and node.js!! Not only that, but make sure you have the same encoding in both ends:
cipher = AES.new(key.decode('hex'), AES.MODE_CBC, iv.decode('hex'))
text = json.dumps(payload)
pad = lambda s: s + (16 - len(s) % 16) * '\x07'
encryptedText = base64.b64encode(cipher.encrypt(pad(text)))
Then in node.js (sorry, no easy access to that code now), also make sure you decode your key and iv to hex