Google Cloud KMS: The checksum in field ciphertext_crc32c did not match the data in field ciphertext - node.js

I am having issues setting up a system to encrypt and decrypt data in my Node.js backend. I am following this guide in the process.
I wrote a helper class KMSEncryption to abstract the logic from the example. Here's the code where I call it:
const kms = new KMSEncryption();
const textToEncrypt = 'hello world!';
const base64string = await kms.encrypt(textToEncrypt);
const decrypted = await kms.decrypt(base64string);
The issue I am having is that the decryption fails with the following error:
UnhandledPromiseRejectionWarning: Error: 3 INVALID_ARGUMENT: The checksum in field ciphertext_crc32c did not match the data in field ciphertext.
I tried to compare side by side with guide from Google docs but I cannot see where I went wrong.
Some of the things I have tried include:
Converting the base64string into a Buffer
Trying to calculate checksum on a Buffer of base64string and not the string itself
Any help is appreciated. Thank you

I believe you are base64 encoding the ciphertext when you do:
if (typeof ciphertext !== 'string') {
return this.toBase64(ciphertext);
}
but you are not reversing the encoding before calculating the crc32c.
I pulled this example together from sample code, it works correctly for me from Cloud Shell. (Sorry it's messy):
// On Cloud Shell, install ts first with:
// npm install -g typescript
// npm i #types/node
// npm i #google-cloud/kms
// npm i fast-crc32c
// Then to compile and run:
// tsc testcrc.ts && node testcrc.js
// Code adapted from https://cloud.google.com/kms/docs/encrypt-decrypt#kms-decrypt-symmetric-nodejs
const projectId = 'kms-test-1367';
const locationId = 'global';
const keyRingId = 'so-67778448';
const keyId = 'example';
const plaintextBuffer = Buffer.from('squeamish ossifrage');
// Imports the Cloud KMS library
const {KeyManagementServiceClient} = require('#google-cloud/kms');
const crc32c = require('fast-crc32c');
// Instantiates a client
const client = new KeyManagementServiceClient();
// Build the key name
const keyName = client.cryptoKeyPath(projectId, locationId, keyRingId, keyId);
// Optional, but recommended: compute plaintext's CRC32C.
async function encryptSymmetric() {
const plaintextCrc32c = crc32c.calculate(plaintextBuffer);
console.log(`Plaintext crc32c: ${plaintextCrc32c}`);
const [encryptResponse] = await client.encrypt({
name: keyName,
plaintext: plaintextBuffer,
plaintextCrc32c: {
value: plaintextCrc32c,
},
});
const ciphertext = encryptResponse.ciphertext;
// Optional, but recommended: perform integrity verification on encryptResponse.
// For more details on ensuring E2E in-transit integrity to and from Cloud KMS visit:
// https://cloud.google.com/kms/docs/data-integrity-guidelines
if (!encryptResponse.verifiedPlaintextCrc32c) {
throw new Error('Encrypt: request corrupted in-transit');
}
if (
crc32c.calculate(ciphertext) !==
Number(encryptResponse.ciphertextCrc32c.value)
) {
throw new Error('Encrypt: response corrupted in-transit');
}
console.log(`Ciphertext: ${ciphertext.toString('base64')}`);
console.log(`Ciphertext crc32c: ${encryptResponse.ciphertextCrc32c.value}`)
return ciphertext;
}
async function decryptSymmetric(ciphertext) {
const cipherTextBuf = Buffer.from(await ciphertext);
const ciphertextCrc32c = crc32c.calculate(cipherTextBuf);
console.log(`Ciphertext crc32c: ${ciphertextCrc32c}`);
const [decryptResponse] = await client.decrypt({
name: keyName,
ciphertext: cipherTextBuf,
ciphertextCrc32c: {
value: ciphertextCrc32c,
},
});
// Optional, but recommended: perform integrity verification on decryptResponse.
// For more details on ensuring E2E in-transit integrity to and from Cloud KMS visit:
// https://cloud.google.com/kms/docs/data-integrity-guidelines
if (
crc32c.calculate(decryptResponse.plaintext) !==
Number(decryptResponse.plaintextCrc32c.value)
) {
throw new Error('Decrypt: response corrupted in-transit');
}
const plaintext = decryptResponse.plaintext.toString();
console.log(`Plaintext: ${plaintext}`);
console.log(`Plaintext crc32c: ${decryptResponse.plaintextCrc32c.value}`)
return plaintext;
}
decryptSymmetric(encryptSymmetric());
You can see that it logs the crc32c several times. The correct crc32c for the example string, "squeamish ossifrage", is 870328919. The crc32c for the ciphertext will vary on every run.
To run this code yourself, you just need to point it at your project, region, key ring, and key (which should be a symmetric encryption key); hopefully comparing this code with your code's results will help you find the issue.
Thanks for using Google Cloud and Cloud KMS!

Related

Implementing JWE encryption for a JWS signed token in Node.JS with Jose 4.11

I have difficulty manipulating the Jose Node.JS documentation to chain the creation of a JWS and JWE. I cannot find the proper constructor for encryption. It looks like I can only encrypt a basic payload not a signed JWS.
Here is the code sample I try to fix to get something that would look like
const jws = await createJWS("myUserId");
const jwe = await encryptAsJWE(jws);
with the following methods
export const createJWS = async (userId) => {
const payload = {
}
payload['urn:userId'] = userId
// importing key from base64 encrypted secret key for signing...
const secretPkcs8Base64 = process.env.SMART_PRIVATE_KEY
const key = new NodeRSA()
key.importKey(Buffer.from(secretPkcs8Base64, 'base64'), 'pkcs8-private-der')
const privateKey = key.exportKey('pkcs8')
const ecPrivateKey = await jose.importPKCS8(privateKey, 'ES256')
const assertion = await new jose.SignJWT(payload)
.setProtectedHeader({ alg: 'RS256' })
.setIssuer('demolive')
.setExpirationTime('5m')
.sign(ecPrivateKey)
return assertion
}
export const encryptAsJWE = async (jws) => {
// importing key similar to createJWS key import
const idzPublicKey = process.env.IDZ_PUBLIC_KEY //my public key for encryption
...
const pkcs8PublicKey = await jose.importSPKI(..., 'ES256')
// how to pass a signed JWS as parameter?
const jwe = await new jose.CompactEncrypt(jws)
.encrypt(pkcs8PublicKey)
return jwe
}
The input to the CompactEncrypt constructor needs to be a Uint8Array, so just wrapping the jws like so (new TextEncoder().encode(jws)) will allow you to move forward.
Moving forward then:
You are also missing the JWE protected header, given you likely use an EC key (based on the rest of your code) you should a) choose an appropriate EC-based JWE Key Management Algorithm (e.g. ECDH-ES) and put that as the public key import algorithm, then proceed to call .setProtectedHeader({ alg: 'ECDH-ES', enc: 'A128CBC-HS256' }) on the constructed object before calling encrypt.
Here's a full working example https://github.com/panva/jose/issues/112#issue-746919790 using a different combination of algorithms but it out to help you get the gist of it.

Unable to Generate correct TOTP code from Twilio Authy App Node JS

Here is the scenrio, Id like to utilize https://npm.io/package/otplib to generate a TOTP code and verify it with the user input. The issue is that I am unable to generate a code using multiple authy apps that matches the one the totp.generate() generates. I think the issue might be either due to me passing an invalid secretKey format/type into totp.generate(). Or the issue might me due to the configuration of the totp component(maybe using the wrong encryption type(i.e sha2)) when compared to the authy app.
Here is my code sample following the guide from: https://npm.io/package/otplib
const generateSecretKey = (size=16) => {
const val = crypto.randomBytes(size).toString('hex').slice(0, size).toUpperCase()
return val;
}
const generateTotp = (secret) => {
const token = totp.generate(secret)
return token;
}
const authChallenge = (token, secret) =>{
const isValid = totp.check(token, secret);
return isValid
}
let secret = generateSecretKey()
console.log("secret => " + secret)
let token = generateTotp(secret)
console.log(`generateTotp => token ${token}`)
let authChallengeResponse = authChallenge(token, secret)
The returned value is
It seems the package is able to generate the code, the issue is it is not the same code as the ones in the authy app. Could this be due to me providing an invalid key type?

Why does decrypting modified AES-CBC ciphertext fail decryption?

I am trying to get familiar with encryption/decryption. I am using deno as it supports the web crypto API.
I can encrypt and decrypt to get back the original plaintext using AES-CBC.
What I am now doing now is to encrypt, then manually modify the ciphertext and then decrypt. My expectation is that this would still work since I understand that AES-CBC does not provide integrity and authenticity check. (AES-GCM is the one that is AEAD)
But when I modify the cipher text and try to decrypt, it fails with the following error:
error: Uncaught (in promise) OperationError: Decryption failed
let deCryptedPlaintext = await window.crypto.subtle.decrypt(param, key, asByteArray);
^
at async SubtleCrypto.decrypt (deno:ext/crypto/00_crypto.js:598:29)
at async file:///Users/me/delete/run.js:33:26
Does AES-CBC also have integrity checks? Or why is the decryption failing?
In Deno I had a similar issue while encrypting a jwt around server and client and could not rely on the TextDecoder class for the same reason:
error: OperationError: Decryption failed
After some hours I played around and found a solution, a bit tricky, but is doing the job right:
(async ()=> {
const textEncoder = new TextEncoder();
const textDecoder = new TextDecoder();
const rawKey = crypto.getRandomValues(new Uint8Array(16));
// we import the key that we have previously generated
const cryptoKey = await crypto.subtle.importKey(
"raw",
rawKey,
"AES-CBC",
true,
["encrypt", "decrypt"],
);
// we generate the IV
const iv = crypto.getRandomValues(new Uint8Array(16));
// here is the string we want to encrypt
const stringToEncrypt = "foobar"
// we encrypt
const encryptedString = await crypto.subtle.encrypt(
{ name: "AES-CBC", iv: iv },
cryptoKey,
textEncoder.encode(stringToEncrypt),
);
// we transform the encrypted string to an UInt8Array
const uint8ArrayEncryptedString = new Uint8Array(encryptedString);
// we transform the Array to a String so we have a representation we can carry around
const stringifiedEncryption =
String.fromCharCode(...uint8ArrayEncryptedString);
/* now is time to decrypt again the message, so we transform the string into
a char array and for every iteration we transform
the char into a byte, so in the end we have a byte array
*/
const stringByteArray =
[...stringifiedEncryption].map((v) => v.charCodeAt(0))
// we transform the byte array into a Uint8Array buffer
const stringBuffer = new Uint8Array(stringByteArray.length);
// we load the buffer
stringByteArray.forEach((v, i) => stringBuffer[i] = v)
// we decrypt again
const againDecrString = await crypto.subtle.decrypt(
{ name: "AES-CBC", iv: iv },
cryptoKey,
stringBuffer,
);
console.log(textDecoder.decode(againDecrString))
})()
For some of you relying on the class for the same purpose, I suggest you to use this solution. The underlying implementation pheraps loses some information while converting back and forth the string (I needed it as string after the encryption) and so the decryption fails.

Can't verify webhook Node.js

I'm attempting to verify a webhook signature from Patreon using Node.js. Here is my code:
const crypto = require("crypto");
...
function validateJsonWebhook(request) {
const secret = SECRET_KEY_GIVEN_BY_PATREON;
const hash = crypto.createHmac("md5", secret)
.update(JSON.stringify(request.body))
.digest("hex");
if (request.header("x-patreon-signature") === hash) {
return true;
} else {
return false;
}
}
Patreon webhooks use MD5 - see https://docs.patreon.com/#webhooks.
I've verified the secret key multiple times so I know that's not the issue.
Both "request.header("x-patreon-signature")" and "hash" are returning the correct format (i.e. they're both a 32 digit letter-number combination) but they're just not matching.
Any idea about what's going on?
So #gaiazov's comment led me to do some Googling which led me to the first two comments on Stripe verify web-hook signature HMAC sha254 HAPI.js by Karl Reid which led me to https://github.com/stripe/stripe-node/issues/331#issuecomment-314917167.
For anyone who finds this in the future: DON'T use JSON.stringify(request.body) - use request.rawBody instead, as the signature is calculated based on the raw JSON. I feel like this should be emphasized in Patreon's documentation, as all the examples I found used the code I originally posted. My new, working code is as follows (I cleaned up the "if (request.header("x-patreon-signature") === hash)" part at the end):
const crypto = require("crypto");
...
function validateJsonWebhook(request) {
// Secret key given by Patreon.
const secret = patreonSecret;
const hash = crypto.createHmac("md5", secret)
.update(request.rawBody)
.digest("hex");
return (request.header("x-patreon-signature") === hash);
}

Can't go through "Requested entity was not found" error with Speech-to-Text API

I call Cloud Speech-to-Text API in a Cloud Function triggered by a GCS event.
Doing it outside of a Cloud Function (running node index.js) is perfectly fine, but my error comes afterwards.
Using this doc, I believed that the error was due to an authentification issue, but I tried several things and I am not so sure now.
My code is :
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const nl = require('#google-cloud/language');
const client_nl = new nl.LanguageServiceClient();
const speech = require('#google-cloud/speech');
const client_speech = new speech.SpeechClient();
exports.getRecording = (data,context) => {
const file = data;
if (file.resourceState === 'not_exists') {
// Ignore file deletions
return true;
} else if (!new RegExp(/\.(wav|mp3)/g).test(file.name)) {
// Ignore changes to non-audio files
return true;
}
console.log(`Analyzing gs://${file.bucket}/${file.name}`);
const bucket = storage.bucket(file.bucket);
const audio = {
uri: 'gs://${file.bucket}/${file.name}'
};
// Configure audio settings for BoF recordings
const audioConfig = {
encoding: 'LINEAR16',
sampleRateHertz: 44100,
languageCode: 'fr-FR'
};
const request = {
audio: audio,
config: audioConfig,
};
return client_speech.recognize(request)
.then(([transcription]) => {
const filename = `analysis.json`;
console.log(`Saving gs://${file.bucket}/${filename}`);
return bucket
.file(filename)
.save(JSON.stringify(transcription, null, 2));
});
I then deploy with :
gcloud functions deploy getRecording --runtime nodejs10 --trigger-resource trigger-bucket-id --trigger-event google.storage.object.finalize --service-account my-service-account
What I tried :
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/file/keyfile.json
added a config.json file with "GOOGLE_APPLICATION_CREDENTIALS":"./keyfile.json" inside, and the keyfile in the root project and a require('./config.json') in the index.js
added an options json
const options = {
projectId: 'my-project-id',
keyFilename: './key-file.json'
};
const client_speech = new speech.SpeechClient(options);
I keep getting this error and would need some help
D getRecording 573287126069013 2019-06-07 15:03:09.609 Function execution started
getRecording 573287126069013 2019-06-07 15:03:09.789 Analyzing gs://my-bucket/audio_trimed.wav
D getRecording 573287126069013 2019-06-07 15:03:10.979 Function execution took 1372 ms, finished with status: 'error'
E getRecording 573291408785013 2019-06-07 15:03:11.990 Error: Requested entity was not found.
at Http2CallStream.call.on (/srv/functions/node_modules/#grpc/grpc-js/build/src/client.js:101:45)
at Http2CallStream.emit (events.js:194:15)
at Http2CallStream.EventEmitter.emit (domain.js:459:23)
at Http2CallStream.endCall (/srv/functions/node_modules/#grpc/grpc-js/build/src/call-stream.js:63:18)
at handlingTrailers (/srv/functions/node_modules/#grpc/grpc-js/build/src/call-stream.js:152:18)
at process._tickCallback (internal/process/next_tick.js:68:7)
Does any of this code help?
There are a few StackOverflow posts that might help here, and here
I suggest more verbose logging / debugging.
For example, can you prove out that these two lines do what you expect? Does the string produce a well formed uri?
const bucket = storage.bucket(file.bucket);
const audio = {
uri: 'gs://${file.bucket}/${file.name}'
};
Let us know if you find a solution
#Bruce was right, I finally spotted the mistake.
I just couldn't see that it was written
uri:'gs://${file.bucket}/${file.name}'
and not
uri: `gs://${file.bucket}/${file.name}`
(yes the mistake was tiny but could have been avoided with more testing...

Resources