I'm encrypting a a text file locally using the crypto module from node.js and upload the file to IPFS. I'm trying to then download the file and decrypt it. But once I upload the encrypted file to IPFS, it seems to change to a buffer.
It looks somewhat like this before upload (I only copied the first line):
N�&��6-d�9L% 9���E��k�ir�C��ڤ|%B5-(���i�
...
...
And it looks like this after download:
{"type":"Buffer","data":[1,78,211,38,190,164,54,25,...])
I tried to convert it to multiple encodings, but that doesn't seem to solve anything. Is there a way to upload it in a different format (which stays the same on IPFS) or is there a way to convert the buffer back and decrypt it?
This is the code I use for encryption:
export const encrypt = (buffer, key) => {
const algorithm = 'aes-256-ctr';
const iv = crypto.randomBytes(16);
const cipher = crypto.createCipheriv(algorithm, key, iv);
const result = Buffer.concat([iv, cipher.update(buffer), cipher.final()]);
return result;
};
I need to write the reverse (encryption) of the following decryption function:
const crypto = require('crypto');
let AESDecrypt = (data, key) => {
const decoded = Buffer.from(data, 'binary');
const nonce = decoded.slice(0, 16);
const ciphertext = decoded.slice(16, decoded.length - 16);
const tag = decoded.slice(decoded.length - 16);
let decipher = crypto.createDecipheriv('aes-256-gcm', key, nonce);
decipher.setAuthTag(tag)
decipher.setAutoPadding(false);
try {
let plaintext = decipher.update(ciphertext, 'binary', 'binary');
plaintext += decipher.final('binary');
return Buffer.from(plaintext, 'binary');
} catch (ex) {
console.log('AES Decrypt Failed. Exception: ', ex);
throw ex;
}
}
This above function allows me to properly decrypt encrypted buffers following the spec:
| Nonce/IV (First 16 bytes) | Ciphertext | Authentication Tag (Last 16 bytes) |
The reason why AESDecrypt is written the way it (auth tag as the last 16 bytes) is because that is how the default standard library implementations of AES encrypts data in both Java and Go. I need to be able to bidirectionally decrypt/encrypt between Go, Java, and Node.js. The crypto library based encryption in Node.js does not put the auth tag anywhere, and it is left to the developer how they want to store it to pass to setAuthTag() during decryption. In the above code, I am baking the tag directly into the final encrypted buffer.
So the AES Encryption function I wrote needed to meet the above circumstances (without having to modify AESDecrypt since it is working properly) and I have the following code which is not working for me:
let AESEncrypt = (data, key) => {
const nonce = 'BfVsfgErXsbfiA00'; // Do not copy paste this line in production code (https://crypto.stackexchange.com/questions/26790/how-bad-it-is-using-the-same-iv-twice-with-aes-gcm)
const encoded = Buffer.from(data, 'binary');
const cipher = crypto.createCipheriv('aes-256-gcm', key, nonce);
try {
let encrypted = nonce;
encrypted += cipher.update(encoded, 'binary', 'binary')
encrypted += cipher.final('binary');
const tag = cipher.getAuthTag();
encrypted += tag;
return Buffer.from(encrypted, 'binary');
} catch (ex) {
console.log('AES Encrypt Failed. Exception: ', ex);
throw ex;
}
}
I am aware hardcoding the nonce is insecure. I have it this way to make it easier to compare properly encrypted files with my broken implementation using a binary file diff program like vbindiff
The more I looked at this in different ways, the more confounding this problem has become for me.
I am actually quite used to implementing 256-bit AES GCM based encryption/decryption, and have properly working implementations in Go and Java. Furthermore, because of certain circumstances, I had a working implementation of AES decryption in Node.js months ago.
I know this to be true because I can decrypt in Node.js, files that I encrypted in Java and Go. I put up a quick repository that contains the source code implementations of a Go server written just for this purpose and the broken Node.js code.
For easy access for people that understand Node.js, but not Go, I put up the following Go server web interface for encrypting and decrypting using the above algorithm hosted at https://go-aes.voiceit.io/. You can confirm my Node.js decrypt function works just fine by encrypting a file of your choice at https://go-aes.voiceit.io/, and decrypting the file using decrypt.js (Please look at the README for more information on how to run this if you need to confirm this works properly.)
Furthermore, I know this issue is specifically with the following lines of AESEncrypt:
const tag = cipher.getAuthTag();
encrypted += tag;
Running vbindiff against the same file encrypted in Go and Node.js, The files started showing differences only in the last 16 bytes (where the auth tag get's written). In other words, the nonce and the encrypted payload is identical in Go and Node.js.
Since the getAuthTag() is so simple, and I believe I am using it correctly, I have no idea what I could even change at this point. Hence, I have also considered the remote possibility that this is a bug in the standard library. However, I figured I'd try Stackoverflow first before posting an Github Issue as its most likely something I'm doing wrong.
I have a slightly more expanded description of the code, and proof of how I know what is working works, in the repo I set up to try to get help o solve this problem.
Thank you in advance.
Further info: Node: v14.15.4 Go: go version go1.15.6 darwin/amd64
In the NodeJS code, the ciphertext is generated as a binary string, i.e. using the binary/latin1 or ISO-8859-1 encoding. ISO-8859-1 is a single byte charset which uniquely assigns each value between 0x00 and 0xFF to a specific character, and therefore allows the conversion of arbitrary binary data into a string without corruption, see also here.
In contrast, the authentication tag is not returned as a binary string by cipher.getAuthTag(), but as a buffer.
When concatenating both parts with:
encrypted += tag;
the buffer is converted into a string implicitly using buf.toString(), which applies UTF-8 encoding by default.
Unlike ISO-8859-1, UTF-8 is a multi byte charset that defines specific byte sequences between 1 and 4 bytes in length that are assigned to characters, s. UTF-8 table. In arbitrary binary data (such as the authentication tag) there are generally byte sequences that are not defined for UTF-8 and therefore invalid. Invalid bytes are represented by the Unicode replacement character with the code point U+FFFD during conversion (see also the comment by #dave_thompson_085). This corrupts the data because the original values are lost. Thus UTF-8 encoding is not suitable for converting arbitrary binary data into a string.
During the subsequent conversion into a buffer with the single byte charset binary/latin1 with:
return Buffer.from(encrypted, 'binary');
only the last byte (0xFD) of the replacement character is taken into account.
The bytes marked in the screenshot (0xBB, 0xA7, 0xEA etc.) are all invalid UTF-8 byte sequences, s. UTF-8 table, and are therefore replaced by the NodeJS code with 0xFD, resulting in a corrupted tag.
To fix the bug, the tag must be converted with binary/latin1, i.e. consistent with the encoding of the ciphertext:
let AESEncrypt = (data, key) => {
const nonce = 'BfVsfgErXsbfiA00'; // Static IV for test purposes only
const encoded = Buffer.from(data, 'binary');
const cipher = crypto.createCipheriv('aes-256-gcm', key, nonce);
let encrypted = nonce;
encrypted += cipher.update(encoded, 'binary', 'binary');
encrypted += cipher.final('binary');
const tag = cipher.getAuthTag().toString('binary'); // Fix: Decode with binary/latin1!
encrypted += tag;
return Buffer.from(encrypted, 'binary');
}
Please note that in the update() call the input encoding (the 2nd 'binary' parameter) is ignored, since encoded is a buffer.
Alternatively, the buffers can be concatenated instead of the binary/latin1 converted strings:
let AESEncrypt_withBuffer = (data, key) => {
const nonce = 'BfVsfgErXsbfiA00'; // Static IV for test purposes only
const encoded = Buffer.from(data, 'binary');
const cipher = crypto.createCipheriv('aes-256-gcm', key, nonce);
return Buffer.concat([ // Fix: Concatenate buffers!
Buffer.from(nonce, 'binary'),
cipher.update(encoded),
cipher.final(),
cipher.getAuthTag()
]);
}
For the GCM mode, a nonce length of 12 bytes is recommended by NIST for performance and compatibility reasons, see here, chapter 5.2.1.1 and here. The Go code (via NewGCMWithNonceSize()) and the NodeJS code apply a nonce length of 16 bytes different from this.
I have a NodeJS script, that reads PDFs, I was wondering how one can go about decrypting a PDF with a supplied password with NodeJS specific.
You can use node-qpdf npm module(https://www.npmjs.com/package/node-qpdf)
Simple example code found in the documentation of node-qpdf module is:
To encrypt a pdf
var qpdf = require('node-qpdf');
var options = {
keyLength: 128,
password: 'YOUR_PASSWORD_TO_ENCRYPT',
restrictions: {
print: 'low',
useAes: 'y'
}
}
qpdf.encrypt(localFilePath, options, outputFilePath);
To decrypt the pdf
var qpdf = require('node-qpdf');
qpdf.decrypt(localFilePath, 'YOUR_PASSWORD_TO_DECRYPT_PDF', outputFilePath);
I also try to decrpt password protected pdf using below function but don't know how to get base64 data or file
const qpdf = require('node-qpdf');
await qpdf.decrypt('lic_demo.pdf', 'awxpt1377q12081992', 'demo.pdf');
I want to encrypt the audio file by android and decrypt it by backend sails js. I developed the program for that but I got error in sails js like
error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt
This is my source code for encrypt the audio file in android
final String ALGORITHM = "blowfish";
String keyString = "DesireSecretKey";
private void encrypt(String file) throws Exception {
File extStore = Environment.getExternalStorageDirectory();
File inputFile = new File(file);
File encryptedFile = new
File(extStore+"/Movies/encryptAudio.amr");
doCrypto(Cipher.ENCRYPT_MODE, inputFile, encryptedFile);
}
private void doCrypto(int cipherMode, File inputFile,
File outputFile) throws Exception {
Key secretKey = new
SecretKeySpec(keyString.getBytes(),ALGORITHM);
Cipher cipher = Cipher.getInstance(ALGORITHM);
cipher.init(cipherMode, secretKey);
FileInputStream inputStream = new FileInputStream(inputFile);
byte[] inputBytes = new byte[(int) inputFile.length()];
inputStream.read(inputBytes);
byte[] outputBytes = cipher.doFinal(inputBytes);
FileOutputStream outputStream = new
FileOutputStream(outputFile);
outputStream.write(outputBytes);
inputStream.close();
outputStream.close();
}
I installed the crypto , fs libraries in sails js backend by following command
npm install crypto
npm install fs
This is my source code for decrypt the audio file in sails js
function decrypt() {
var crypto = require('crypto'),
algorithm = 'blowfish',
password = 'DesireSecretKey';
var fs = require('fs');
// input file
var r = fs.createReadStream(config.UPLOAD_FILES_PATH
+'/encryptAudio.amr');
var decrypt = crypto.createDecipher(algorithm, password,"");
// write file
var w = fs.createWriteStream(config.AUDIO_PATH+'decryptAudio.amr');
// start pipe
r.pipe(decrypt).pipe(w);
}
Encryption is working properly & i can get the encrypted audio file.But the issue is i couldn't get the decrypted audio file by sails js. Can you identify the issue?
maybe the answer to this question might help as you are using Java's library for cryptography this should pretty much guide you in right direction.
Encrypt with Node.js Crypto module and decrypt with Java (in Android app)
I would have mentioned this in comments section but I do not have enough reputation points to comment.
This is opposite of what you are trying to achieve but still it could help you to analyze your issue.
I'm following the encryption example on this URL (code sample below) (http://lollyrock.com/articles/nodejs-encryption/). The problem is that I'm encrypting a .zip file, which seems to work just fine. The decryption is the problem. If I perform the code example below on something like a jpg, the picture comes out just fine. But if I run a zip file through it and I try to unzip the result, I get the following error:
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
Code:
// Nodejs encryption of buffers
var crypto = require('crypto'),
algorithm = 'aes-256-ctr',
password = 'd6F3Efeq';
var fs = require('fs');
var zlib = require('zlib');
// input file
var r = fs.createReadStream('file.txt');
// zip content
var zip = zlib.createGzip();
// encrypt content
var encrypt = crypto.createCipher(algorithm, password);
// decrypt content
var decrypt = crypto.createDecipher(algorithm, password)
// unzip content
var unzip = zlib.createGunzip();
// write file
var w = fs.createWriteStream('file.out.txt');
// start pipe
r.pipe(zip).pipe(encrypt).pipe(decrypt).pipe(unzip).pipe(w);
So it turns out the difference in my code was it was reading from a request stream. Apparently you can't just pipe a request stream through gunzip through decryption? I'm not sure why.
But if I same the stream to a file and THEN run it through gunzip and decryption it works.
If anyone has any input as to why, I'd at least like to understand!