In Postgres, I am using:
encrypt('text', 'mykey', 'aes')
https://www.postgresql.org/docs/8.3/pgcrypto.html - F.20.4. Raw encryption functions
I assume this is done using the defaults of AES-128 algorithm, CBC mode.
It appears to be stored as BINARY (16 bytes) in the database and comes back as a base64? encoded string when I request the column on the server with my ORM.
In Node.js, assuming I have 'mykey', how do I convert this value back to its plaintext using crypto or crypto-js libraries?
Could you use PostgreSQL's Pgp_sym_encrypt() instead, and then use it with https://www.npmjs.com/package/openpgp to decode? PostgreSQL's raw encryption functions are not really suitable for compatible use with external systems (and really, not suitable for use at all)
Related
I need to implement an api endpoint that just takes a id in query param, then sends its encrypted value back. For that I was looking into the crypto module in node, and I found it a bit complex. One thing that I donot get is how am I suppose to use the iv? I plan to store the encryption key in the env such that every id can be decrypted using that same key. So, should I also store the iv in the env? Is that a good practice?
I have seen some apis actually randomly generates iv for each request, and return it alongside the encrypted text, such that the user can send them both later for decryption. But for my usecase, I cannot send two separate data back to the user. I can concat iv in the encrypted text, but for some values, the encrypted text in itself is too long for my use case. Any suggestion on what might be the best approach for my case?
Initialisation vectors are important to prevent attackers using brute force methods to decrypt data after a breach has occurred, i.e. in the event the DB has been copied/stolen.
In summary, if you encrypted the same password twice, with the same key, but without an IV, you will get the same encrypted string output. By adding an IV you will get a different output with the same password, but you have to store the IV along with the encrypted data, see Cipher Block Chaining. This makes it much harder to decrypt breached password databases as the attacker cannot use dictionaries of common passwords to test keys for a match within the data. In relational databases an IV is typically called a 'salt', in Postgres for example, you should generate a new salt when storing each password, like so:
UPDATE user SET password = crypt('new password', gen_salt('md5'));
For your use case I'm not certain if you need an IV, it depends on how the encrypted data is supposed to be used and/or stored. If you decide you don't need one, you can just omit it either of these ways:
1: Pass null instead of an IV:
const cipher = createCipheriv('aes-192-ccm', key, null);
2: (Deprecated since Node 10) Use the createCipher function:
const cipher = crypto.createCipher('aes-192-ccm', key);
I have a user table which stores the encrypted data( name, last name, DOB, email,...) all this data has been encrypted by the crypto package of nodejs using the AES-128-CBC algorithm. I use TypeOrm and I have defined a BeforeInsert and AfterLoad which encrypt/decrypt the data accordingly and all the results in the code are in plaintext.
right now due to some features like search, filtering/sorting(based on column) the performance of the application is very low.
Is there a way to put this encryption and decryption into some wrapper function into the Postgres and get the plaintext when we query the database? (same for inserting, insert plaintext but some function in Postgres get the values and encryption those and then insert it). can we have such functionality with Postgres? can we then use filtering and sorting in the code and database to take care of the rest? (decrypt the fields then filter/sort/search and return the plaintext result)
Avro files are coming from source as flowfiles in Apache Nifi and i want to fetch 'Password' column and hash it with 'SHA2-256' and give it as an output flowfile and finally put it in AWS s3.
You can modify the content of an individual "column" of Avro data (or any record format in NiFi) with the UpdateRecord processor.
You should not be doing this. SHA-256 is a cryptographic hash function but it is not designed for nor safe for "password hashing". Taking passwords, running them through a single SHA-256 digest function, and putting them on an S3 bucket is a very bad and very unsafe idea. Do not do it. Research how password hashing works before implementing any password protection.
I need to optimize disk usage and amount of data transferred during replication with my CouchDB instance. Does storing numerical data as int/floats instead of as string make a difference to file storage and or during http requests? I've read that JSON treats everything as strings, but newer JSON specs make use of different datatypes (float/int/boolean). What about for PouchDB?
CouchDB stores JSON data in native JSON types, so ints and floats are actual number types when serialised to disk. But I doubt you save much disk space over when that wouldn’t be the case. The replication protocol uses JSON and the internal encoding has no effect on this.
PouchDB in WebSQL and Sqlite store your document as string (I don't know what IndexedDb).
So to optimize disk usage, just keep less data. :)
I wanted to know what was the best practice followed for storing sensitive fields like email and phone number in the database. Lets say you want to search by email and phone number , and the application sends emails and sms to its users as well.
Because this data is sensitive you need to encrypt it. Hashing is not an option because you cant unhash it.
Encryption standards like Rjindael or AES makes the data secure, but you cannot search the db by it because the encrypted string produced for the same input is always different.
So in a case like this do I need to store both the hash as well as the encrypted field in the table ? Or is there some other strong encryption technique deployed for fields like these.
Check out CipherSweet. It's a very permissively-licensed open source library that provides searchable encryption in PHP.
Its implementation is similar to Ebbe's answer, but with a lot more caveats:
CipherSweet automatically handles key splitting, through a well-defined protocol.
CipherSweet supports multiple functional blind indexes (truncated hashes of transformations of the plaintext) to facilitate advanced searching.
More about the security implications of its design are available here.
Furthermore, the API is relatively straightforward:
<?php
use ParagonIE\CipherSweet\BlindIndex;
use ParagonIE\CipherSweet\CipherSweet;
use ParagonIE\CipherSweet\CompoundIndex;
use ParagonIE\CipherSweet\EncryptedRow;
use ParagonIE\CipherSweet\Transformation\LastFourDigits;
/** #var CipherSweet $engine */
// Define two fields (one text, one boolean) that will be encrypted
$encryptedRow = (new EncryptedRow($engine, 'contacts'))
->addTextField('ssn')
->addBooleanField('hivstatus');
// Add a normal Blind Index on one field:
$encryptedRow->addBlindIndex(
'ssn',
new BlindIndex(
'contact_ssn_last_four',
[new LastFourDigits()],
32 // 32 bits = 4 bytes
)
);
// Create/add a compound blind index on multiple fields:
$encryptedRow->addCompoundIndex(
(
new CompoundIndex(
'contact_ssnlast4_hivstatus',
['ssn', 'hivstatus'],
32, // 32 bits = 4 bytes
true // fast hash
)
)->addTransform('ssn', new LastFourDigits())
);
Once you have your object instantiated and configured, you can insert rows like so:
<?php
/* continuing from previous snippet... */
list($encrypted, $indexes) = $encryptedRow->prepareRowForStorage([
'extraneous' => true,
'ssn' => '123-45-6789',
'hivstatus' => false
]);
$encrypted['contact_ssnlast4_hivstatus'] = $indexes['contact_ssnlast4_hivstatus'];
$dbh->insert('contacts', $encrypted);
Then retrieving rows from the database is as simple as using the blind index in a SELECT query:
<?php
/* continuing from previous snippet... */
$lookup = $encryptedRow->getBlindIndex(
'contact_ssnlast4_hivstatus',
['ssn' => '123-45-6789', 'hivstatus' => true]
);
$results = $dbh->search('contacts', ['contact_ssnlast4_hivstatus' => $lookup]);
foreach ($results as $result) {
$decrypted = $encryptedRow->decrypt($result);
}
CipherSweet is currently implemented in PHP and Node.js, with additional Java, C#, Rust, and Python implementations coming soon.
Actually, encrypting the same message twice with AES with the same key and the same initialization vector (IV) will produce the same output - always.
However, using the same key and the same IV would leak information about the encrypted data. Due to the way AES encrypts in blocks of 16 bytes, two email addresses starting with the same 16 bytes and encrypted with the same key and the same IV would also have the same 16 bytes in the start of the encrypted message. Those leaking the information that these two emails start with the same. One of the purposes of the IV is to counter this.
A secure search field can be created using an encrypted (with same key and same IV) one-way-hash. The one-way-hash ensures that the encryption don't leak data. Only using a one-way-hash would not be enough for e.g telephone numbers as you can easily brute force all one-way-hash'es for any valid phone numbers.
If you want to encrypt your data, place the table on an encrypted filesystem or use a database that provides a facility for encrypted tables.
Encrypting data in the database itself would lead to very poor performance for a number of reasons, the most obvious being that a simple table scan (let's say you're looking for a user by email address) would require a decryption of the whole recordset.
Also, your application shouldn't deal with encryption/decryption of data: if it is compromised, then all of your data is too.
Moreover, this question probably shouldn't be tagged as 'PHP' question.