I have a database and an API in NodeJS, I create users with web app, and each user can create/update/delete data.
To secure this a bit, I need to encrypt data of users. So what I want is creating a pair of SHA256 public private keys each time a user is created.
Actually what I do is storing thoses keys in database, by encrypting them with global SHA256 pair of key.
So, in a nutshell, I have a global pair of key to encrypt each specific pair of keys for each user.
The fact is that seems to be not really secure because finally each user have his own encryption/decryption method stored in the database.
For example I can have 2 tables :
User table :
id_user | firstname | lastname | encrypted_data
-----------------------------------------------
1 | John | Doe | QMwmuCMmI..
2 | Jane | Doe | QMwmuCMmI..
...
Keys table :
id_user | public | private
------------------------------
1 | MIICIjA.. | MIIJrT..
2 | MIICIjA.. | MIIJrT..
...
So the link from John Doe to his public and private keys in simple.
A problem is that I can't ask for user to create a pair of private/public key and send me only public, because all need to be automatic, user don't have to do anything.
Another problem is that the application should be usable on any device, so the private key can't be stored in client side.
You can't just store them on the server as the attacker is on the server. The malicious hacker can easily find the keys (as they are un-encrypted), download the entire database and decrypt everything.
So, your best options are storing the keys on a completely different machine and let that machine do the encryptions and decryptions. That way the attacker would first have to break into your machine, and then the extra step of into the encryption/decryption machine. Not impossible, but hopefully much harder.
Basically, nothing is un-hackable. Making something as difficult as it can be for the attacker is the way to go.
Also, even if you strictly have to store the data in a database that your main server has access to, do NOT store them in the same table (as shown in your example how you kept them in separate tables). Keep the indexes of all data added to the user database the same, so that index 1 of users database will result in the correct key for that index (exactly what you did).
Another option would be cryptographic hardware. I'm not familiar with this, but I do know that some companies sell hardware that perform all of your cryptographic needs.
Disclaimer: This answer exposes two solutions to secure user data, but exclude SHA256 public/private keys from the equation (for, I believe, something more secure). It might not be an acceptable solution.
The "easy" way
I believe it's how Termius does it. In this case you would use the password to secure both the account access and encrypt data.
You would end up with something like that in database:
id
email/username
password_hash (to secure the account access)
... as many fields as you want, values are encrypted using a symmetric algorithm (i.e. AES) using the unhashed password
Advantage:
(optional) end-to-end encryption
instead of doing the decryption server-side, send the encrypted data as-is to the client. It will be the client app responsibility to do the decryption before the visualization, and to encrypt everything before sending it back to the server.
Drawbacks:
potential data loss
if an user loose its password, he also looses access to all its data.
the encryption key is stored in the database
sure, it's hashed. If the hashing algorithm is strong enough, it could be enough security for quite some time. But for some 3 letters agencies, it might be reversable relatively fast 🤷♂️
Warnings:
don't use the password hash as the key, only the unhashed password. Otherwise, the encryption key is stored "as-is" in your database (the password_hash field).
always use a salt when you compute passwords hashes (might sound obvious to say this in 2021, but still, I prefer to say it)
Even more robust encryption
I believe this is the way ProtonMail encrypts its users' data.
When you create a Protonmail account, you need to provide two passwords: the first one is used for account login, the second one for data encryption.
Advantages:
you don't need to store the second password in your database at all (hashed or not).
If the data can't be decrypted, it means the second password (aka the encryption key) is incorrect, its as simple as that.
(optional) end-to-end encryption
same as described in "the easy way".
Drawback:
potential data loss
if an user looses its second password, he also looses access to all its data. If he looses its first password, then he just looses access to "authentication". The typical "I forgot my password" flow works here, without raising any issue related to the encrypted data.
Measuring the pros and cons
Both solutions have a main drawback: potential data loss.
If you go for the Protonmail-like solution, I would say this drawback is compensated by a major advantage: the user is the only actor of its data's security, you store no hints about the encryption key in your database. Therefore, if you use industry-standard encryption algorithms you aren't responsible at al. Hackers would have no solution other than brute-forcing the encryption key of every user, one by one.
Plus, if you educate your users about the consequences of loosing their password, you can also claim that you are 100% unable to access their data at the same time. This is the highest possible level of data privacy, and this is what the world needs IMHO (no matter what type of data we are talking about).
Related
I am making a social media type website, and I store user details such as emails, names and other personal details.
I will be encrypting the personal details using an Encrypt-then-MAC concept. When the user registers, a cryptographically secure string will be made to use as the private encryption key. When the user selects a password, the encryption key will be encrypted using the password.
The password will NOT be stored in the database, but will be the private key to decrypt the encryption key used to encrypt the personal details. The only person who knows the password is the user. My question is: how can I store the encryption key once decrypted?
I have thought of having a table with one column for IP and another column for the encryption key, but some people close the browser window without logging out, therefore there would not a possible way to remove the entry from the database when they have finished their session on the website.
Another way would be to store it in a cookie, but that could be intercepted when sent back to the server. I would like to know if there is a secure, nearly foolproof way to store the encryption key, client side or server side.
Thanks in advance.
EDIT:
In reply to TheGreatContini's answer -
The idea of a "zero-knowledge web application" (in your blog) is a good one, however, for zero-knowledge, even the key cannot be stored in the database, this complicates things a bit, as you would then have to use the user's password as the key. Using the password isn't as secure, as it is a bit harder to verify the password to prevent data which has been "decypted with the wrong key" from passing. There is the concept of Encrypt-then-MAC but that only verifies if the data is legit, and will assume that a hacker has messed with some data and data cannot be trusted, however, as you cannot actually verify the password (the hash would not be stored as it is "zero-knowledge"), so the password may just be wrong.
Not sure I have the answer, but a few considerations:
(1) Sessions need to be timed out. Perhaps you can do this by periodically running batch jobs that scan the database looking for sessions that have lacked activity. This requires storing in the db the date of the last action from the user.
(2) Generally keys are higher value than the content they protect because the keys have a longer lifetime than the individual data elements that the protect (because the data may change or additional data may be added). Rather than storing the key in the db, you can store the decrypted contents in the database for the length of the session. Of course, this is provided that you did (1).
Perhaps I am not adding much beyond what you already know, however may be worth considering a blog I wrote exactly about this topic. The low level details start in the section "A second line of defence for all the sensitive data." Prior to that it mainly motivates the concept. Glad to see somebody actually doing something like this.
I have questions regarding the best way to secure the authentication of users.
I have come across a web application that encrypts the user password in the back end. However this encryption is done with the same key to all passwords.
Also, this key is "hardcoded" in the back end.
They (the app developers) claim that this is perfectly secure. However I have my doubts. I believe that this method can cause two problems:
The reason to encrypt passwords is to avoid access to the passwords in the event of an unauthorized database access. However if you store the key in the same server chances are they will also be able to obtain the key.
The same password will yield the same encrypted value therefore it will be easier to attack the system.
My questions are the following:
Am I right about my claims? And if it is really that insecure, should I warn them about the possible threat?
What would be the pros and cons of using a hash + salt approach instead?
Thanks!
I'm not sure if you might mistakenly mixed up encryption and hashing together. If the user's password is be encrypted and not hashed then there is potential for an attacker to steal all the user password in the event of a data breach.
There are a number of factors that you seem to be looking over when it comes to authentication. Firstly, any hashing should be done in the back-end and never in the front-end. Hashing in the front-end still leaves you vulnerable to hash attacks.
Some developers adopt a double-hash approach in which they hash the password in the front-end and then re-hash it in the back-end. I believe this is unnecessary, the front-end password should be covered by the HTTPS layer (TLS), however that is subject to discussion.
First, let's clarify two key terms before explaining how to securely store and authenticate users.
Encryption
You specify that the user's passwords are being encrypted, rather than hashed. What encryption functions do is map an input (user's password) to an output (encrypted password) in a 1-to-1 fashion, meaning that this is reversible.
This means that if the hacker gains access to the encryption key (private key), they can reverse the entire process easily.
Hashing
Instead, the user's password should be hashed on the server-side. Why? Because you can get away with comparing two hashes to check whether they match without ever storing the plain-text representation of that value.
And once again, you may be asking, "Why"? Well because hashing functions are one-way, meaning that the plain-text value cannot be reversed (well, they are very hard to), I shall not be going into to much detail.
What should I do?
The user's passwords should never be stored as plain-text in any part of the web server. Instead, you should be storing the user's hash. When the user then tries to login, you receive their plain-text password securely over HTTPS/TLS, hash it and if both hashes match, authenticate the user.
So a database table might look like so:
+--------------------------------------+
| ID | Username | Password Hash |
+--------------------------------------+
| 1 | foo | $2a$04$/JicM |
| 2 | bar | $2a$04$cxZWT |
+--------------------------------------+
Note, the hashes are truncated BCrypt hashes with 4 rounds (AKA - Invalid)
Now let's take an example, between Alice and our server. Don't take the data too literally.
Alice sends a request to login with her credentials, which first passes through our secure transport layer:
{username: "foo", password: "bar"} -> TLS() -> ZwUlLviJjtCgc1B4DlFnK -> | Server |
Our server receives this, then uses it's certificate key to decrypt this:
ZwUlLviJjtCgc1B4DlFnK -> KEY() -> {username: "foo", password: "bar"} -> Web Application
Great! Our credentials have been passed securely, now what? Hash that password and compare against what we got in our database.
BCRYPT('bar') -> $2a$04$/JicM
if ($2a$04$/JicM == user.get_password_hash) {
authenticate();
}
else {
return status_code(401);
}
We have now been able to authenticate a user, storing an irreversible hash value and without ever storing the plain-text value. This should have answered your first and second question.
Yes, your analysis is correct, this is insecure.
It would certainly fail any formal audit, e.g. PCI-DSS. the developers/operators may argue that the asset these accounts provide access to is of little value and hence they have no need to provide such a level of protection, however they still have a duty of care to their customers - and the majority of people will use the same password for different sites/services.
It does provide a means for users to "recover" their passwords without the complexity of creating an expiring OTP - however mailing a plain text password further undermines security.
Indeed, even if an attacker only had access to the encrypted password data (particularly if it contained a known encrypted value / does not use initialization vectors) it may be possible to derive the encryption key.
I am trying to build an application that stores user related information client side in localstorage. I am encrypting that data with a password given by user.
If I implement forgot password and generate a new password how can I get back my data that is encrypted on old password.
I am using sjcl for encrypting data. Is there any technique to encrypt data with 2 passwords??
What would be an ideal pattern for this scenario??
The conventional approach for this is called "key escrow." Basically, it means giving a copy of the key to someone that you trust.
If you won't trust anyone, then key escrow is not for you. Your only option is to make sure that you don't lose the one-and-only key. And this is a fairly common approach too. Many products that advertise secure storage emphasize this point. As examples, see Bruce Schneier's password manager "PasswordSafe," and LaCie's security-focused DropBox alternative, "Wuala."
There are accepted methods for encrypting data so that it could be decrypted with any one of several passwords. But I don't see how this helps; if you can't remember one password, how will you remember two?
Any other approach that pretends to avoid key escrow but still provides a backdoor to access your data if you lose the key is not secure and no one should trust it.
In my web application, I'm implementing a blacklist of passwords that user's won't be able to select. As mentioned in Jeff's God Login post, this is because some passwords are very commonly used and exist in readily available wordlists used by brute forcing tools.
I had planned to store the blacklisted passwords in a database table (in the clear), with an MD5 hash of it as a Functional Index. So, when the query is sent to the server, it looks like this:
SELECT 1
FROM blacklist AS a
WHERE MD5(a.password) = 'MD5stringOfPasswordSubmitted';
I don't think the "in the clear" storage of these passwords is an issue since the passwords are blacklisted. No user can set it to one of these.. so who cares if passwords in this table are stored in the clear.
But, is the transmission of this query to the database server a problem I should worry about?
If my user is trying to set their password, at the moment the app will MD5 the password, send that to the database to query this blacklist table. If no result is returned, the app will allow them to have it as their password (as long as other validation requirements are also met).
Is this something I should worry about?
Could this be implemented another way so passwords users are trying to set are kept secure still? Is it really necessary to resort to storing a salted hash via Bcrypt (like is in my user table) even just for this blacklist usage? Would using a YAML file in the local directory structure of my app have any of this same risk?
The aim is to prevent users choosing a password that's common, and checking that in a very fast way (hence MD5) as part of the validation process.
I don't see how the transmission of the query could be a problem. If your web application does the MD5 encoding and an attacker intercepts the communication with the database, there is no way he can get back the user's password from it.
MD5 is not safe for storing passwords as attackers may be able to find passwords that result in the same hash value (collisions), but there is no way to convert a hash value back to the clear text from which it originated.
If you're worried about leaking other sensitive data when querying the database, you could consider encrypting the communication channel.
I wouldn't worry about the transmission of passwords since they are hashed using a one-way algorithm (as user18044 pointed out) however expanding more on the weaknesses of MD5 - I wouldn't use that algorithm at all especially if you aren't using a salt. The reason why is because MD5 rainbow tables have been created for a very large set of possible combinations of passwords. In fact it's very likely that the password lists that you are referring to have been generated after searching MD5 tables or using online services that will give you a password by submitting an MD5 hash (if the password has already been cracked previously or is in some table). I would recommend using a salt or using another algorithm like SHA-256. Security is my specialty and I have a rig that is capable of cracking MD5 hashes at hundreds of billions per second however if there is a salt involved it either slows me down or stops me all together (if the salt is not known). That same rig can crack SHA-256 but it takes much longer for it to crack each one because SHA-256 iterates over itself enough times to make each iteration slow enough to make cracking less feasible.
As was already mentioned I would certainly use SSL either way to better protect all data that is transmitted.
But, is the transmission of this query to the database server a problem I should worry about?
This depends on your network topology. e.g.
Is the database server on the same private local network? Then this is low risk.
Is the database server on the same network, but shared with other machines and users. Then this is medium risk (you need to trust all the other machines and users).
Is the database server across the internet? Then this is high risk.
Depending on your accepted risk level, you may want to protect the connection to the database with SSL/TLS.
Even though you are hashing with MD5, MD5 is considered a weak hashing algorithm so if a MITM grabs the password as it is queried on your database, they could run it through a tool such as John the Ripper using a word list and intercept any password set by your users.
You could try and hash the password using another algorithm for this lookup - however this means implementing a "static salt" (called a pepper) on your system to avoid any hash comparisons of intercepted data. It may be much easier to simply connect with SSL/TLS to avoid this completely.
For storage of the password itself (once it has passed your check), definitely use a secure algorithm here such as bcrypt or scrypt.
I'd like to encrypt the user data I store on my server so that it is safe if my server were to be compromised. Is the following a good way to do it:
Each user's password is the passphrase to a GPG keypair. All the user's data is encrypted using the public key before being saved to the database. The password is discarded and the keypair is kept only for the duration of the session, so the data can only be decrypted when the password is supplied.
From the point of view of someone compromising your server, the only way to ensure the data is safe is the way you are doing, when the user have to supply the key to decrypt every time.
Any other technique leaves some weakness that could be exploited.
But you have to be sure the other way (I mean when user provides the password) is secure too, using https and preventions against session attacks, cross scripting and etc.
If you do not have specific hardware to do an extra line of protection as when they are generated pseudo-random numbers based on time (as do the banks tokens) or something like that, the best idea is to keep the keys with the user or to use a third part storage with greater security as the SQL on Azure or Amazon.
I used the same approach after thought a lot about where to put my encrytion keys to make data obscure even if my server got compromised. The only secure way I found was "with the user".
your approach protects you from only 1 attack: stealing your database (and only if you encrypted keys properly). if your server gets compromised they can take your ssl private key and listen your network traffic (with users' keys)