Hi (sorry about the wordy post)
I'm working on a web app that will store some of the users personal data.
I have looked into various ways of encrypting the data to protect it from intruders if the database or web server was compromised and think that creating and storing a unique “master key” encrypted with a password derived key for each user is the best way to go for my set-up.
The only issue I can see is with forgotten passwords. Currently, the users can reset a forgotten password by receiving an email containing a link with a unique token that takes then to the password reset form.
My idea is to add security questions to the registration page that will be used to create a 2nd derived key and use it to encrypt the users master key. The answers will not be stored in the database, just the key derived from them. The questions would be more opinion based than personal or factual.
This will mean there are two encrypted versions of the “master key”.
These questions would be added to the existing password reset form and the answers used to recreate the derived key and decrypt the master key. This would mean that the original master key can then be encrypted using the new password.
I have read many posts on how security questions should not be used to reset passwords but in this case it seems like an additional layer of security.
Does anyone see any potential problems with this approach or have any other suggestions?
Thanks
It is not clear who you are trying to protect the data from (your organisation or potential intruders), but I would presume that intruders is your main concern, since your organisation has access to the code of the application and therefore can view the passwords that the user is inputting (unless encryption/decryption takes place on the client side).
I would not encrypt the data using the passwords of the users. Instead, I would generate random key(s) for each user, store them on a separate server, and retrieve them once the user authenticates successfully. That separate server would have to be inaccessible from the outside world.
Note that user authentication should also take place on the second server as well. Do not return the keys without prior user authentication, in case the first server gets compromised. You can additionally encrypt these keys using the user passwords and on a different column, using a master key that you are holding in case someone forgets their passwords.
This way you will get stronger passwords as well, than what the average user is going to use.
Related
I'm working on implementing 2FA with Google Authenticator on our website. If I understand correctly every user will have their own secret code, which I will need on login to verify the 6 digit code they enter.
Storing these secret codes in the same database as the user passwords seems like a bad idea (although, if someone got a hold of the database we have bigger problems), is there anyway around it? Or should they just be treated like a password and encrypted?
You cannot hash the secret used to generate the TOTP code for Google Authenticator because you need the original secret to actually generate the code.
It pretty much is as you say, if someone has your database then you're in bigger trouble anyway. However this is how 2 Factor Authentication is supposed to work. If the password is indeed hashed securely and the attacker has only the TOTP secret then all they can do is generate 1 out of the 2 factors required to login and they have more work to do to break or steal the password.
If you'd rather not worry about storing these secrets for your users and letting a third party take care of it, can I recommend you take a look at Twilio's Two Factor Authentication API. Full disclosure, I work for Twilio, but if you don't want to worry about looking after secrets that you can't hash, as well as take advantage of other things like the Authy app (including secret transfer without QR codes) and the extra device data that is now available with authentications then it might be of interest.
You are right.
Is true that the 2FA increase the user security, but is not so strong at server side by definition. If a hacker or malicious employee with database access dump and publish the users secrets, the adtional security is gone.
What can be done ?
You can create a external isolated microservice, that receive a user hash and generate a 2FA secret keys, cryptography it and store in a key-value database, like elasticsearch. You can set the cryptographic key dynamically after the server start, to not store it hard-coded. You can store the database at a external server where the employees have no access other than via API.
This way if a malicious actor dump the elasticsearch database, they can not know what is the secret, and even if he gain access to the crypto keys he doesn't know who is the user that use that secret, because the key is the user id hash(not the user id).
Nothing is perfect, but 2FA targets to make harder to a attacker to have success. I think it help.
I am setting up a website to use Google's OAuth2 interface for user authentication. The website will store private data associated with each user - which I'm planning to encrypt.
If I implemented my own authentication method for the website, I could easily derive a key from the user's credentials (which include the user's password), allowing the data for each user to be strongly protected. But with OAuth2, I believe I can only receive an access token, granting that user permission for a period of time - the problem is that the access token value will change over time.
Is there a way that OAuth2 can provide me with an immutable secret tied to the user which I can use to derive a secure key from? Or is there some other method of creating a secure persistent secret using OAuth2?
--- Edit ---
In response to the questions and comments, here are some thoughts to consider:
All user information should always be protected with strong encryption and user authentication - the reason we read so many news articles about website & database hacks is because developers say "do we really need to secure that" and then respond with "No - because no-one but us will be able to access the database, security is hard, etc". The hacker downloads the database et violá. Credit cards, email addresses, phone numbers, passwords, you name it, then become compromised.
There are only two real secrets - one is a password stored in someone's head, the other is a strong random value that only the authorised user has access to (like a physical token). If you think a secure key can be derived from an email address alone, or that a secret needs to be stored in a database, you don't really understand security.
I guess what I was trying to discover was whether an OAuth provider can provide to the OAuth client an immutable value securely linked to both user and client - effectively, this would be a key that could only be unlocked by the OAuth provider using a combination of the user's secret (their authentication password) and the client's secret (used in the OAuth protocol). The client could then use this value to provide a reasonable level of security for the user's data.
Of course this implementation is not perfect from abuse, but implemented correctly, could provide a reasonable way to secure data whilst still using the good practices of the OAuth scheme.
The point of the token is that you can then use the token to obtain information from Google about the user. During the initial authentication, you will tell the user, and google, that you want to access certain information about the user:
https://developers.google.com/+/api/oauth
Assuming that the user allows you to access their information, such as their email address, you can then get their email address from google. Once you have their email address, you can generate a secret key for their user, store this in your user table, and use it to encrypt their data. Then, when they login again, you can lookup their email address and find their key.
Is there a specific need for the immutable information be 'secret'? Or is it just a key to identify a user?
If the information that you're storing is truly private, and you want to make it so that you can't access your user's data, then all you have to do is store the encrypted blob for your users. Once the user had downloaded their data, they can use their key to decrypt the data client-side.
My first question would be: Why do you want to derive your encryption keys from some tokens?
The tokens and your encryption keys could remain independent and can be associated to a user identified by a unique id. User authentication can be done by whatever way you need either via credentials or open ID authentication or something else. But, once a user is authenticated, your decryption APIs can fetch the decryption key associated with the authenticated user and do whatever decryption it has to.
This way you can potentially allow users to tie multiple open ID accounts with the same user similar to what Stackoverflow does. I can associate my yahoo, facebook and google accounts with my Stackoverflow user and can sign in with any of those providers. I can disassociate those accounts any time I want. But that does not affect my Stackoverflow profile and data.
So, it is not a good idea to derive your keys from something that is not constant and keeps changing. Instead, keep them separate.
If I implemented my own authentication method for the website, I could easily derive a key from the user's credentials
This schema has a terrible weakness - if a user forgets /resets its credentials, the key is lost forever
Is there a way that OAuth2 can provide me with an immutable secret tied to the user which I can use to derive a secure key from?
OAuth2 is an authorization protocol. It is not intended to give you any user secrets.
the Google's OAuth2 should provide a user info service returning username (email) and some id (sub).
However these are identity information, not any secrets. And mixing user credentials is imho bad idea (as already mentioned), using an external IdP (google) you will have no access to credentials tat all.
And now what?
My suggestion:
If you really want to encrypt user's data using a user-provided secret, let the user provide the secret or encrypt the data encryption key by the user itself (or using user's public key?). The user must be aware that if this secret is lost, the data will be unaccessible. It is quiet uncomfortable in long run. Some DMS systems use this approach to encrypt the stored sensitive documents.
if you want to encrypt data in rest (on the server, database), you may have an application specific key, imho best stored somewhere else (key vault, key management service,..). Indeed then there is a different risk profile (you have to protect the key,..) but it's much more convient for users. This is usually good enough along other security measures (even big enterprises don't ask for a separate password to encrypt your credit card number or email)
What you need is constant secure (random) key for each user that you could get from authentication service that gives OAuth2 endpoint (in this case - Google).
OAuth2 protocol itself does not provide such value - Authentication server uses generated values that are not constant. But OAuth2 does not prohibit from giving this value from Resource server (together with user id, email etc). So basically OAuth2 lets you secure data in the way you want, but Google, which you currently use, does not give this type of constant random value.
Also note, that this would not work if you would let user relate few accounts, like Google and Facebook, as they would give different random keys.
If you derive secret from credentials, this would also mean that resetting password would reset user account.
Furthermore, if you encrypt data like emails in this way, it becomes impossible to decrypt them without currently signed-in user. So emailing newsletter becomes practically impossible. You cannot query the data in SQL also.
I could only suggest some countermeasures:
Do not store sensitive data at all, or store it hashed. Passwords must be hashed, not encrypted. Do not store CC numbers, store tokens that represent them.
Use encryption with key, stored in another data source. This adds at least some security - attacker must get not only DB copy, but also encryption key.
As data is encrypted, storing it in database is no longer necessary. You can store encrypted data in files or some other source, where it is safer than in DB (no risk of SQL injections etc)
I've been having this same issue. So far, I can't find a secure way around it.
Basically, we need a per-site randomly generated secret provided only with implicit flow that can be used to derive credentials to access systems and decrypt data.
Because I want to protect the data from myself, I could write the client to salt/hash the secret in two ways, one way to retrieve the data and another to decrypt it.
Alas, this is not the case.
I could derive credentials from things in the basic scope of the oAuth and that would protect the data against me, but that leaves the user wide open for cross-site vulnerabilities, and besides, personally identifiable information makes for a poor secret.
The best I got is to use implicit flow oAuth2 to acquire the user's email address, randomly generate a client side secret, and force the user to email themselves the secret (as a recovery key), then store the secret in localStorage. Salt/Hash the secret + oauth scope variable to derive the credentials client side (so the user must be logged in) needed to access, encrypt and decrypt data.
If the user ever clears their localStorage, they need to go click the link in the recovery email, which places the secret back into localStorage.
This places the scope of vulnerability back on the client, but is resistant to public machines (would have to know who was logged in last, and get access to the localStorage token), allows for recovery, and weakly requires the user to be logged in. Still vulnerable to plugin injection attacks and physical access + knowing the user.
Update: I have decided to use some oAuth extensions (hello.js, folder APIs) to store the keys in the user account as files. It requires some permissions and some APIs to implement, but appears to be viable.
Imagine this situation: your users give you their credentials (username/password) to access a third party service. So you have to produce those credentials when connecting to the service, you cannot just store a salted hash.
The environment is Grails, with psql as DB. From the programmer point of view, ideally the user/password would still be part of the domain objects (so they are easy to use).
What would be the best practice to securely store them?
*(I'm not a security or crypto expert; this is my understanding based on my reading and research, but is very far from authoritative advice. Get the advice of web-app security professionals and get a proper security audit.)*
The best you can really do is have your app unable to decrypt them when the user isn't actively logged in.
Encrypt the credentials with a key based partially on the user's raw, unhashed password. You never store the user's password to log into your service on your systems, of course, so you only have access to it for a brief moment during authentication (and only then because the web hasn't caught up with the mid-90's and adopted sane challenge-response authentication schemes). You can, at the moment of user log-in, decrypt the saved credentials for the 3rd party services and store them in the volatile server-side session for the user.
For the encryption key you might hash the username and user raw password with a large-ish salt value you generate randomly for each (user,3rd-party-credential) pair and store alongside the encrypted credentials. The salt should be different to their salt used for their stored hashed password.
This is far from ideal and has all sorts of problems - but the credentials won't be accessible after the user's session expires or they log our and you purge their session.
It also means your app cannot act on their behalf when they aren't actively logged in, a limitation that may be a showstopper for you depending on your requirements.
A weaker option is to have a key for all user credentials that's manually entered by the sysadmin when the app is re-started. This key has to be stored in memory, but it's at least not sitting on the disk or in the database, so someone stealing a dump of your database will have a much harder time decrypting the stored credentials.
Neither option will help you if the attacker finds a way to trick your app into revealing those domain objects after decryption - or getting it to let them impersonate that user, getting it to perform actions on the 3rd party service on behalf of another user, etc. It'll at least protect against theft of database dumps and similar attacks, though.
One further recommendation: Rather than using pgcrypto to the crypto in the DB, do it on the application side. This means the DB never sees the key material required to decrypt the data; it can never be leaked into database logs, sniffed out of pg_stat_activity, etc.
I'm not sure the title is accurate, please feel free to rename the question if appropriate.
I'm thinking about a service that stores sensitive data on a remote server. For maximum privacy, the data would be encrypted on the client (using AES) and the encryption key not stored in any place, so that should the server be compromised, sensitive data would still be relatively safe.
Now the problem is, I need a second password to access the service, but this second password must be stored somewhere on the server.
This would be what happens when a user registers:
user picks a username/password (Password A) and an encryption key (Password B).
hash(Password A) is stored on the server
Password B isn't stored anywhere.
then:
user enters Password A => Password A is transmitted to the server
server checks hash(Password A) against the user db, grants access
client downloads preferences and encrypted data
User enters Password B -> encrypted data is decrypted (locally).
This looks good to me, but the drawback is that the user needs two distinct passwords. This is impractical and also carries the risk that users pick the same string for both, strongly reducing the model's effectiveness.
What I would like is to use two independent passwords but only ask the user a single one.
First attempt
The first idea I had is to ask the user a password, then split it in two and use the first part as service credentials (Password A) while the second part would be the encryption key (Password B).
This has the drawback of reducing the strength of the passwords: if the user-provided string is already weak/short, Password A and Password B would be even weaker.
Second attempt
Another option: use the user-provided password as encryption key (Password B) and use a hash (SHA-256) of that password as service credentials.
Registration:
user picks a single password PASS
hash(PASS) is transmitted to the server
server stores hash(hash(PASS)) with username in the user db
Then:
user enters PASS
service sends Password A = hash(PASS) to the server
server checks hash(Password A) against the user db, grants access
client downloads preferences and encrypted data
client decrypts encrypted data with Password B = PASS.
This means that
a hash of the encryption password travels over the network 2) a
a hash(hash(encryption key)) is now stored on the server, i.e. alongside the encrypted data
Does this significantly reduces the security of the system? I.e. when an attacker gains access to the server, is it easier to decrypt the sensitive data when knowing hash(hash(encryption key))?
Is there another (better) way to get two independent passwords starting from a single string?
I would go a different approach. Using openid to authenticate to your system. This way you do not have to transmit a password to your server at all.
On your first approach, you should only transmit the hash.
If I remember correctly, LastPass uses this system:
User enters email address and password
Password is hashed and the hash used as key to encrypt the user's sensitive data
Password hash + email is hashed into a second password which is used to log into the server and upload the encrypted data blob
I may have it backwards which hash is used for which, but it sounds like you want something very similar. The whole thing was explained in detail in Security Now #256, if you're interested.
The advice to use a secure password scheme is correct. What you might want to do is to make a small change before passing the user's text to the password scheme. If the user enters "password" then pass "passwordLOCAL" to the local password scheme and "passwordREMOTE" to the remote password scheme.
That allows the user to enter a single password yet still have two different local and remote passwords. Don't actually use "LOCAL" and "REMOTE" of course, far too insecure. Use two different random strings, much like two different salts.
As suggested by #Tim in his answer - do not create a new user authentification scheme (your own user-id & password combination).
Based on what you are building this for, you basically have 2 options:
If this in an corporate internal application just integrate to Active Directory or whatever internal user database and authentication backend you have
If this is public, use OpenID (if you need just authentication) or OAuth if you also need authorization.
Please note that your application does not have to be web based in order to use them. (You do need internet access, though). See this question for details
For the encryption keys, use some sort of Key Derivation Function (also known as PRF+) for the key derivation - preferably with other components as well. Feed the PRF/KDF for example the following (concatenated):
User's unique identifier from the authenticator (OpenID, AD, whatever)
a system-wide application identifier string (e.g Francescos secureapp keypad)
per user random data stored with the user information (e.g. 16 bytes from the system random source)
user's encryption password - this is the single password you have to care about.
This is somewhat similar to AUTH payload calculation in the IKEv2 protocol
This way the potential attacker would need to access bits of information from various locations in order to be able to decrypt user's sensitive data.
Also, don't store the user's encryption password on the server side. The client should always ask it (or cache it for certain amount of time) from the user and send it (in plain or hashed) over an encrypted channel to the server whenever requesting something that needs [en|de]cryption.
Furthermore, if you want to enforce some sort of a password policy on the encryption passwords, see this comic and this answer for reference how it should be done.
You could generate and store an IV (or two, if needed) on the server, and send it to the client to HMAC. This produces the encryption keys that are needed.
There are a lot of questions on Stack Overflow about how to store user passwords, and the general advice of course is to hash the passwords and compare hashes.
However, imagine you are building a shrinkwrap intranet application (like SharePoint) that people deploy in their own environments. And suppose it requires a username/password combination to access an external service via HTTP (solutions that rely on API keys or federated security aren't supported).
In this case, we can't hash the password because we will need to pass the original password to the web service that we call. Encrypting would be the second best solution, but what would we use for the encryption key? If an attacked compromised the database, presumably they would have access to whatever key is used to encrypt the data in the first place?
If it was really a requirement for you to get the plain-text version of a stored password, how would you approach the problem in the most secure way?
This is actually a really interesting question. I'll join in.
You should encrypt it when storing it. No matter how you look at it it's better than storing it in plain text. Let's say an attacker finds an sql injection ad dumps the db, he still don't hold the encryption key. On the other hand, if he gets access to the server he will probably also find the encryption key.
To improve it a bit, you could store the encryption key in the server configuration. Assuming you are using Apache, you could use SetEnv.
I in my environment are required to enter the encryption key when Apache starts, this is then stored as en environment variable, so the key isn't really stored anywhere on my server.
There is no way, unless you require the user to enter a key to decrypt the password that you will be 100% safe.
You could generate the encryption key from the user's password. (Not their password for the external service—their password for your service.) Since you're not storing their password in plain text, an attacker who compromised your database wouldn't be able to decrypt the passwords. The downside is that you have to ask them for their password (for your service) whenever you need their external password.
You have the question inverted. The problem is not how to let the consumer "view" the password; the problem is how to let the consumer verify authentication.
In your implementation provide a means by which the consumer can provide a password and a username and get either a yes or a no. Then you continue to store encrypted (not hashed) passwords in the database.