The subject of how to store web site users passwords in tables has come up several times on SO and the general advice is to store a hash of the password, eventually an HMAC hash. This works fine for Basic authentication or for forms based authentication (really the same thing). My problem is that I must provide also Digest authentication, aimed at the automated tools connecting to my service. I've been looking at this problem and as I see it, the only hash I can store is the HA1 part of the Digest: the hash of username : realm : password. This way I can validated both Basic/forms and Digest.
My problem is that I don't see any benefit in doing so. Now indeed an attacker cannot use Basic or forms based authentication if he gets hold of my password table (since he only has the hashed value and he needs to send the clear password), but nothing prevents him from using Digest authentication and give a valid response to my service challenge: he simply starts from the pre-computed HA1 from the table and continues the crafting of the response from there (ie. the same thing I'd do to validate a user on the back-end).
Am I missing something? Does the addition of Digest requirement basically makes the storing of hashed passwords a no-op from security pov, an obfuscation at best?
The reason I am using pre-computed hashes is not protection against attacks, but to secure users privacy.
Attacker can indeed authenticate, but he cannot (easily) see password of my precious users and compromise other services they are using etc.
Related
I'm working on implementing 2FA with Google Authenticator on our website. If I understand correctly every user will have their own secret code, which I will need on login to verify the 6 digit code they enter.
Storing these secret codes in the same database as the user passwords seems like a bad idea (although, if someone got a hold of the database we have bigger problems), is there anyway around it? Or should they just be treated like a password and encrypted?
You cannot hash the secret used to generate the TOTP code for Google Authenticator because you need the original secret to actually generate the code.
It pretty much is as you say, if someone has your database then you're in bigger trouble anyway. However this is how 2 Factor Authentication is supposed to work. If the password is indeed hashed securely and the attacker has only the TOTP secret then all they can do is generate 1 out of the 2 factors required to login and they have more work to do to break or steal the password.
If you'd rather not worry about storing these secrets for your users and letting a third party take care of it, can I recommend you take a look at Twilio's Two Factor Authentication API. Full disclosure, I work for Twilio, but if you don't want to worry about looking after secrets that you can't hash, as well as take advantage of other things like the Authy app (including secret transfer without QR codes) and the extra device data that is now available with authentications then it might be of interest.
You are right.
Is true that the 2FA increase the user security, but is not so strong at server side by definition. If a hacker or malicious employee with database access dump and publish the users secrets, the adtional security is gone.
What can be done ?
You can create a external isolated microservice, that receive a user hash and generate a 2FA secret keys, cryptography it and store in a key-value database, like elasticsearch. You can set the cryptographic key dynamically after the server start, to not store it hard-coded. You can store the database at a external server where the employees have no access other than via API.
This way if a malicious actor dump the elasticsearch database, they can not know what is the secret, and even if he gain access to the crypto keys he doesn't know who is the user that use that secret, because the key is the user id hash(not the user id).
Nothing is perfect, but 2FA targets to make harder to a attacker to have success. I think it help.
Imagine this situation: your users give you their credentials (username/password) to access a third party service. So you have to produce those credentials when connecting to the service, you cannot just store a salted hash.
The environment is Grails, with psql as DB. From the programmer point of view, ideally the user/password would still be part of the domain objects (so they are easy to use).
What would be the best practice to securely store them?
*(I'm not a security or crypto expert; this is my understanding based on my reading and research, but is very far from authoritative advice. Get the advice of web-app security professionals and get a proper security audit.)*
The best you can really do is have your app unable to decrypt them when the user isn't actively logged in.
Encrypt the credentials with a key based partially on the user's raw, unhashed password. You never store the user's password to log into your service on your systems, of course, so you only have access to it for a brief moment during authentication (and only then because the web hasn't caught up with the mid-90's and adopted sane challenge-response authentication schemes). You can, at the moment of user log-in, decrypt the saved credentials for the 3rd party services and store them in the volatile server-side session for the user.
For the encryption key you might hash the username and user raw password with a large-ish salt value you generate randomly for each (user,3rd-party-credential) pair and store alongside the encrypted credentials. The salt should be different to their salt used for their stored hashed password.
This is far from ideal and has all sorts of problems - but the credentials won't be accessible after the user's session expires or they log our and you purge their session.
It also means your app cannot act on their behalf when they aren't actively logged in, a limitation that may be a showstopper for you depending on your requirements.
A weaker option is to have a key for all user credentials that's manually entered by the sysadmin when the app is re-started. This key has to be stored in memory, but it's at least not sitting on the disk or in the database, so someone stealing a dump of your database will have a much harder time decrypting the stored credentials.
Neither option will help you if the attacker finds a way to trick your app into revealing those domain objects after decryption - or getting it to let them impersonate that user, getting it to perform actions on the 3rd party service on behalf of another user, etc. It'll at least protect against theft of database dumps and similar attacks, though.
One further recommendation: Rather than using pgcrypto to the crypto in the DB, do it on the application side. This means the DB never sees the key material required to decrypt the data; it can never be leaked into database logs, sniffed out of pg_stat_activity, etc.
Am I correct that OAuth 1.0a credentials need to be stored in plaintext (or in a way that can be retrieved as plaintext) on the server, at least when doing 2-legged authentication? Isn't this much less secure than using a username and salted+hashed password, assuming you're using HTTPS or other TLS? Is there a way to store those credentials in such a way that a security breach doesn't require every single one to be revoked?
In more detail: I'm implementing an API and want to secure it with OAuth 1.0a. There will possibly be many different API clients in the future, but the only one so far has no need for sensitive user data, so I'm planning to use "2-legged" OAuth.
As I understand it, this means I generate a consumer key and a shared secret for each API client. On every API request, the client provides both the consumer key, and a signature generated with the shared secret. The secret itself is not sent over the wire, and I definitely understand why this is better than sending a username and password directly.
However, as I understand it, both the consumer and the provider must explicitly store both the consumer key and the shared secret (correct me if I'm wrong), and this seems like a major security risk. If an attacker breached the provider's data store containing the consumer keys and shared secrets, every single API client would be compromised and the only way to re-secure the system would be to revoke every single key. This is in contrast to passwords, which are (ideally) never stored in a reversible fashion on the server. If you're salting and hashing your passwords, then an attacker would not be able to break into any accounts just by compromising your database.
All the research I've done seems to just gloss over this problem by saying "secure the credentials as you would with any sensitive data", but that seems ridiculous. Breaches do occur, and while they may expose sensitive data they shouldn't allow the attacker to impersonate the user, right?
You are correct. oAuth allows you however to login on the behalf of a user, so the target server (the one you access data from) needs to trust the token you present.
Password hashes are good when you are the receiver of the secret as keyed-in by the user (which, by the way, is what effectively what happens when oAuth presents the login/acceptance window to the user to generate afterwards the token). This is where the "plaintext" part happens (the user inputs his password in plaintext).
You need to have an equivalent mechanism so that the server recognizes you; what oAuth offers is the capacity to present something else than a password - a limited authorization form the use to login on his behalf. If this leaks then you need to invalidate it.
You could store these secrets in more or less elaborated ways, at the end of the day you still need to present the "plaintext" version t the server (that server, however, may use a hash to store it for checking purposes, as it just needs to verify that what you present in plain text, when hashed, corresponds to the hash they store)
There are a lot of questions on Stack Overflow about how to store user passwords, and the general advice of course is to hash the passwords and compare hashes.
However, imagine you are building a shrinkwrap intranet application (like SharePoint) that people deploy in their own environments. And suppose it requires a username/password combination to access an external service via HTTP (solutions that rely on API keys or federated security aren't supported).
In this case, we can't hash the password because we will need to pass the original password to the web service that we call. Encrypting would be the second best solution, but what would we use for the encryption key? If an attacked compromised the database, presumably they would have access to whatever key is used to encrypt the data in the first place?
If it was really a requirement for you to get the plain-text version of a stored password, how would you approach the problem in the most secure way?
This is actually a really interesting question. I'll join in.
You should encrypt it when storing it. No matter how you look at it it's better than storing it in plain text. Let's say an attacker finds an sql injection ad dumps the db, he still don't hold the encryption key. On the other hand, if he gets access to the server he will probably also find the encryption key.
To improve it a bit, you could store the encryption key in the server configuration. Assuming you are using Apache, you could use SetEnv.
I in my environment are required to enter the encryption key when Apache starts, this is then stored as en environment variable, so the key isn't really stored anywhere on my server.
There is no way, unless you require the user to enter a key to decrypt the password that you will be 100% safe.
You could generate the encryption key from the user's password. (Not their password for the external service—their password for your service.) Since you're not storing their password in plain text, an attacker who compromised your database wouldn't be able to decrypt the passwords. The downside is that you have to ask them for their password (for your service) whenever you need their external password.
You have the question inverted. The problem is not how to let the consumer "view" the password; the problem is how to let the consumer verify authentication.
In your implementation provide a means by which the consumer can provide a password and a username and get either a yes or a no. Then you continue to store encrypted (not hashed) passwords in the database.
Let's assume I must store user's sensitive data, which was optionally encrypted on the client side.
Encryption (optional) should be done with user's passphrase.
User login (optional) should be done with user's password.
Notes:
A plain-text password is not stored on the server or transferred over the network.
My options and their drawbacks:
1. No authentication, Client-side authorization:
The server gives the data to everyone, but only the original user have the means to decode.
Data can be used by anyone to try to crack the encryption - not the best way to secure it.
2. Server-side authentication, no authorization:
Server stores user's password to access the data, and only gives the data to the user that can provide the right password.
Users don't trust the network for transferring their data without encryption.
3. Authentication and authorization:
Server stores user's password to access the encrypted data, the encryption is done using the passphrase that is different from user's password.
Good security, but users don't want to remember two passwords.
4. Authentication vs. Authorization:
Server stores user's password to access the encrypted data, the encryption is done using the same password.
Users are happy. Some security concerns.
I prefer the latest fourth option, but my concern is:
What if the server will get compromised, how can I be sure that encrypted password and encrypted data can't be used together to break the encryption?
How can I make it harder to break the encryption?
Some thoughts:
Use different encryption algorithms for password and data.
Add fixed string to the end of the user's password before encryption.
Pad user's password to some length.
EDIT:
The system should be very similar to a backup system that should be secure from all sides: the server should not be able to read the data, only the original client should be able to access the data and man in the middle attacks should be prevented. So if someone hacks the server authentication or the client encryption, the data should not be revealed.
It should be web based, so the man in the middle attack should be prevented with HTTPS.
To prevent server hacks revealing the data, the data is encrypted in client-side.
To prevent client encryption tampering, the access to the data should be protected on the server side with some log in and password or a token (may be unique URL).
#Vitaly, permit me to clarify some terms before I answer, as you seem to be using a different meaning for some than is commonly used.
Authentication - the process of proving who you are (more accurately, that you own the identity you are claiming).
Authorization - the mechanism used to restrict, control, and grant access.
Encryption - a mechanism for protecting data, even from someone who has access to it.
Now, allow me to rephrase your options, and then I'll suggest something else:
No Authentication, No Authorization, Client-side encryption
Server-side authentication, Server-side authorization, Server-side encryption
Server-side authentication, Server-side authorization, Client-side encryption
Server-side authentication, Server-side authorization, Client-side encryption using server credentials.
Now, I think it can be clearer where each one stands.
In general, you really want to follow the "best practice" (dont get me started on those) principle of "Defense in depth", i.e. dont use only encryption, or only access control, instead use both! But, as you pointed out, this can be in contrast (if the user is required to remember TWO passwords) to another principle, "Keep Security Simple".
Without trying to be TOO annoying, you didn't give much information in the way of your environment. For example, is this e.g. a Web application? If so, why is SSL/TLS not enough encryption for you? Or is this a question of users uploading personal data that you (and your system) should not see either (e.g. a backup-type service)? In which case client-side encryption would be necessary...
So, (finally) my proposed options, depending on your environment / requirements:
If you can, rely on secure protocols (e.g. SSL/TLS) for encryption. Use server-side authentication + authorization, protocol encryption.
If your system needs to further protect this data, e.g. credit cards (note that I am not currently a PCI:QSA ;) ), use the previous option, and in addition server-side encryption using a server-generated encryption key (NOT the password) (and of course protect that).
If the data needs to be protected FROM your system, you will need to do client-side encryption IN ADDITION to server-side authentication+authorization (your option 3 as I restated it).
However, you don't necessarily need to force the user to remember an additional password/phrase. Again, depending on your environment, you might be able to consider some form of key stored on the client, e.g. a certificate in the user's certificate store / keyring, or even stored in a protected configuration file; a key based on biometric data (not easy but i've seen this done successfully, though it has its own set of issues), out of band key distribution (e.g. via cellphone), etc. This would enable you both to use strong keys, prevent the server from accessing those keys, not require the user to remember two keys, and doesn't re-use a single password for different usages in different contexts.
You could take a look at zero-knowledge protocols for authentication, in particular to the Secure Remote Password protocol, which makes it possible to perform password-based authentication without revealing the password to the server. This way the same password can be used both for authentication and for deriving a data encryption key.
Also, you could take a look at the Clipperz online service, which implements something similar to your needs and is also open source.
Use option one and make the URL for the data contain a long random string. Anybody who knows the random string can get the data. Of course, only the client who created the data is going to have that URL right off.
If someone wants to give someone else revokable access, allow them to generate a new random URL and provide a means for them to name that random URL and revoke its ability to get at the data.
Capability based security is easier to get right, more flexible and makes more sense to users. There is a really excellent YouTube video about capability based security and a nice website with some essays about it.