Related
does using hash functions and algorithims suffice the need to encrypt the data, while communicating with the server
I am applying salt mechanism in my project and I will be concatenating the salt with the entered password, and then hash them all.
do I still need to encrypt the result?
The usual workflow for a website to transmit user passwords looks like this:
The client sends the password plaintext to the server.
The transmission is done with an encrypted connection (HTTPS/SSL), to prevent a ManInTheMiddle attack.
The server calculates a hash of the plaintext password, and this hash is then stored in the database. It is not necessary to encrypt the hash any further.
Make sure you use a random unique salt for each password, and a slow hash function with a cost factor for hashing passwords. Good algorithms are BCrypt, PBKDF2 or SCrypt.
Storing passwords
To store user passwords securely, you need 3 things:
Do not store the plain password, store a hash instead
The hash makes it extremely difficult to recuperate the password even if an attacker manages to capture the entire database
To prevent the use of rainbow tables, you need a salt
The salt is stored in the clear (can be along with the hash) and is random, one for every user and you can easily chose a few one whenever the user changes their password.
You need a SLOW hash, not a fast hash
What are fast hashes: MD5 (consider it broken), SHA-1, SHA-2, ... are unsuitable as the attacker can perform them too fast and use dictionary attacks to try common passwords and find that way up to 95% of you user's passwords in mere hours on modern rigs.
How slow does it need to be ? As slow as you can afford.
And there's a rule 0: Do not invent crypto yourself, you will make serious mistakes before you know it.
Other privacy sensitive info
You're most probably also storing other sensitive information of your visitors in addition to the passwords (email addresses, IP addresses, names, postal address, ...), CC numbers (you better not go there), ...
You need to protect that as well and using hashes isn't the way to do that in most cases. Some of these have requirements and regulations of their own (e.g. Credit Card data is regulated by the issuers who'll force you to be compliant with PCI-DSS).
In essence you need to do a risk analysis and manage that risk by either accepting it ("so be it"), transferring it ("get insurance"), avoid it ("we're not storing that"), or mitigating it ("we're going to improve our way of working").
encryption
Why the media will make you believe there's a "magic" solution in that incomprehensible "encryption" thing, in reality it needs to be done right and in the right conditions to have any meaning at all.
E.g. If you encrypt the entire disk of a server: it will not protect you from an attacker abusing your server scripts and getting to the database (as the database engine and webserver scripts have access to the decrypted disk already)
So, you really need to go back to the risk analysis and chose the measures there instead of getting ahead of yourself and suggesting encryption as a tool that's unlikely to help you for your biggest risks.
I'm building a REST API for a mobile app. I need to handle authentication. As far as I could find, these are the best practices regarding password storing.
On the client side-register
Let user choose a password
Create a sufficiently long random salt from a strong random number generator
Append the the salt to the password and hash it.
Send the hash and salt to the server and store it in database
Server side-log in
Expose a method that returns user's salt for a given userid(email)
Client side-log in
User types in his userid and password
Retrieve the salt for the given userid
Append the the salt to the password and hash it.
Send the hash to the server, check that it's the same and authenticate user
Obviously the week point in this approach is that an attacker can see the salt for any user effortlessly. Is this ok? Obviously I can store the salt on the client side as well but if e.g. the user is logging in from a different phone he has to be able to retrieve the salt.
The alternative to this is to send the password in plaintext and handle all the salting and hashing on the server side. I would use HTTPS.
Which of these approaches is preferable? Is there a third one I'm missing?
Obviously the week point in this approach is that an attacker can see the salt for any user effortlessly.
No, the weak point is that your server is trusting the client. Here is MSDN's brief and pointed take on the situation:
You should never trust user input directly, especially if the user input is anonymous. Remember the two golden rules: never trust user input, and always check data as it moves from an untrusted to a trusted domain.
Source: Do Not Trust User Input Directly
Here you have user data moving from their untrusted client to your server. You need to verify their input on your server. There is no attack that you can prevent by performing the generation of the salt and the hashing of the password on the client. The ONLY way to secure the server-client communication is with secure handshake key exchange for asymmetric encryption. This occurs over HTTPS, and once the link is secure, then sending the password in plaintext is no less dangerous than sending the code to the client to perform the hashing client side.
If you do not secure the connection to the client with HTTPS, you are always vulnerable to a Man-in-the-Middle attack. Even though the password is nominally obscured when returning to your server, an attacker could instead impersonate your server to the client, sending an HTML form that skips the hash or sends the password in plaintext to their server as well.
Exposing the salt in your scenario is an additional security hole, but it is minor next to the glaring hole that would allow an attacker to get the password directly. I can explain this attack further if you desire, but it is a totally pointless attack unless the password itself is better secured; and once that happens, you have no need to expose the salt in the first place.
Responding to comments, it is true that salts can be made publicly known without reducing security if you are certain that they were generated correctly. For a salt, that means they are of sufficient length, sufficiently random, and in some schemes that they are cryptographically secure. The problem is that you allow the client to generate the salt, which means, I think we all agree now, that they can't be trusted. Maybe an attacker corrupted an external library you're relying on for PRNG. Maybe some user devices have an insufficient entropy pool. Maybe their device isn't secure against side-channel attacks. If you're making an Android App, there are certainly some very insecure devices out there. In Apple's walled garden, these risks might be tolerably low, but you can never eliminate them.
Like I said originally, however, these risks are minuscule next to the real dangers of your scheme. Namely, you're transmitting the hashed passed in the clear to your server. Even if you hash it further before storing it in your database, so a SQL injection can't just dump all the user passwords, you're still vulnerable to Man-in-the-Middle attacks. Unencrypted HTTP traffic can be observed and recorded by any router it passes through. Once an attacker observes a hashed password, they can re-send the same request to your server, and be authenticated incorrectly.
You should strictly go for the alternative, "to send the password in plaintext and handle all the salting and hashing on the server side. I would use HTTPS."
Of course the sentence above is highly contradictory: if you would be using HTTPS then the password would not be in plaintext. You should use HTTPS.
At the server you can store the randomly generated salt as plaintext in the database. You should not opt for a normal hash but a password hash - such as one generated by PBKDF2 - instead.
The salt is there if your DB gets stolen: the salt will make sure that identical passwords are not revealed to be identical. Furthermore the salt will make it impossible to use rainbow tables with precomputed password hashes.
Two important things to notice: for web clients you will have to rely on the trusted TLS certificate store for creating a secure TLS connection. Furthermore, your client may not have access to a good random number generator.
I have been looking for a good explanation of how to implement a password login system in a typical website environment. I have read some great wikipedia articles and SO Q&A and blogs etc but they always seem to focus on purely generating the hash rather than the whole process of creating hash sending which parts of it, storing which parts of it, what the server side code does with it etc. If there is already a good answer on SO I apologise for reposting, and please link.
My current understanding is:
1) A new user creates a new account on your website. They enter a "password", the client side code then generates and appends a long random string "salt" to the end and generates a hash -> BCrypt(password+salt) for example. The client code then sends the full hash plus the unhashed salt to the server.
2) The server stores the full hash and the unhashed salt in the users entry in a DB.
3) During the user login they type their password which is then hashed with a salt again,
Question 1) How does the client side code generate the same 'random' salt value for each user?
Question 2) at this point does the client side code just send the full hash without the salt?
Question 3) what does the server side do with the full hash once it has received it? (simply compare the sent full hash with the stored full hash? If that's the case then can't an attacker upon breaking into the db and getting the stored full hash values just use them directly to send to the server to log in? This is based on my assumption that the log in process essentially involves the server comparing the full hash sent from the client with the full hash stored in the db.
Question 4) should passwords always be sent over secure connection? or does salting and hashing them make it ok for anyone to see?
You are confusing the purpose of the hashing. It is not intended to secure the password for wire transmission. The client does not generate the hash. the purpose of the hash is to prevent an attacker who compromises your database from being able to quickly use a pre-generated hash lookup table to determine what your user's passwords are.
A trivial example follows- as #jhoyla points out in the comments below, industrial grade production schemes are even more complex.
To create an account:
The client establishes a secure (encrypted, e.g. SSL) connection with the server, and sends the username and password, usually in plaintext (which is OK, because it is encrypted).
The server generates a random salt, appends it to the password, hashes the result, and stores the hash and the unhashed salt value.
To log in:
The client establishes a secure (encrypted, e.g. SSL) connection with the server, and sends the username and password, usually in plaintext (which is OK, because it is encrypted).
The server retrieves the salt from storage, appends it to the password, hashes it, and compares the result to the hashed password in storage. If they match, the user is logged in.
To establish why we do this, imagine that I have successfully attacked a website's database server and downloaded the database. I now have a list of usernames, probably email addresses, and password hashes. If the passwords are not salted, then there is a very high probability that many of the hashes will be the same (because many people use the same weak passwords). I know that the likelihood of one of those users having that same weak password on (for example) their email account is quite high. So I go to work and hash the whole dictionary, plus many other likely passwords, looking for a hash that matches one of these popular ones. If I get a hit, I've just broken a bunch of passwords. If I was smart, I'd have generated this list in advance so that I can do it quickly.
Now imagine that the passwords are salted. Now, even if two people use the same password, a different salt will have been generated for each of them, and the resulting hashes will be different. I have no way of knowing which passwords are weak, common passwords, and which ones are strong passwords. I can try my dictionary attack by appending the salt to each possible password, but the difficulty (in terms of time) of cracking a password has now gone up exponentially.
never ever implement it yourself! if you need it just for learning then #Chris answered you. but if you need for for a working software then don't do it. every language has security libraries and every data store (ldap, database) has password storing mechanism already implemented. use it, don't invent the wheel again because you will most probably miss some detail
When authenticating a user to a website, should the hash generation and comparison be done in the database or the website?
My argument is the website should pass the user supplied password (possibly encrypted by the web server) to the database. The database then re-encrypts it with the salt and compares the hash's. The database the responds to the web server whether the user's credentials are valid or not. This way, the very minimum ever leaves the database, essentially either a yes or no, none of the stored credential info. Downside is, the database has to do more work.
The other argument is that the work should be done in the web server. Here the web server would create the hash and request the stored hash from the database and compare. In this situation the salt needs to be passed from the database back to the web server for the hash to be created. but, work is shared as # of web servers increase.
Personally I see the second method as a potential security risk. Should the web server be compromised, salts and hashes can be requested from the database and easily cracked.
What is the best practise for performing the above operation? Am I overlooking/missing something?
Thanks
The first problem I suspect you will run into (and it's a big one) is that your database does not have a password hash function. Sure, it probably has MD5() and SHA1() but these are cryptographic hash functions. Does it have bcrypt() or scrypt() or PBKDF2()?
Using a cryptographic hash function rather than a password hash function is what meant that the LinkedIn passwords could be cracked so quickly. If you don't use one of the above functions then you will be similarly vulnerable if your hashes are leaked.
Going on to answer your question assuming that your database does support a password hashing algorithm (using bcrypt simply because I have to pick one). The two alternatives are:
Hashing in the database:
$db->query("SELECT COUNT(*) FROM users WHERE username = '?' AND password = BCRYPT(?, (SELECT salt FROM user WHERE username = '?'))", $username, $password, $username);
if($row['count'] != 1)
{
// Not authenticated. Throw exception.
}
In this case, the raw password is sent to the database and a simple yes or no (1 or 0) is returned. This database communication can be encrypted. The hash and salt are never held in the application.
Hashing in the application:
$db->query("SELECT username, salt, password FROM users WHERE username = '?', $username);
if(bcrypt($password, $row['salt']) != $row['password'])
{
// Not authenticated. Throw exception.
}
In this case, the hash and salt are pulled from the database into the application and the hashing of the raw password and comparison is done there. The communication to the database can still be encrypted. The raw password is never held in the database memory.
For efficiency, we can assume that both hashing algorithms are written in C (or some compiled language) and are possibly provided by the OS so take the same time. The application hashing option receives more data over the wire and the database hashing option sends more and has a more complex query (essentially two queries, one to get the salt and one to effect the comparison). It may not be possible to use an index the way I have written that query but the query could be rewritten. Since the size of the data in both cases is likely still one TCP packet, the speed difference will be negligible. I would call this one a win for the application hashing option due to the subquery.
For exposure. I would consider the raw password to be more sensitive than the hash and the salt together. Therefore, limiting the exposure of the raw password seems like the safer bet, making application hashing the best practice.
There's a really good article on how to store passwords securely here:
http://throwingfire.com/storing-passwords-securely/
You are overlooking the purpose of a salt.
A salt is used to prevent a dictionary attack against hashed passwords. If your password is "peanut" and hashes to 12345, then I can pre-generate a list of hashes for every word in a dictionary (including your password) and quickly find your password by doing a lookup against my pre-generated set of password hashes. This is what happened to LinkedIn recently. If the passwords are salted, I'd have to pre-generate a dictionary for each salt value after compromising the database, which would be prohibitively expensive.
Furthermore, proper randomly-generated salts prevent an attacker from knowing that you and I have the same password (without the salt, we'd have the same hash).
My point is that the salts are not intended to be a secret. They are not public information, but an attacker getting access to the salt values + the hashes does not necessarily mean that the passwords have been compromised.
A good rule of thumb for computer security is that if you have to ask, you shouldn't do it yourself. But if your concern is exposure of password details if the web server is compromised, then one approach is to move authentication onto its own system, and don't give the web server access to the password database at all.
I noticed that most sites send the passwords as plain text over HTTPS to the server. Is there any advantage if instead of that I sent the hash of the password to the server? Would it be more secure?
This is an old question, but I felt the need to provide my opinion on this important matter. There is so much misinformation here
The OP never mentioned sending the password in clear over HTTP - only HTTPS, yet many seem to be responding to the question of sending a password over HTTP for some reason. That said:
I believe passwords should never be retained (let alone transmitted) in plain text. That means not kept on disk, or even in memory.
People responding here seem to think HTTPS is a silver bullet, which it is not. It certainly helps greatly however, and should be used in any authenticated session.
There is really no need to know what an original password is. All that is required is a reliable way to generate (and reliably re-generate) an authentication "key" based on the original text chosen by the user. In an ideal world this text should immediately generate a "key" by salting then irreversibly hashing it using an intentionally slow hash-algorithm (like bcrypt, to prevent Brute-force). Said salt should be unique to the user credential being generated.
This "key" will be what your systems use as a password. This way if your systems ever get compromised in the future, these credentials will only ever be useful against your own organisation, and nowhere else where the user has been lazy and used the same password.
So we have a key. Now we need to clean up any trace of the password on the clients device.
Next we need to get that key to your systems. You should never transmit a key or password "in the clear". Not even over HTTPS. HTTPS is not impenetrable. In fact, many organisations can become a trusted MITM - not from an attack perspective, but to perform inspections on the traffic to implement their own security policies. This weakens HTTPS, and it is not the only way it happens (such as redirects to HTTP MITM attacks for example). Never assume it is secure.
To get around this, we encrypt the key with a once off nonce.
This nonce is unique for every submission of a key to your systems - even for the same credential during the same session if you need to send it multiple times. You can reverse said nonce (decrypt), once it arrives in your own systems to recover the authentication key, and authenticate the request.
At this point I would irreversibly hash it one last time before it is permanently stored in your own systems. That way you can share the credential's salt with partner organisations for the purposes of SSO and the like, whilst being able to prove your own organisation cannot impersonate the user. The best part of this approach is you are never sharing anything generated by the user without their authorisation.
Do more research, as there is more to it than even I have divulged, but if you want to provide true security to your users, I think this method is currently the most complete response here.
TL;DR:
Use HTTPS.
Securely hash passwords, irreversibly, with a unique salt per password. Do this on the client - do not transmit their actual password. Transmitting the users original password to your servers is never "OK" or "Fine". Clean up any trace of the original password.
Use a nonce regardless of HTTP/HTTPS. It is much more secure on many levels. (Answer to OP).
Since it's over HTTPS, it's definitely just fine to send the password without hashing (over HTTPS it's not plaintext). Furthermore, if your application is depending on HTTPS to keep it's content secure, then it's useless to hash the password before sending it over HTTPS (i.e. if an attacker can unencrypt the data on the wire, you're screwed anyways)
No, in fact this would be a vulnerability. If the attacker is able to obtain the hash from the database, then he could use it to authenticate without needing to crack it. Under no circumstance should a user or an attacker be able to obtain a hashes password.
The whole point of hashing passwords is to add an extra layer of security. If an attacker is able to obtain the hash and salt from the database using SQL Injection or an insecure backup then he has to find the plain text by brute forcing it. John The Ripper is commonly used to break salted password hashes.
Not using https is a violation of the OWASP Top 10: A9-Insufficient Transport Layer Protection
EDIT:
If in your implementation you calculate a sha256(client_salt+plain_text_password) and then calculate another hash on the server side sha256(server_salt+client_hash) then this is not a serious vulnerability. However, it is still susceptible to eavesdropping and replaying the request. Thus this is still a clear violation of WASP A9. However, this is still utilizing a message digest as a security layer.
The closest thing i have seen to a client-side replacement for https is a diffie-hellman in key exchange in javascript. However, this does prevent active MITM attacks and thus is till technicality a violation of OWASP A9. The Authors of the code agree that this is not a complete replacement for HTTPS, however it is better than nothing and better than a client-side hashing system.
Sending a hash over the wire completely defeats the purpose of the hash, because an attacker can simply send the hash and forget about the password. In a nutshell, a system that athenticates using a hash in clear text is wide open and can be compromise with nothing more than network sniffing.
The password in plaintext show never (not even when using HTTPS) leave the client. It should be irreversibly hashed before leaving the client as there is no need for the server to know the actual password.
Hashing then transmitting solves security issues for lazy users that use the same password in multiple locations (I know I do). However this does not protect your application as a hacker that gained access to the database (or in any other way was able to get his hands on the hash) as the hacker could just transmit the hash and have the server accept it.
To solve this issue you could of course just hash the hash the server receives and call it a day.
My approach to the issue in a socket-based web application I'm creating is that on connection to the client the server generates a salt (random string to be added before hashing) and stores it on the sockets variable, then it transmits this hash to the client. The client takes the users password, hashes it, adds the salt from the server and hashes the whole thing, before transmitting it to the server. Then it's sent to the server which compares this hash to the hash(hash in the DB + salt). As far as I know this is a good approach, but to be fair I haven't read a lot on the topic and if I'm wrong about anything I'd love to be corrected :)
Disclaimer: I'm by no stretch a security expert-- and I'm posting with the hope that others will critique my position as overly cautious or improvable and I will learn from it. With that said, I just want to emphasize that hashing when it leaves your client doesn't mean you get to don't have to hash on the backend before putting it in the database.
Do both
Do both because:
Hashing on the ride over helps cover vulnerabilities of transport, if SSL connection is compromised, they still can't see the raw password. It won't matter in terms of being able to impersonate authorized users, but it will protect your users from having their passwords read in association w/ their email. Most people don't follow best practice and use the same password for many their accounts, so this can be a serious vulnerability to your visitors.
If someone, somehow was able to read passwords from the database (this does happen, think SQL injection), they still won't be able to execute privileged actions impersonating users through my API. This is because of hash asymmetry; even if they know the hash stored in your DB, they won't know the original key used to create it and that's what your auth middleware uses to authenticate. This is also why you should always salt your hash storage.
Granted, they could do a lot of other damage if they had free rein to read what they want from your database.
I just want to emphasize here that if you do decide to hash the key before departure from your clients, that isn't enough-- the backend hashing is, imo, much more important and this is why: If someone is intercepting traffic from your client, then they will see the contents of the password field. Whether this is a hash, or plain text, it doesn't matter-- they can copy it verbatim to impersonate an authorized client. (Unless you follow the steps which #user3299591 outlines, and I recommend you do). Hashing the DB column, on the other hand, is a necessity and not at all difficult to implement.
Use HTTP Digest - it secures the password even over http (but best useage would be http digest over https)
Wikipedia:
HTTP digest access authentication is one of the agreed methods a web server can use to negotiate credentials with a web user (using the HTTP protocol). Digest authentication is intended to supersede unencrypted use of the Basic access authentication, allowing user identity to be established securely without having to send a password in plaintext over the network. Digest authentication is basically an application of MD5 cryptographic hashing with usage of nonce values to prevent cryptanalysis.
Link: http://en.wikipedia.org/wiki/Digest_access_authentication
If you want to see a "real life" use, you could look at phpMyID - a php openid provider that uses http digest authentication http://siege.org/phpmyid.php
.. or you could start from the php auth samples at http://php.net/manual/en/features.http-auth.php
Http digest rfc: http://www.faqs.org/rfcs/rfc2617
From my tests all modern browsers support it...
If you're looking to replace a clear-text password over HTTPS with a hashed password over HTTP then you're asking for trouble. HTTPS generates a random, shared transaction key when opening up a communication channel. That's hard to crack, as you're pretty much limited to brute forcing the shared key used for a (relatively) short-term transaction. Whereas your hash can be just sniffed, taken off-line and looked up in a rainbow table or just brute forced over a long amount of time.
However, a basic client-side password obfuscation (not hash) sent over HTTPS does have some value. If I'm not mistaken this technique is actually used by some banks. The purpose of this technique is not to protect the password from sniffing over the wire. Rather, it's to stop the password from being usable to dumb spying tools and browser plug-ins that just grab every HTTPS GET/POST request that they see. I've seen a log file captured from a malicious website that was 400MB of random GET/POST transactions captured from user sessions. You can imagine that websites that used just HTTPS would show up with clear-text passwords in the log, but websites with very basic obfuscation (ROT13) as well would show up with passwords that are not immediately of use.
Whether there's an advantage, and whether it's more (or less) secure really depends on implementation. There's arguably some advantage, but if you implement it poorly, you could definitely create a solution that is less secure than passing even a plaintext password.
This can be looked at from the perspective of two types of attacks-- one with access to the network traffic, and another with access to the database.
If your attacker can intercept the plaintext version of the network traffic, then seeing a hash of the password is more secure than seeing the password in plaintext. Although the attacker could still log in to your server using that hash, it would require a brute-force crack (sometimes pre-computed) of that hash to determine the password that might be useful on other systems. People should use different passwords on different systems, but often don't.
If an attacker gained access to the database, perhaps through a copy of a backup, then you'd want to ensure that one couldn't log in with only that knowledge. If, for example, you stored a hash salted with the login name as hash(login_name+password), and you passed that same hash from the client for direct comparison, then the attacker could pick a user at random, send the hash read from the database and log in as that user without knowing the password, increasing the scope of the breach. In that case, sending the password in plaintext would have been more secure because the attacker would need to know the plaintext in order to log in, even having a copy of the database. This is where implementation is key. Whether you send a plaintext password or a client-side hash of that password, you should hash that value at the server-side and compare that hash with the hash stored in the user record.
Concepts to keep in mind:
You "salt" a hash by mixing in some scope-unique value to your hash, typically row-unique. Its purpose is to guarantee uniqueness of hashes from each other even if the plaintext values they represent are the same, so two users with the same password would still have different hashes. It's unnecessary to treat a salt as a secret.
When authenticating, always hash on the server-side whatever value you pass from the client as a password (even if it's already hashed) and compare it with a pre-hashed value stored on the database. This may necessitate storing a double-hashed version of the original password.
When you make a hash, consider adding a server/cluster-unique salt to the hash as well as a row-unique salt to safeguard against matching any pre-computed hashes in lookup tables.
If you're connected to an https server the data stream between the server and browser should be encrypted. The data is only plain text before being sent and after being recieved. Wikipedia article
If you want to achieve the same reliability as when transferring over https, then yes - one option - the implementation of an asymmetrically encrypted channel at the ajax level.
If we are not talking about registration (for example, the first transmission of the password is always protected), then there are options.
For example,
The server generates a random string and generates a salt sends it to the user.
The user calculates a hash from his password and using this hash as
a key encrypts this random string with a blowfish, for example (there
is an implementation in JS for sure) and sends it back to you.
You, on your own, using the hash stored on the server, also encrypt this
random string with a blowfish.
Compare.
An attacker would have to attack the bluefish key using a random source and ciphertext. The task is not easy.
Isn't SSL/TLS replacing the nonce? I don't see added value of this since SSL/TLS also protects against Replay Attacks.
Ref.
https://en.wikipedia.org/wiki/Cryptographic_nonce
It would actually be less secure to hash the password and send it over a non-encrypted channel. You will expose your hashing algorithm on the client. Hackers could just sniff the hash of the password and then use it to hack in later.
By using HTTPS, you prevent a hacker from obtaining the password from a single source, since HTTPS uses two channels, both encrypted.