TLDR;
The hashing function generates a different hash every time for the same piece of data, but it can determine if a particular hash was generated with the piece of data or not.
Eg:
hash_func(xyz): abc123
hash_func(xyz): jhg342 // different hash, even if the data was same.
decode_hash(jhg324) == xyz
This gives true, because the hash function determined that jhg324 is indeed the hash of xyz
The Question
For an Open Source website, I want to store the email in hashed form (because all the users will be public), but the site needs to know if an email was used to register for another account so that it can ensure one account per email.
However, all the emails are from one organization only. This means, they all look exactly like uid#org_name.com. This means anyone can run through all the UIDs and find out which hash belongs to which email, and thus, which person.
Therefore, is there a way to hash the email such that the hash knows which email it belongs to, but hashing the same email does not generate the same hash.
P.S. Please note that I cannot use Salting as the site will be Open Source and the salt will be publicly available.
This doesn't make sense - you're conflating hashing and encryption in a very strange way. What you're describing wouldn't really be a cryptographically secure hash function. By definition, cryptographically secure hash functions are one way. In fact, if you could reverse it, there would be little point to using it at all because it would no longer be secure. This would make it possible to brute-force passwords and would "break" passwords that were used in multiple places.
Also, why would you want it to hash to different values each time? That's what you use a salt for.
If you want to be able to reverse it later, just use an encryption algorithm like AES. Even better, many databases even offer features for securely storing sensitive information; see, for example, SQL Server's Always Encrypted feature.
First, I apologise for the question. I know there are a ton of similar obvious questions that ask for how to implement bcrypt in X app, but here I'm asking between 2 strategies.
I just recently implemented bcrypt into the user login. Basically:
User sends credentials to my rest api server
I bcrypt.hash the given password and compare it to the database with a simple where clause as the following:
WHERE mail = $mail and password = $my_just_hashed_password
If I find a user on the database I assume that the given password is correct. Otherwise, I reject the request.
This is almost the same as first selecting the user from the unique mail given and then bcrypt.compare the given password and the hashed one from the database.
My question is, which one is better? I know that the bcrypt hashing function is intended to be used to safely store sensible data, but why not using it to also validate users?
I will provide an alternative answer to the already approved answer, with emphasis on security...
As the email is unique, I would first find the user based on the email alone, get the password from the retrieved user, and compare these using bcrypt. If the email exists, but the password doesn't match, there may be a brute force attack in place, you may want to log these failed attempts so you can lock accounts or add a captcha perhaps to slow down/prevent the attack.
How is this different
ORIGINAL SOLUTION
Get the user from the email and hashed password
If the credentials are right, great, the user can log in. No problems here
If the user is not found, was the email or password incorrect? You can't know with this one query alone.
The potential attacker is free to continue trying to access a potential account by trying different passwords on the inputted email.
ALTERNATIVE SOLUTION
Get the user from the email alone
If the user exists, compare the hashed user password in the DB with the hash password of the inputted credentials (using bcrypt compare).
If the password is incorrect, log this in another table (USER_FAILED_LOGINS), after 3 failed attempts, add a some security measure such as a captcha.
If the password is correct, great, log the user in.
You could of course, use the original solution and just do another query if the credentials were incorrect to figure out if the email exists and implement the security measures at this point. Just be wary of these types of attacks and prevent against them. Also, make sure you are using prepared statements in your SQL.
When you use bcrypt to hash a password correctly, you append a string of at least sixteen random characters to the password and run the whole thing through bcrypt many thousands of times. It should take at least a quarter second to complete the hash algorithm. Then you store the random. string, known as a salt, along with the hashed password.
To verify a password, you extract the salt from the stored password hash, then hash the password you're verifying along with the salt. The two hashes must match.
Because of the salt, there's no way you'll ever be able to compare one hash to another and expect them to be equal.
Read this, please. https://en.m.wikipedia.org/wiki/Bcrypt#Description
Why is this so complex? Because cybercreeps. Because cybercreeps sometimes can steal the users table for a large web app. It is bad enough that they steal the usernames and emails, but we don't want them to steal the passwords too. See Adobe. See Ashley Madison.
Without the random salts, it's possible for attackers to construct lookup tables to help guess passwords from hashes: RAM is cheap these days. The random salts mean that the lookup tables are no longer feasible. You know some user will choose 654321 as a password, and we don't want attackers to be able to search all the passwords for a hashed version of that and other common passwords.
Also, we use the Blowfish cypher algorithm because it is slower than MD-5 and the various SHA hashes. That's good: we want attackers to be faced with a slow hashing algorithm, so they can't guess many thousands of passwords each second. Brute force password guessing is prohibitively time consuming with bcrypt.
Finally, it's a principle of information security to use carefully validated encryption algorithms. It's unwise to roll your own. Assume that the cybercreeps are smarter and more highly motivated than you, and you will be more secure.
Do not use the method you describe, please.
I have legacy browser game which historicaly uses simple hashing function for password storage. I know that it' far from ideal. However time has proven that most of the cheaters (multiaccounts) use same password for all of fake accounts.
In update of my game I want to store passwords more safely. I already know, that passwords should by randomly salted, hashed by safe algorithms etc. That's all nice.
But is there any way, how to store passwords properly and determine that two (or more) users use same password? I don't want to know the password. I don't want to be able to search by password. I only need to tell, that suspect users A, B and C use same one.
Thanks.
If you store them correctly - no. This is one of the points of a proper password storage.
You could have very long passwords, beyond what is available on rainbow tables (not sure about the current state of the art, but it used to be 10 or 12 characters) and not salt them. In this case two passwords would have the same hash. This is a very bad idea (but a solution nevertheless) - if your passwords leak someone may be able to guess them indirectly (xkcd reference).
You may also look at homomorphic encryption, but this is in the realm of science fiction for now.
Well, if you use salt + hashing, you have all the salts as plain text. When a user enters a password, before storing/verifying it, you can hash it with all the salts available and see if you get the corresponding existing hash. :)
The obvious problem with this is that if you are doing it properly with bcrypt or pbkdf2 for hashing, this would be very slow - that's kind of the point in these functions.
I don't think there is any other way you can tell whether two passwords are the same - you need at least one of them plain text, which is only when the user enters it. And then you want to remove it from memory asap, which contradicts doing all these calculations with the plain text password in memory.
This will reduce the security of all passwords somewhat, since it leaks information about when two users have the same password. Even so, it is a workable trade-off and is straightforward to secure within that restriction.
The short answer is: use the same salt for all the passwords, but make that salt unique to your site.
Now the long answer:
First, to describe a standard and appropriate way to handle passwords. I'll get to the differences for you afterwards. (You may know all of this already, but it's worth restating.)
Start with a decent key-stretching algorithm, such as PBKDF2 (there are others, some even better, but PBKDF2 is ubiquitous and sufficient for most uses). Select a number of iterations depending on what is client-side environment is involved. For JavaScript, you'll want something like 1k-4k iterations. For languages with faster math, you can use 10k-100k.
The key stretcher will need a salt. I'll talk about the salt in a moment.
The client sends the password to the server. The server applies a fast hash (SHA-256 is nice) and compares that to the stored hash. (For setting the password, the server does the same thing; it accepts a PBKDF2 hash, applies SHA-256, and then stores it.)
All that is standard stuff. The question is the salt. The best salt is random, but no good for this. The second-best salt is built from service_id+user_id (i.e. use a unique identifier for the service and concatenate the username). Both of these make sure that every user's password hash is unique, even if their passwords are identical. But you don't want that.
So now finally to the core of your question. You want to use a per-service, but not per-user, static salt. So something like "com.example.mygreatapp" (obviously don't use that actual string; use a string based on your app). With a constant salt, all passwords on your service that are the same will stretch (PBKDF2) and hash (SHA256) to the same value and you can compare them without having any idea what the actual password is. But if your password database is stolen, attackers cannot compare the hashes in it to hashes in other sites' databases, even if they use the same algorithm (because they'll have a different salt).
The disadvantage of this scheme is exactly its goal: if two people on your site have the same password and an attacker steals your database and knows the password of one user, they know the password of the other user, too. That's the trade-off.
I've been told that you should not store the users password in a database, but how can I authenticate users if I cannot save their password? Is simply encrypting them enough to keep them safe?
There have been several stories in the news lately of high-profile sites that have been compromised, like LinkedIn, and I don't think such a high profile site would store plain-text passwords, so would assume they were encrypted.
Disclaimer: I've originally posted this on Quora but felt that the answer was more suited to Stack Overflow.
The method used to store and check user passwords without actually keeping the passwords is to compare the user input to the stored hash.
What is hashing?
Hashing is the process of passing data of variable length (small passwords, big passwords, binary files, whatever) through an algorithm that returns it as a set of fixed length called a hash value. Hashes only work one way. An *.img file consisting of several Mb can be hashed exactly the same as a password. (actually it's a common practice to use hashes on large files to check for their integrity; say you download a file using bittorrent, when it's complete the software hashes it and compares the hash of what you have with the hash of what you where supposed to have, if they match the download is not corrupt).
How does auth with hashes work?
When the user registers he gives a password, say pass123 that is then hashed (by any of the available hashing algorithms: sha1, sha256, etc, on this case md5) to the value 32250170a0dca92d53ec9624f336ca24 and that value is stored on database. Every time you try to login the system will hash you password in real time and compare it to the stored hash, if it matches, you're good to go. You can try an online md5 hasher here: http://md5-hash-online.waraxe.us/
What if two hashes are the same? Could a user login with a different pass?
He could! That is called a collision. Say that on a fictional hashing algorithm the value pass123 would produce the hash ec9624 and the value pass321 would produce the exact same hash, that hashing algorithm would be broken. Both common algorithms md5 and sha1 (the one LinkedIn used) are broken as collisions have been found. Being broken does not necessarily means it's unsafe.
How can you exploit collisions?
If you can generate a hash, that is the same as the hash generated by the user password you can identify to that site as the user.
Rainbow table-attacks.
Crackers quickly understood that once they had captured a table of hashed-passwords it would not be feasible to exploit passwords one by one so they devised a new attack vector. They would generate every single password in existence (aaa, aab, aac, aad, etc, etc) and store all the hashes in a database. Then they would only need to search for the stolen hash on the database with all the sequentially generated hashes (a sub-second query) and get the according password.
Salt to the rescue (and where LinkedIn failed big!)
Security is defined by the amount of time it will take for a cracker to break your password and the frequency by which you change it. With rainbow tables security drops really fast so the industry came up with salt. What if every password had a unique twist? That's salt! For every user that registers you generate a random string, say 3 characters (the industry recommends 16 chars - https://stackoverflow.com/a/18419...). Then you concatenate the user's password with your random string.
password - salt - sha1 hash
qwerty - 123 - 5cec175b165e3d5e62c9e13ce848ef6feac81bff
qwerty - 321 - b8b92ab870c50ce5fc59571dc0c77f9a4a90323c
qazwsx - abc - c6aec64efe2a25c6bc35aeea2aafb2e86ac96a0c
qazwsx - cba - 31e42c24f71dc5a453b2635e6ec57eadf03090fd
As you can see the exact same passwords, given different values of salt, generate completely different hashes. That is the purpose of salt and why LinkedIn failed big. Notice that on table you will only store the hash and the salt! Never the password!
The first thing the guys that got their hand on the LinkedIn hashes did was to sort of the hashes and see if there were matches (there were because multiple users had the same password - shame on them!) those users were the first to drop. If the pass table was salted... none of that would have happened and they would need an excruciating amount of time (and computer resources) to crack every single password. That would have given LinkedIn plenty of time to enforce a new password policy.
Hope the technical side of the answer gave insight as to how authentication works (or should work).
Really like it, when somebody askes this question, because this somebody want's to do it better. With knowing only a few important points, even well-known sites could have avoided lots of troubles.
Recently i wrote a tutorial about Hashing passwords, it uses a hopefully easy and understandable language. It allows to play around with SQL-injection, explains usage of salt and pepper and points out the need for slow key-derivation functions.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed last year.
The community reviewed whether to reopen this question last year and left it closed:
Original close reason(s) were not resolved
Improve this question
I've inherited a web app that I've just discovered stores over 300,000 usernames/passwords in plain text in a SQL Server database. I realize that this is a Very Bad Thing™.
Knowing that I'll have to update the login and password update processes to encrypt/decrypt, and with the smallest impact on the rest of the system, what would you recommend as the best way to remove the plain text passwords from the database?
Any help is appreciated.
Edit: Sorry if I was unclear, I meant to ask what would be your procedure to encrypt/hash the passwords, not specific encryption/hashing methods.
Should I just:
Make a backup of the DB
Update login/update password code
After hours, go through all records in the users table hashing the password and replacing each one
Test to ensure users can still login/update passwords
I guess my concern is more from the sheer number of users so I want to make sure I'm doing this correctly.
EDIT (2016): use Argon2, scrypt, bcrypt, or PBKDF2, in that order of preference. Use as large a slowdown factor as is feasible for your situation. Use a vetted existing implementation. Make sure you use a proper salt (although the libraries you're using should be making sure of this for you).
When you hash the passwords use DO NOT USE PLAIN MD5.
Use PBKDF2, which basically means using a random salt to prevent rainbow table attacks, and iterating (re-hashing) enough times to slow the hashing down - not so much that your application takes too long, but enough that an attacker brute-forcing a large number of different password will notice
From the document:
Iterate at least 1000 times, preferably more - time your implementation to see how many iterations are feasible for you.
8 bytes (64 bits) of salt are sufficient, and the random doesn't need to be secure (the salt is unencrypted, we're not worried someone will guess it).
A good way to apply the salt when hashing is to use HMAC with your favorite hash algorithm, using the password as the HMAC key and the salt as the text to hash (see this section of the document).
Example implementation in Python, using SHA-256 as the secure hash:
EDIT: as mentioned by Eli Collins this is not a PBKDF2 implementation. You should prefer implementations which stick to the standard, such as PassLib.
from hashlib import sha256
from hmac import HMAC
import random
def random_bytes(num_bytes):
return "".join(chr(random.randrange(256)) for i in xrange(num_bytes))
def pbkdf_sha256(password, salt, iterations):
result = password
for i in xrange(iterations):
result = HMAC(result, salt, sha256).digest() # use HMAC to apply the salt
return result
NUM_ITERATIONS = 5000
def hash_password(plain_password):
salt = random_bytes(8) # 64 bits
hashed_password = pbkdf_sha256(plain_password, salt, NUM_ITERATIONS)
# return the salt and hashed password, encoded in base64 and split with ","
return salt.encode("base64").strip() + "," + hashed_password.encode("base64").strip()
def check_password(saved_password_entry, plain_password):
salt, hashed_password = saved_password_entry.split(",")
salt = salt.decode("base64")
hashed_password = hashed_password.decode("base64")
return hashed_password == pbkdf_sha256(plain_password, salt, NUM_ITERATIONS)
password_entry = hash_password("mysecret")
print password_entry # will print, for example: 8Y1ZO8Y1pi4=,r7Acg5iRiZ/x4QwFLhPMjASESxesoIcdJRSDkqWYfaA=
check_password(password_entry, "mysecret") # returns True
The basic strategy is to use a key derivation function to "hash" the password with some salt. The salt and the hash result are stored in the database. When a user inputs a password, the salt and their input are hashed in the same way and compared to the stored value. If they match, the user is authenticated.
The devil is in the details. First, a lot depends on the hash algorithm that is chosen. A key derivation algorithm like PBKDF2, based on a hash-based message authentication code, makes it "computationally infeasible" to find an input (in this case, a password) that will produce a given output (what an attacker has found in the database).
A pre-computed dictionary attack uses pre-computed index, or dictionary, from hash outputs to passwords. Hashing is slow (or it's supposed to be, anyway), so the attacker hashes all of the likely passwords once, and stores the result indexed in such a way that given a hash, he can lookup a corresponding password. This is a classic tradeoff of space for time. Since password lists can be huge, there are ways to tune the tradeoff (like rainbow tables), so that an attacker can give up a little speed to save a lot of space.
Pre-computation attacks are thwarted by using "cryptographic salt". This is some data that is hashed with the password. It doesn't need to be a secret, it just needs to be unpredictable for a given password. For each value of salt, an attacker would need a new dictionary. If you use one byte of salt, an attacker needs 256 copies of their dictionary, each generated with a different salt. First, he'd use the salt to lookup the correct dictionary, then he'd use the hash output to look up a usable password. But what if you add 4 bytes? Now he needs 4 billion copies of the the dictionary. By using a large enough salt, a dictionary attack is precluded. In practice, 8 to 16 bytes of data from a cryptographic quality random number generator makes a good salt.
With pre-computation off the table, an attacker has compute the hash on each attempt. How long it takes to find a password now depends entirely on how long it takes to hash a candidate. This time is increased by iteration of the hash function. The number iterations is generally a parameter of the key derivation function; today, a lot of mobile devices use 10,000 to 20,000 iterations, while a server might use 100,000 or more. (The bcrypt algorithm uses the term "cost factor", which is a logarithmic measure of the time required.)
I would imagine you will have to add a column to the database for the encrypted password then run a batch job over all records which gets the current password, encrypts it (as others have mentiond a hash like md5 is pretty standard edit: but should not be used on its own - see other answers for good discussions), stores it in the new column and checks it all happened smoothly.
Then you will need to update your front-end to hash the user-entered password at login time and verify that vs the stored hash, rather than checking plaintext-vs-plaintext.
It would seem prudent to me to leave both columns in place for a little while to ensure that nothing hinky has gone on, before eventually removing the plaintext passwords all-together.
Don't forget also that anytime the password is acessed the code will have to change, such as password change / reminder requests. You will of course lose the ability to email out forgotten passwords, but this is no bad thing. You will have to use a password reset system instead.
Edit:
One final point, you might want to consider avoiding the error I made on my first attempt at a test-bed secure login website:
When processing the user password, consider where the hashing takes place. In my case the hash was calculated by the PHP code running on the webserver, but the password was transmitted to the page from the user's machine in plaintext! This was ok(ish) in the environment I was working in, as it was inside an https system anyway (uni network). But, in the real world I imagine you would want to hash the password before it leaves the user system, using javascript etc. and then transmit the hash to your site.
Follow Xan's advice of keeping the current password column around for a while so if things go bad, you can rollback quick-n-easy.
As far as encrypting your passwords:
use a salt
use a hash algorithm that's meant for passwords (ie., - it's slow)
See Thomas Ptacek's Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes for some details.
I think you should do the following:
Create a new column called HASHED_PASSWORD or something similar.
Modify your code so that it checks for both columns.
Gradually migrate passwords from the non-hashed table to the hashed one. For example, when a user logs in, migrate his or her password automatically to the hashed column and remove the unhashed version. All newly registered users will have hashed passwords.
After hours, you can run a script which migrates n users a time
When you have no more unhashed passwords left, you can remove your old password column (you may not be able to do so, depends on the database you are using). Also, you can remove the code to handle the old passwords.
You're done!
As the others mentioned, you don't want to decrypt if you can help it. Standard best practice is to encrypt using a one-way hash, and then when the user logs in hash their password to compare it.
Otherwise you'll have to use a strong encryption to encrypt and then decrypt. I'd only recommend this if the political reasons are strong (for example, your users are used to being able to call the help desk to retrieve their password, and you have strong pressure from the top not to change that). In that case, I'd start with encryption and then start building a business case to move to hashing.
For authentication purposes you should avoid storing the passwords using reversible encryption, i.e. you should only store the password hash and check the hash of the user-supplied password against the hash you have stored. However, that approach has a drawback: it's vulnerable to rainbow table attacks, should an attacker get hold of your password store database.
What you should do is store the hashes of a pre-chosen (and secret) salt value + the password. I.e., concatenate the salt and the password, hash the result, and store this hash. When authenticating, do the same - concatenate your salt value and the user-supplied password, hash, then check for equality. This makes rainbow table attacks unfeasible.
Of course, if the user send passwords across the network (for example, if you're working on a web or client-server application), then you should not send the password in clear text across, so instead of storing hash(salt + password) you should store and check against hash(salt + hash(password)), and have your client pre-hash the user-supplied password and send that one across the network. This protects your user's password as well, should the user (as many do) re-use the same password for multiple purposes.
Encrypt using something like MD5, encode it as a hex string
You need a salt; in your case, the username can be used as the salt (it has to be unique, the username should be the most unique value available ;-)
use the old password field to store the MD5, but tag the MD5 (i.e.g "MD5:687A878....") so that old (plain text) and new (MD5) passwords can co-exist
change the login procedure to verify against the MD5 if there is an MD5, and against the plain password otherwise
change the "change password" and "new user" functions to create MD5'ed passwords only
now you can run the conversion batch job, which might take as long as needed
after the conversion has been run, remove the legacy-support
Step 1: Add encrypted field to database
Step 2: Change code so that when password is changed, it updates both fields but logging in still uses old field.
Step 3: Run script to populate all the new fields.
Step 4: Change code so that logging in uses new field and changing passwords stops updating old field.
Step 5: Remove unencrypted passwords from database.
This should allow you to accomplish the changeover without interruption to the end user.
Also:
Something I would do is name the new database field something that is completely unrelated to password like "LastSessionID" or something similarly boring. Then instead of removing the password field, just populate with hashes of random data. Then, if your database ever gets compromised, they can spend all the time they want trying to decrypt the "password" field.
This may not actually accomplish anything, but it's fun thinking about someone sitting there trying to figure out worthless information
As with all security decisions, there are tradeoffs. If you hash the password, which is probably your easiest move, you can't offer a password retrieval function that returns the original password, nor can your staff look up a person's password in order to access their account.
You can use symmetric encryption, which has its own security drawbacks. (If your server is compromised, the symmetric encryption key may be compromised also).
You can use public-key encryption, and run password retrieval/customer service on a separate machine which stores the private key in isolation from the web application. This is the most secure, but requires a two-machine architecture, and probably a firewall in between.
MD5 and SHA1 have shown a bit of weakness (two words can result in the same hash) so using SHA256-SHA512 / iterative hashes is recommended to hash the password.
I would write a small program in the language that the application is written in that goes and generates a random salt that is unique for each user and a hash of the password. The reason I tend to use the same language as the verification is that different crypto libraries can do things slightly differently (i.e. padding) so using the same library to generate the hash and verify it eliminates that risk. This application could also then verify the login after the table has been updated if you want as it knows the plain text password still.
Don't use MD5/SHA1
Generate a good random salt (many crypto libraries have a salt generator)
An iterative hash algorithm as orip recommended
Ensure that the passwords are not transmitted in plain text over the wire
I would like to suggest one improvement to the great python example posted by Orip. I would redefine the random_bytes function to be:
def random_bytes(num_bytes):
return os.urandom(num_bytes)
Of course, you would have to import the os module. The os.urandom function provides a random sequence of bytes that can be safely used in cryptographic applications. See the reference help of this function for further details.
To hash the password you can use the HashBytes function. Returns a varbinary, so you'd have to create a new column and then delete the old varchar one.
Like
ALTER TABLE users ADD COLUMN hashedPassword varbinary(max);
ALTER TABLE users ADD COLUMN salt char(10);
--Generate random salts and update the column, after that
UPDATE users SET hashedPassword = HashBytes('SHA1',salt + '|' + password);
Then you modify the code to validate the password, using a query like
SELECT count(*) from users WHERE hashedPassword =
HashBytes('SHA1',salt + '|' + <password>)
where <password> is the value entered by the user.
I'm not a security expert, but i htink the current recommendation is to use bcrypt/blowfish or a SHA-2 variant, not MD5 / SHA1.
Probably you need to think in terms of a full security audit, too