I'm switching a site over to rails. It's quite a large site with 50k+ users. The problem is, the existing password hashing method is extremely weak. I have two options:
1) Switch to a new algorithm, generate random passwords for everyone and then email them those passwords and require the change immediately after
2) Implement new algorithm but use the the old one before and then hash the result. For example:
Password: abcdef =Algorithm 1=> xj31ndn =Algorithm 2=> $21aafadsada214
Any new passwords would need to go through the original algorithm (md5) and then have the result of that hashed if that makes any sense? Is there any disadvantage to this?
Normally it's not necessary to reset the passwords, one can just wait until the user logs in the next time.
First try to verify the entered password with the new algorithm. New passwords and already converted passwords will not take longer for verification then.
If it does not match, compare it with the old hash algorithm.
Should the old hash value match, then you can calculate and store the new hash, since you know the password then.
Every password-storing-system must have the option to switch to a better hash algorithm, your problem is not a one-time migration problem. Good password hash algorithms like BCrypt have a cost factor, from time to time you have to increase this cost factor (because of faster hardware), then you need the exact same procedure as you need for the migration.
Your option 2 with hashing the old hash is a good thing, if your first algorithm is really weak, and you want to give more protection immediately. In this case you can calculate a double-hash and replace the old hash in the database with the new double-hash.
$newHashToStoreInTheDb = new_hash($oldHashFromDb)
You should also mark this password-hash (see why), so you can recognize it as double-hash. This can be done in a separate database field, or you can include your own signature. Modern password hash functions also include a signature of the algorithm, so that they can upgrade to newer algorithms, and still can verify older hashes. The example shows the signature of a BCrypt hash:
$2y$10$nOUIs5kJ7naTuTFkBy1veuK0kSxUFXfuaOKdOKf9xYT0KKIGSJwFa
___
|
signature of hash-algorithm = 2y = BCrypt
The verification would run like this:
Decide whether it is a double-hash.
If it is a new hash, call the new hash-function to verify the entered password, and you are done.
If it is a double-hash, compare it with the double-hash algorithm new_hash(old_hash($password)).
Should the double-hash value match, then you can calculate and store the new hash.
The simplest solution is probably to add a "password hash type" column to the database. Set it initially to "old"; when a user logs in, re-hash the password using the new algorithm and set the database type to "new".
A variant of this method is to store the hash type as part of the hash string. This works just as well, as long as you can unambiguously tell the different hash formats apart, and has the advantage that you can also include any other needed parameters (such as the salt and the work factor for key stretching) in the same string without having to add extra fields for each to your database.
For example, this is the approach typically used by modern Unix crypt(3) implementations (and the corresponding functions in various high-level languages like PHP): a classic DES-based (and horribly weak) password hash would look something like abJnggxhB/yWI, while a (slightly) more modern hash might look like $1$z75qouSC$nNVPAk1FTd0yVd62S3sjR1, where 1 specified the hashing method, z75qouSC is the salt and nNVPAk1FTd0yVd62S3sjR1 the actual hash, and the delimiter $ is chosen because it cannot appear in an old-style DES hash.
The method you suggest, where the new hashes are calculated as:
hash = new_hash( old_hash( password ) )
can be useful in some cases, since it allows all existing records to be updated without having to wait for users to log in. However, it's only safe if the old hash function preserves enough of the entropy in the passwords.
For example, even a fairly old and weak cryptographic hash function, like unsalted MD5, would be good enough, since its output depends on the entire input and has up to 128 bits of entropy, which is more than almost any password will have (and more than enough to withstand a brute force attack, anyway). On the other hand, trying to apply this construction using the old DES-based crypt(3) function as the old hash would be disastrous, since old crypt(3) would ignore all but the first 8 characters of each password (as well as the most significant bits of even those characters).
You can create a new password field with all users that has updated their password with the new password method, and just update everybody with your option 2.
Combined this with forcing password update on login for all users with the old password method will automatically move all active users to the new password method.
An alternative could be to keep both hashes available for the migration phase in separate columns of the database:
If the new hash does not exist during login, check with the old hash and save the new hash and delete the old hash.
If the new hash exists, use only this to verify.
Thus, after some time you will be left with the new hashes only - at least for those users who logged in at least one time.
Suppose you were at liberty to decide how hashed passwords were to be stored in a DBMS. Are there obvious weaknesses in a scheme like this one?
To create the hash value stored in the DBMS, take:
A value that is unique to the DBMS server instance as part of the salt,
And the username as a second part of the salt,
And create the concatenation of the salt with the actual password,
And hash the whole string using the SHA-256 algorithm,
And store the result in the DBMS.
This would mean that anyone wanting to come up with a collision should have to do the work separately for each user name and each DBMS server instance separately. I'd plan to keep the actual hash mechanism somewhat flexible to allow for the use of the new NIST standard hash algorithm (SHA-3) that is still being worked on.
The 'value that is unique to the DBMS server instance' need not be secret - though it wouldn't be divulged casually. The intention is to ensure that if someone uses the same password in different DBMS server instances, the recorded hashes would be different. Likewise, the user name would not be secret - just the password proper.
Would there be any advantage to having the password first and the user name and 'unique value' second, or any other permutation of the three sources of data? Or what about interleaving the strings?
Do I need to add (and record) a random salt value (per password) as well as the information above? (Advantage: the user can re-use a password and still, probably, get a different hash recorded in the database. Disadvantage: the salt has to be recorded. I suspect the advantage considerably outweighs the disadvantage.)
There are quite a lot of related SO questions - this list is unlikely to be comprehensive:
Encrypting/Hashing plain text passwords in database
Secure hash and salt for PHP passwords
The necessity of hiding the salt for a hash
Clients-side MD5 hash with time salt
Simple password encryption
Salt generation and Open Source software
Password hashes: fixed-length binary fields or single string field?
I think that the answers to these questions support my algorithm (though if you simply use a random salt, then the 'unique value per server' and username components are less important).
The salt just needs to be random and unique. It can be freely known as it doesn't help an attacker. Many systems will store the plain text salt in the database in the column right next to the hashed password.
The salt helps to ensure that if two people (User A and User B) happen to share the same password it isn't obvious. Without the random and unique salt for each password the hash values would be the same and obviously if the password for User A is cracked then User B must have the same password.
It also helps protect from attacks where a dictionary of hashes can be matched against known passwords. e.g. rainbow tables.
Also using an algorithm with a "work factor" built in also means that as computational power increases the work an algorithm has to go through to create the hash can also be increased. For example, bcrypt. This means that the economics of brute force attacks become untenable. Presumably it becomes much more difficult to create tables of known hashes because they take longer to create; the variations in "work factor" will mean more tables would have to be built.
I think you are over-complicating the problem.
Start with the problem:
Are you trying to protect weak passwords?
Are you trying to mitigate against rainbow attacks?
The mechanism you propose does protect against a simple rainbow attack, cause even if user A and user B have the SAME password, the hashed password will be different. It does, seem like a rather elaborate method to be salting a password which is overly complicated.
What happens when you migrate the DB to another server?
Can you change the unique, per DB value, if so then a global rainbow table can be generated, if not then you can not restore your DB.
Instead I would just add the extra column and store a proper random salt. This would protect against any kind of rainbow attack. Across multiple deployments.
However, it will not protect you against a brute force attack. So if you are trying to protect users that have crappy passwords, you will need to look elsewhere. For example if your users have 4 letter passwords, it could probably be cracked in seconds even with a salt and the newest hash algorithm.
I think you need to ask yourself "What are you hoping to gain by making this more complicated than just generating a random salt value and storing it?" The more complicated you make your algorithm, the more likely you are to introduce a weakness inadvertently. This will probably sound snarky no matter how I say it, but it's meant helpfully - what is so special about your app that it needs a fancy new password hashing algorithm?
Why not add a random salt to the password and hash that combination. Next concatenate the hash and salt to a single byte[] and store that in the db?
The advantage of a random salt is that the user is free to change it's username. The Salt doesn't have to be secret, since it's used to prevent dictionary attacks.
I need to store hashes of passwords on disk. I am not entirely sure which hash function to use (they all seem somewhat troubled at the moment), but I am leaning towards SHA-256.
My plan is to take the user's password and combine it with their user ID, a random user-specific salt, and a universal site-wide salt. Should I concatenate these values together and then hash the single resulting string, or should I hash each separately, concatenate the hashes, and then hash that? Also, does the order (password, user id, user salt, site salt) matter? Can I rearrange them however I like, or is it a bad idea to have something that doesn't change (site salt) or something completely predictable (user id/user salt) first?
Thanks.
SHA-256 seems to be one of the better options available right now.
Concatenating everything should be fine and order isn't all that important. Just make sure that you are using a significantly long salt value.
This post has some good recommendations-
What algorithm should I use to hash passwords into my database?
Why not bcrypt? Password hashing should be very slow, but SHA* is designed to be very fast. bcrypt is specifically designed for password hashing.
Previous SO questions about this:
Password handling best practices?
What algorithm should I use to hash passwords into my database?
But to provide brief answers to your specific questions:
SHA-256 is a viable option.
You can hash the single string.
Order doesn't matter.
You don't need two salts. Just a user-specific salt is fine, the site-wide one is unnecessary and doesn't actually contribute anything.
Never hash hashes!!
An article that I stumbled upon here in SO provided links to other articles which in turn provided links to even more articles etc.
And in the end I was left completely stumped - so what is the best way to store passwords in the DB? From what I can put together you should:
Use a long (at least 128 fully random bits) salt, which is stored in plaintext next to the password;
Use several iterations of SHA-256 (or even greater SHA level) on the salted password.
But... the more I read about cryptography the more I understand that I don't really understand anything, and that things I had thought to be true for years are actually are flat out wrong. Are there any experts on the subject here?
Added: Seems that some people are missing the point. I repeat the last link given above. That should clarify my concerns.
https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2007/july/enough-with-the-rainbow-tables-what-you-need-to-know-about-secure-password-schemes/
You got it right. Only two suggestions:
If one day SHA1 becomes too weak and you want to use something else, it is impossible to unhash the old passwords and rehash them with the new scheme. For this reason, I suggest that attached to each password a "version" number that tells you what scheme you used (salt length, which hash, how many times). If one day you need to switch from SHA to something stronger, you can create new-style passwords while still having old-style passwords in the database and still tell them apart. Migrating users to the new scheme will be easier.
Passwords still go from user to system without encryption. Look at SRP if that's a problem. SRP is so new that you should be a little paranoid about implementing it, but so far it looks promising.
Edit: Turns out bcrypt beat me to it on idea number 1. The stored info is (cost, salt, hash), where cost is how many times the hashing has been done. Looks like bcrypt did something right. Increasing the number of times that you hash can be done without user intervention.
In truth it depends on what the passwords are for. You should take storing any password with care, but sometimes much greater care is needed than others. As a general rule all passwords should be hashed and each password should have a unique salt.
Really, salts don't need to be that complex, even small ones can cause a real nightmare for crackers trying to gain entry into the system. They are added to a password to prevent the use of Rainbow tables to hack multiple account's passwords. I wouldn't add a single letter of the alphabet to a password and call it a salt, but you don't need to make it a unique guid which is encrypted somewhere else in the database either.
One other thing concerning salts. The key to making a password + salt work when hashing is the complexity of the combination of the two. If you have a 12 character password and add a 1 character salt to it, the salt doesn't do much, but cracking the password is still a monumental feat. The reverse is also true.
Use:
Hashed password storage
A 128+ bit user-level salt, random, regenerated (i.e. you make new salts when you make new password hashes, you don't persistently keep the same salt for a given user)
A strong, computationally expensive hashing method
Methodology that is somewhat different (hash algorithm, how many hashing iterations you use, what order the salts are concatenated in, something) from both any 'standard implementation guides' like these and from any other password storage implementation you've written
I think there no extra iteration on the password needed, juste make sure there is a salt, and a complexe one ;)
I personnaly use SHA-1 combined with 2 salt keyphrases.
The length of the salt doesnt really matter, as long as it is unique to a user. The reason for a salt is so that a given generated attempt at a hash match is only useful for a single row of your users table in the DB.
Simply said, use a cryptographically secure hash algorithm and some salt for the passwords, that should be good enough for 99.99% of all use cases. The weak link will be the code that checks the password as well as the password input.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed last year.
The community reviewed whether to reopen this question last year and left it closed:
Original close reason(s) were not resolved
Improve this question
I've inherited a web app that I've just discovered stores over 300,000 usernames/passwords in plain text in a SQL Server database. I realize that this is a Very Bad Thing™.
Knowing that I'll have to update the login and password update processes to encrypt/decrypt, and with the smallest impact on the rest of the system, what would you recommend as the best way to remove the plain text passwords from the database?
Any help is appreciated.
Edit: Sorry if I was unclear, I meant to ask what would be your procedure to encrypt/hash the passwords, not specific encryption/hashing methods.
Should I just:
Make a backup of the DB
Update login/update password code
After hours, go through all records in the users table hashing the password and replacing each one
Test to ensure users can still login/update passwords
I guess my concern is more from the sheer number of users so I want to make sure I'm doing this correctly.
EDIT (2016): use Argon2, scrypt, bcrypt, or PBKDF2, in that order of preference. Use as large a slowdown factor as is feasible for your situation. Use a vetted existing implementation. Make sure you use a proper salt (although the libraries you're using should be making sure of this for you).
When you hash the passwords use DO NOT USE PLAIN MD5.
Use PBKDF2, which basically means using a random salt to prevent rainbow table attacks, and iterating (re-hashing) enough times to slow the hashing down - not so much that your application takes too long, but enough that an attacker brute-forcing a large number of different password will notice
From the document:
Iterate at least 1000 times, preferably more - time your implementation to see how many iterations are feasible for you.
8 bytes (64 bits) of salt are sufficient, and the random doesn't need to be secure (the salt is unencrypted, we're not worried someone will guess it).
A good way to apply the salt when hashing is to use HMAC with your favorite hash algorithm, using the password as the HMAC key and the salt as the text to hash (see this section of the document).
Example implementation in Python, using SHA-256 as the secure hash:
EDIT: as mentioned by Eli Collins this is not a PBKDF2 implementation. You should prefer implementations which stick to the standard, such as PassLib.
from hashlib import sha256
from hmac import HMAC
import random
def random_bytes(num_bytes):
return "".join(chr(random.randrange(256)) for i in xrange(num_bytes))
def pbkdf_sha256(password, salt, iterations):
result = password
for i in xrange(iterations):
result = HMAC(result, salt, sha256).digest() # use HMAC to apply the salt
return result
NUM_ITERATIONS = 5000
def hash_password(plain_password):
salt = random_bytes(8) # 64 bits
hashed_password = pbkdf_sha256(plain_password, salt, NUM_ITERATIONS)
# return the salt and hashed password, encoded in base64 and split with ","
return salt.encode("base64").strip() + "," + hashed_password.encode("base64").strip()
def check_password(saved_password_entry, plain_password):
salt, hashed_password = saved_password_entry.split(",")
salt = salt.decode("base64")
hashed_password = hashed_password.decode("base64")
return hashed_password == pbkdf_sha256(plain_password, salt, NUM_ITERATIONS)
password_entry = hash_password("mysecret")
print password_entry # will print, for example: 8Y1ZO8Y1pi4=,r7Acg5iRiZ/x4QwFLhPMjASESxesoIcdJRSDkqWYfaA=
check_password(password_entry, "mysecret") # returns True
The basic strategy is to use a key derivation function to "hash" the password with some salt. The salt and the hash result are stored in the database. When a user inputs a password, the salt and their input are hashed in the same way and compared to the stored value. If they match, the user is authenticated.
The devil is in the details. First, a lot depends on the hash algorithm that is chosen. A key derivation algorithm like PBKDF2, based on a hash-based message authentication code, makes it "computationally infeasible" to find an input (in this case, a password) that will produce a given output (what an attacker has found in the database).
A pre-computed dictionary attack uses pre-computed index, or dictionary, from hash outputs to passwords. Hashing is slow (or it's supposed to be, anyway), so the attacker hashes all of the likely passwords once, and stores the result indexed in such a way that given a hash, he can lookup a corresponding password. This is a classic tradeoff of space for time. Since password lists can be huge, there are ways to tune the tradeoff (like rainbow tables), so that an attacker can give up a little speed to save a lot of space.
Pre-computation attacks are thwarted by using "cryptographic salt". This is some data that is hashed with the password. It doesn't need to be a secret, it just needs to be unpredictable for a given password. For each value of salt, an attacker would need a new dictionary. If you use one byte of salt, an attacker needs 256 copies of their dictionary, each generated with a different salt. First, he'd use the salt to lookup the correct dictionary, then he'd use the hash output to look up a usable password. But what if you add 4 bytes? Now he needs 4 billion copies of the the dictionary. By using a large enough salt, a dictionary attack is precluded. In practice, 8 to 16 bytes of data from a cryptographic quality random number generator makes a good salt.
With pre-computation off the table, an attacker has compute the hash on each attempt. How long it takes to find a password now depends entirely on how long it takes to hash a candidate. This time is increased by iteration of the hash function. The number iterations is generally a parameter of the key derivation function; today, a lot of mobile devices use 10,000 to 20,000 iterations, while a server might use 100,000 or more. (The bcrypt algorithm uses the term "cost factor", which is a logarithmic measure of the time required.)
I would imagine you will have to add a column to the database for the encrypted password then run a batch job over all records which gets the current password, encrypts it (as others have mentiond a hash like md5 is pretty standard edit: but should not be used on its own - see other answers for good discussions), stores it in the new column and checks it all happened smoothly.
Then you will need to update your front-end to hash the user-entered password at login time and verify that vs the stored hash, rather than checking plaintext-vs-plaintext.
It would seem prudent to me to leave both columns in place for a little while to ensure that nothing hinky has gone on, before eventually removing the plaintext passwords all-together.
Don't forget also that anytime the password is acessed the code will have to change, such as password change / reminder requests. You will of course lose the ability to email out forgotten passwords, but this is no bad thing. You will have to use a password reset system instead.
Edit:
One final point, you might want to consider avoiding the error I made on my first attempt at a test-bed secure login website:
When processing the user password, consider where the hashing takes place. In my case the hash was calculated by the PHP code running on the webserver, but the password was transmitted to the page from the user's machine in plaintext! This was ok(ish) in the environment I was working in, as it was inside an https system anyway (uni network). But, in the real world I imagine you would want to hash the password before it leaves the user system, using javascript etc. and then transmit the hash to your site.
Follow Xan's advice of keeping the current password column around for a while so if things go bad, you can rollback quick-n-easy.
As far as encrypting your passwords:
use a salt
use a hash algorithm that's meant for passwords (ie., - it's slow)
See Thomas Ptacek's Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes for some details.
I think you should do the following:
Create a new column called HASHED_PASSWORD or something similar.
Modify your code so that it checks for both columns.
Gradually migrate passwords from the non-hashed table to the hashed one. For example, when a user logs in, migrate his or her password automatically to the hashed column and remove the unhashed version. All newly registered users will have hashed passwords.
After hours, you can run a script which migrates n users a time
When you have no more unhashed passwords left, you can remove your old password column (you may not be able to do so, depends on the database you are using). Also, you can remove the code to handle the old passwords.
You're done!
As the others mentioned, you don't want to decrypt if you can help it. Standard best practice is to encrypt using a one-way hash, and then when the user logs in hash their password to compare it.
Otherwise you'll have to use a strong encryption to encrypt and then decrypt. I'd only recommend this if the political reasons are strong (for example, your users are used to being able to call the help desk to retrieve their password, and you have strong pressure from the top not to change that). In that case, I'd start with encryption and then start building a business case to move to hashing.
For authentication purposes you should avoid storing the passwords using reversible encryption, i.e. you should only store the password hash and check the hash of the user-supplied password against the hash you have stored. However, that approach has a drawback: it's vulnerable to rainbow table attacks, should an attacker get hold of your password store database.
What you should do is store the hashes of a pre-chosen (and secret) salt value + the password. I.e., concatenate the salt and the password, hash the result, and store this hash. When authenticating, do the same - concatenate your salt value and the user-supplied password, hash, then check for equality. This makes rainbow table attacks unfeasible.
Of course, if the user send passwords across the network (for example, if you're working on a web or client-server application), then you should not send the password in clear text across, so instead of storing hash(salt + password) you should store and check against hash(salt + hash(password)), and have your client pre-hash the user-supplied password and send that one across the network. This protects your user's password as well, should the user (as many do) re-use the same password for multiple purposes.
Encrypt using something like MD5, encode it as a hex string
You need a salt; in your case, the username can be used as the salt (it has to be unique, the username should be the most unique value available ;-)
use the old password field to store the MD5, but tag the MD5 (i.e.g "MD5:687A878....") so that old (plain text) and new (MD5) passwords can co-exist
change the login procedure to verify against the MD5 if there is an MD5, and against the plain password otherwise
change the "change password" and "new user" functions to create MD5'ed passwords only
now you can run the conversion batch job, which might take as long as needed
after the conversion has been run, remove the legacy-support
Step 1: Add encrypted field to database
Step 2: Change code so that when password is changed, it updates both fields but logging in still uses old field.
Step 3: Run script to populate all the new fields.
Step 4: Change code so that logging in uses new field and changing passwords stops updating old field.
Step 5: Remove unencrypted passwords from database.
This should allow you to accomplish the changeover without interruption to the end user.
Also:
Something I would do is name the new database field something that is completely unrelated to password like "LastSessionID" or something similarly boring. Then instead of removing the password field, just populate with hashes of random data. Then, if your database ever gets compromised, they can spend all the time they want trying to decrypt the "password" field.
This may not actually accomplish anything, but it's fun thinking about someone sitting there trying to figure out worthless information
As with all security decisions, there are tradeoffs. If you hash the password, which is probably your easiest move, you can't offer a password retrieval function that returns the original password, nor can your staff look up a person's password in order to access their account.
You can use symmetric encryption, which has its own security drawbacks. (If your server is compromised, the symmetric encryption key may be compromised also).
You can use public-key encryption, and run password retrieval/customer service on a separate machine which stores the private key in isolation from the web application. This is the most secure, but requires a two-machine architecture, and probably a firewall in between.
MD5 and SHA1 have shown a bit of weakness (two words can result in the same hash) so using SHA256-SHA512 / iterative hashes is recommended to hash the password.
I would write a small program in the language that the application is written in that goes and generates a random salt that is unique for each user and a hash of the password. The reason I tend to use the same language as the verification is that different crypto libraries can do things slightly differently (i.e. padding) so using the same library to generate the hash and verify it eliminates that risk. This application could also then verify the login after the table has been updated if you want as it knows the plain text password still.
Don't use MD5/SHA1
Generate a good random salt (many crypto libraries have a salt generator)
An iterative hash algorithm as orip recommended
Ensure that the passwords are not transmitted in plain text over the wire
I would like to suggest one improvement to the great python example posted by Orip. I would redefine the random_bytes function to be:
def random_bytes(num_bytes):
return os.urandom(num_bytes)
Of course, you would have to import the os module. The os.urandom function provides a random sequence of bytes that can be safely used in cryptographic applications. See the reference help of this function for further details.
To hash the password you can use the HashBytes function. Returns a varbinary, so you'd have to create a new column and then delete the old varchar one.
Like
ALTER TABLE users ADD COLUMN hashedPassword varbinary(max);
ALTER TABLE users ADD COLUMN salt char(10);
--Generate random salts and update the column, after that
UPDATE users SET hashedPassword = HashBytes('SHA1',salt + '|' + password);
Then you modify the code to validate the password, using a query like
SELECT count(*) from users WHERE hashedPassword =
HashBytes('SHA1',salt + '|' + <password>)
where <password> is the value entered by the user.
I'm not a security expert, but i htink the current recommendation is to use bcrypt/blowfish or a SHA-2 variant, not MD5 / SHA1.
Probably you need to think in terms of a full security audit, too