Some time ago I joined new project. It was under development for quite a long time.
The thing that surprised me was that all users' passwords are stored in non-encrypted form.
I explained huge security vulnerabilities of this to our management - it looks like they agree with that and want to make project more secure. Team members agree too.
We have about 20K users in the system.
Actually it is quite stressful to make this work - migrate non-encrypted passwords to encrypted form. If something goes wrong it can lead to project's disaster.
How can I lower this stress?
Backup?
Unit-tests(integrational tests)?
Well, be careful with your backup because it will contain unencrypted user passwords :-)
Assuming that the passwords are stored in a database, an easy solution would go something like this:
1) Make a secure backup of the entire table data
2) Create new column (PasswordEncrypted or similar name)
3) Use an UPDATE query to update each row's new column with an MD5 of the unencrypted password while using a 32 byte or larger salt. Pretty much every database system today has an MD5 function so you won't even have to leave your SQL prompt
4) Keep the plaintext column in the interim and update your application/scripts accordingly to work with the salted password.
5) Rename the plaintext old password column to temporarily take it out of play and test your application- if there are any problems then go back to step 4 and fix your mistakes.
6) When everything is working properly drop the plaintext password column
7) Encourage users to pick a new password now that you have some level of security in place to mitigate the effects of any previous attacks which may have been successful.
What sort of project is this? A web application, desktop application?
If you're going down the refactoring road, is there a reason that the passwords need to be stored in something reversible like encryption? In general, it's good practice to hash your passwords with something like SHA then hash the input with the same algorithm and compare the results. You only store the hashed values, not the actual passwords. This gives you the ability to check that someone has entered the correct password without exposing your users to the possibility of your encryption being broken and their passwords exposed.
Specific information about your approach isn't something I can provide (since I don't know how it works), but your best bet is to create an additional column to store the hashed passwords, hash the existing passwords, then keep them up to date with any password changes. Use this new column for all new development, then once the move is complete and tested, delete the column with the plaintext passwords.
Write lots of tests, that test lots of corner cases (upper and lower case, numbers, symbols, Unicode characters, long passwords, etc). When you're developing and testing, create a system to move back to the old system (by providing the old password list, of course, since once the passwords are hashed you won't be able to convert them back directly). Save a copy of the current password list. Convert the passwords over in a test file or test database, and then use the saved copied of the passwords to test that everything worked. Now move the system into production, and make sure it works for your users. If it does not, you have already tested the plan for migrating back to the old system. Once it has been demonstrated to work for a couple of weeks, you can delete the cleartext password list and you're all set.
I would just hash the current passwords, store them in a new database field, and begin using that field while deleting the password field. I would then notify my users that now would be a good time to change passwords as you've implemented more security mechanisms to keep their data safe.
To have a backup, just do SELECT * INTO Backup FROM UserData
You can get extra confidence by running both authentication methods (encrypted and unencrypted) for each login attempt, and if they yield a different outcome, get an alert e-mail sent to you. This change is not visible to your users, so it can run for weeks and even months. Once you see that the old and the new authentication works for a high enough percentage of your users, deactivate the old one.
If possible you can try this: Send out an email to all your users to update their passwords with a time-out period, after which they cannot work if they do not change their passwords. Hash these new passwords and store the hash. This would require some changes on the frontend (i.e. the data it sends back).
Related
I would like to protect my users' username in an online service, as it may be personally identifying (e.g., an email-address), but am wondering if it's even possible...
My first inclination was to hash it (unsalted), but am worried about possible hash collisions. Not so much worried about the probability of a collision in an SHA256 32-bit hash, but more about the possibility that the class of usernames used could be just prone to collisions.
I also looked into perfect hashes, but as the users can be added dynamically, that's going to be too hard to manage.
Another option I've thought of was that (when adding the user) if there were a hash collision, I would reply to the client with a request for another hash, and repeat until there was no collision. I'd repeat this process during log-in. However, I am also wondering if this actually makes it easier for an attacker, as they'd have more feedback about what hashes were successful, and if the database were compromised, all the additional hashes would make recovering the original value easier.
I was also considering encrypting the username using the username as a password, but I'm guessing this also suffers from collisions (because each entry has a unique password--two different plain-texts with two different passwords could result in the same cipher-text), so I'm thinking it's not worth exploring this further.
I don't really want to go with a custom username (where the user has to come up with something that hasn't been taken when they sign-up), as I'm expecting the user to very rarely use the service, and are likely to forget their username.
I'm currently thinking I will just go with the first idea of hashing once, and if there is a collision, have the password decide (and hope there's no collision there too--I could put a warning when the user signs-up saying that their username/password is not allowed because it will log them in as another user perhaps /S).
Is there any non-colliding way of creating a secure form of username?
Thank you.
Assuming we are talking about emails, as there aren't many other options usable for login names.
I was also considering encrypting the username using the username as a password, but I'm guessing this also suffers from collisions (because each entry has a unique password--two different plain-texts with two different passwords could result in the same cipher-text), so I'm thinking it's not worth exploring this further.
Collisions here are the wrong thing to worry about here ...
Mandatory disclaimer: Encryption keys are not the same things as passwords. And encrypting the plainText with itself as the key is even worse.
The problem with encryption is that cipherTexts aren't searchable; i.e. you cannot verify for uniqueness, unless you decrypt all user records each time, so this just isn't sustainable - your server loads will grow exponentially with each new user record.
That's because while encryption makes use of IVs (Initialization Vector; i.e. the equivalent of salts in password hashing), which results in different cipherText even if you encrypt the same plainText twice, using the same key.
However, it is very likely that you will need to encrypt those emails, as if you need to send out password reset links, notifications, etc. - you'll need a two-way mechanism. You can't do these things with hashes, because they are one-way only.
There's a reason why every website couples its user accounts with email addresses, even if they are not the login names. :)
What you can do for login checks only, is to store a HMAC (Hash-based Message Authentication Code) of the email.
HMACs look just like regular hashes, but are actually "keyed hashes" (i.e. you would use a key while hashing, similarly to encryption). And in addition to that, nobody has managed to find collisions with the HMAC construct so far, even with the now famously insecure MD5 (still, please use a modern algorithm; at least SHA-2).
I should note that HMACs aren't nearly as strong as password hashing algorithms, so your users emails certainly won't be as strongly protected as their passwords, but it's not like there's anything else you can do about it, and it should be OK.
In summary, you'll need to have two separate cryptographic keys configured in your application - one for encryption, and one for the HMACs - and the following data stored:
userLoginLookup - HMAC of the email, using one of the two keys
userLoginMailer - cipherText of the email, using the second configured key
userPassword - a standard password hash; using bcrypt, PBKDF2 or scrypt
Note: Cryptography is always case-sensitive, so to accomodate lookups, you need to always normalize the email addresses first; i.e. make them all-lowercase or all-uppercase.
When a user attempts to login, you do a HMAC(emailInput, hmacKey) and search for a match with the userLoginLookup field in your database.
When you need to send a notification or password reset email, you decrypt the userLoginMailer.
The approach that I'm taking right now is:
take the PasswordSalt
hash it once more with SHA1 and some application-level salt
and then generate the Reset Password link from that
The advantage is that it is pretty straightforward, and I don't need to create any new database columns.
The disadvantage could be that the link is always the same for that user.
Is this a security problem?
This seems a big compromise simply to avoid extra database columns and to provide simplicity for you.
You should securely generate a token for use in the link and for storage in the DB. This has the advantage that it can be randomly generated each time and an expiry data can be associated to the token.
This will minimise the risk as the reset link has a limited life span and has less chance of falling into the wrong hands.
I disagree with security questions being used at all (in #Marcus Adams's answer) because these questions can usually be easily guessed if the attacker knows the victim (or can usually easily find out these days via publically available information such as Facebook profiles). Also, they have the disadvantage of not being able to be updated easily (e.g. your favourite pet's name will remain the same unless you buy a new, better pet - or if you make one up you'll probably forget it, defeating the object of it in the first place).
I'm assuming that you're emailing the password reset links and that the user has to enter a userid/email address to start the process.
Make sure you generate a new random salt each time the user resets their password, so that there can't be any replay attacks.
For example, if the salt doesn't change, the password reset link wouldn't change, even after a password reset. If a hacker were able to obtain the link, even a year later, the hacker would be able to re-use that link to reset the password to whatever they want.
Quite often, after clicking the password reset link, the user is asked to answer a security question that they setup previously. This helps mitigate the issue of a hacker obtaining and using the link before the user does.
The fact that you can reset a password at any time, even if it wasn't requested, smells, but since doing this requires the password salt, secret application-level salt (really, a key, since it's not random), and perhaps a security question, not to mention the fact that requesting a password reset link is trivial (only need userid/email address), it should be good enough.
You may have some performance issues down the road since in order to verify a password reset link, you'll have to scan all the rows in the user table, applying the hash to each row, until you find one that matches.
Probably a super basic question. I know many online services hash and salt passwords instead of storing them as plaintext for security purposes. My university's web portal requires students to change their passwords every 6 months. From what I know, the system is built on Oracle software.
My question is, however, how does the system know when my 20 character long password (with capitals, numbers, and symbols) contains 3 characters in the same order as the new password I'm trying to set? If the passwords are hashed, shouldn't the algorithm be one-way? Or is it possible that system encrypts the plaintext passwords and stores them? Wouldn't that be less secure?
Sorry if the question is hard to understand. Let me know if you need me to clarify. Thanks in advance!
If you have to enter your previous password when creating a new one, the system can compare them directly. This could even be done client-side.
EDIT
There are only a few other possibilities
They store your password in plaintext (in which case they should fire their entire IT department)
Their encryption method is two-way i.e. it can be decrypted (in which case they should fire their entire IT department)
They temporarily store your password when you log in. Maybe in a cookie or on the server. (In which case they should fire their entire IT department)
It is likely that the prevoius password table is encrypted (possibly using rot26).
The system can only check if the new password matches the old password exactly (compares the hashes). If it's checking substring matches, the passwords are likely being stored in plaintext.
No bueno.
EDIT: Or what Nick said, of course.
I've been asked to implement some changes/updates to an intranet-site; make it 'future proof' as they call it.
We found that the passwords are hashed using the MD5 algorithm. (the system has been around since 2001 so it was adequate at time).
We would now like to upgrade the hashing-algorithm to a stronger one (BCrypt-hash or SHA-256).
We obviously do not know the plaintext-passwords and creating a new password for the userbase is not an option*).
So, my question is:
What is the accepted way to change hashing-algorithm without having access to the plaintext passwords?
The best solution would be a solution that is entirely 'behind the scenes'.
*) we tried; tried to convince them, we used the argument of 'password age', tried to bribe them with coffee, tried to bribe them with cake, etc. etc. But it is not an option.
Update
I was hoping for some sort of automagic solution for solving the problem, but apparently there are no other options than just 'wait for the user to log in, then convert'.
Well, at least now I now there is no other solution available.
First, add a field to the DB to identify whether or not the password is using MD5 or the new algorithm.
For all passwords still using MD5:
-- In the login process, where you verify a user's entered password: temporarily store the user's submitted password in memory (no security issue here, as it is already in memory somewhere) and do the usual MD5 hash & compare with the stored hash;
-- If the correct password was given (matches the existing hash), run the temporarily stored password through the new algorithm, store that value, update the new field to identify that this password has been updated to the new algorithm.
(Of course you would just use the new algorithm for any new users/new passwords.)
I'm not entirely sure about this option, since I'm not an expert on cryptography. Please correct me if I'm wrong at some point here!
I think Dave P. has clearly the best option.
... but. There is an automagic solution - hash the older hashes themselves. That is, take the current hashes, and hash them again with a stronger algorithm. Notice that as far as I understand, you don't get any added security from hash length here, only the added cryptographical complexity of the new algorithm.
The problem is, of course, that checking a password would then have to go through both hashes. And you'd have to do the same for evey new password as well. Which is, well, pretty much silly. Unless you want to use a similar scheme like Dave P. explained to eventually graduate back to single-hashed passwords with the new hashing algorithm... in which case, why even bother with this? (Granted, you might use it in a flashy "Improved security for all passwords, applied immediately!"-way at a presentation to corporate suits, with a relatively straight face...)
Still, it's an option that can be applied immediately to all current passwords, without any gradual migration phase.
But boy, oh boy, is someone going to have a good laugh looking at that code later on! :)
Add passwordChange datetime field to the database.
All password set before day X, check using MD5
All passwords set after day X, check using BCrypt or whatever.
You could store, either in the hash field itself (e.g. "MD5:d41d8cd98f00b204e9800998ecf8427e") or in another column, which algorithm was used to create that hash. Then you'd have to modify the login process to use the correct algorithm when checking the password. Naturally, any new passwords will be hashed using the new algorithm. Hopefully, passwords eventually expire, and over time all of the MD5 hashes will be phased out.
Since you don't know plaintext password, maybe you should to create a field which indicates encription version (like PasswordVersion bit default 0)
Next time user tries to log in, check hashed password using current algorithm version, just like you do today. If it matches, hash it again and update PasswordVersion field.
Hopefully you'll not need a PasswordVersion column bigger than bit. =)
You should change your password database to store 3 items:
An algorithm identifier.
A random salt string chosen by the server when it first computes and stores the password hash.
The hash of the concatenation of salt+password using the specified algorithm.
Of course these could just be stored together in one text field with a delimiter:
"SHA256:this-is-salt:this-is-hash-value"
Now convert you existing entries to a value with empty salt and the old algorithm
"MD5::this-is-the-old-md5-hash-without-salt"
Now you have enough information to verify all you existing password entries, but you can also verify new entries (since you know which hash function was used). You can convert the old entries to the new algorithm the next time the existing users login since you will have their password available during this process:
If your database indicates they are using the old algorithm with no salt, first verify the password the old way by checking that the MD5 hash of the password matches. If not, reject the login.
If the password was verified, have the server choose a random salt string, compute the SHA256 hash of the salt+password, and replace the password table entry with a new one specifiy the new algorithm, salt and hash.
When the user logs in again, you'll see they are using the new algorithm, so compute the hash of the salt+password and check that it matches the stored hash.
Eventually, after this system has been running for a suitable time, you can disable accounts that haven't been converted (if desired).
The addition of a random salt string unique to each entry makes this scheme much more resistent to dictionary attacks using rainbow tables.
The best answer is from an actual cryptography expert
https://paragonie.com/blog/2016/02/how-safely-store-password-in-2016#legacy-hashes
This post also helps explain which hashing you should use. It's still current even if it says 2016. If in doubt use bcrypt.
Add a column to your user accounts table, called legacy_password (or equivalent). This is just a Boolean
Calculate the new stronger hash of the existing password hashes and store them in the database.
Modify your authentication code to handle the legacy flag.
When a user attempts to login, first check if the legacy_password flag is set. If it is, first pre-hash their password with your old password hashing algorithm, then use this prehashed value in place of their password. Afterwards (md5), recalculate the new hash and store the new hash in the database, disabling the legacy_password flag in the process.
Ok, so the whole problem with hashes is that users don't enter passwords over 15 characters long. Most only use 4-8 characters making them easy for attackers to crack with a rainbow table.
Solution, use a user salt to make hash input more complex and over 50chars so that they will never be able to generate a table (way to big for strings that size). plus, they will have to create a new table for each user. Problem: if they download the db they will get the user salt so you are back to square one if they care enough.
Solution, use a site "pepper" plus the user salt, then even if they get the DB they will still have to know the config file. Problem: if they can get into your DB chances are they might also get into your filesystem and discover your site pepper.
So, with all of this known - lets assume that an attacker makes it into your site and gets everything, EVERYTHING. So what do you do now?
At this point in the discussion, most people reply with "who cares at this point?". But that is just a cheap way of saying "I don't know what to do next so it can't be that important". Sadly, everywhere else I have asked this question that has been the reply. Which shows that most programmers miss a very important point.
Lets image that your site is like the other 95% of sites out there and the user data - or even full sever access - isn't worth squat. The attacker happens to be after one of your users "Bob" because he knows that "Bob" uses the same password on your site as he does on the banks site. He also happens to know Bob has his life savings in there. Now, if the attacker can just crack our sites hashes the rest will be a piece of cake.
So here is my question - How do you extend the length of the password without any traceable path? Or how do you make the hashing process to complex to duplicate in a timely manner? The only thing that I have come up with is that you can re-hash a hash several thousand times and increase the time it would take to create the final rainbowtable by a factor of 1,000. This is because the attacker must follow that same path when creating his tables.
Any other ideas?
Solution, use a user salt to make hash
input more complex and over 50chars so
that they will never be able to
generate a table (way to big for
strings that size). plus, they will
have to create a new table for each
user. Problem: if they download the db
they will get the user salt so you are
back to square one if they care
enough.
This reasoning is fallacious.
A rainbow table (which is a specific implementation of the general dictionary attack) trades space for time. However, generating a dictionary (rainbow or otherwise) takes a lot of time. It is only worthwhile when it can be used against multiple hashes. Salt prevents this. The salt does not need to be secret, it just needs to be unpredictable for a given password. This makes the chance of an attacker having a dictionary generated for that particular salt negligibly small.
"The only thing that I have come up with is that you can re-hash a hash several thousand times and increase the time it would take to create the final rainbowtable by a factor of 1,000."
Isn't that exactly what the Blowfish-based BCrypt hash is about? Increasing the time it takes to compute a hash so that brute force cracking (and rainbow table creation) becomes undoable?
"We present two algorithms with adaptable cost (...)"
More about adaptable cost hashing algorithms: http://www.usenix.org/events/usenix99/provos.html
How about taking the "pepper" idea and implementing it on a separate server dedicated to hashing passwords - and locked down except for this one simple and secure-as-possible service - possibly even with rate-limits to prevent abuse. Gives the attacker one more hurdle to overcome, either gaining access to this server or reverse engineering the pepper, custom RNG and cleartext extension algorithm.
Of course if they have access to EVERYTHING they could just evesdrop on user activity for a little while..
uhmm... Okay, my take on this:
You can't get the original password back from a hash. I I have your hash, I may find a password that fits that hash, but I can not log in to any other site that uses this password, assuming they all use salting. No no real issue here.
If someone gets your DB or even your site to get your config, you're screwed anyway.
For Admin or other Super Accounts, implement a second mean of verification, i.e. limit logins to certain IP ranges, use Client-Side-SSL Certificates etc.
For normal users, you won't have much chance. Everything you do with their password needs to be stored in some config or database, so if have your site, I have your magic snake oil as well.
Strong Password limitations don't always work. Some sites require passwords to have a numeric character - and as a result, most users add 1 to their usual password.
So I'm not entirely sure what you want to achieve here? Adding a Salt to the front of the users password and protecting Admin accounts with a second mean of authentication seems to be the best way, given the fact that users simply don't pick proper passwords and can't be forced to either.
I was hoping that someone might have a solution but sadly I am no better off then when I first posted the question. It seems that there is nothing that can be done but to find a time-costly algorithm or re-hash 1,000's of times to slow down the whole process of generating rainbow tables (or brute-forcing) a hash.