I need to hash some passwords with salt on postgresql, and I haven't been able to find any relevant documentation on how to get that done.
So how can I hash passwords (with some salts) in postgresql?
It's been a while since I asked this question, and I'm much more familiar with the cryptographic theory now, so here is the more modern approach:
Reasoning
Don't use md5. Don't use a single cycle of sha-family quick hashes. Quick hashes help attackers, so you don't want that.
Use a resource-intensive hash, like bcrypt, instead. Bcrypt is time tested and scales up to be future-proof-able.
Don't bother rolling your own salt, you might screw up your own security or portability, rely on gen_salt() to generate it's awesome unique-to-each-use salts on it's own.
In general, don't be an idiot, don't try to write your own homegrown crypto, just use what smart people have provided.
Debian/Ubuntu install packages
sudo apt-get install postgresql // (of course)
sudo apt-get install postgresql-contrib libpq-dev // (gets bcrypt, crypt() and gen_salt())
sudo apt-get install php5-pgsql // (optional if you're using postgresql with php)
Activate crypt() and bcrypt in postgresql in your database
// Create your database first, then:
cd `pg_config --sharedir` // Move to the postgres directory that holds these scripts.
echo "create extension pgcrypto" | psql -d yOuRdATaBaSeNaMe // enable the pgcrypo extension
Use crypt() and gen_salt() in queries
Compare :pass to existing hash with:
select * from accounts where password_hash = crypt(:pass, password_hash);
//(note how the existing hash is used as its own individualized salt)
Create a hash of :password with a great random salt:
insert into accounts (password) values crypt(:password, gen_salt('bf', 8));
//(the 8 is the work factor)
From-in-Php bcrypt hashing is slightly preferrable
There are password_* functions in php 5.5 and above that allow trivially simple password hashing with bcrypt (about time!), and there is a backward compatibility library for versions below that. Generally that hashing falls back to wrapping a linux system call for lower CPU usage anyway, though you may want to ensure it's installed on your server. See: https://github.com/ircmaxell/password_compat (requires php 5.3.7+)
Be careful of logging
Note that with pg_crypto, the passwords are in plaintext all during the transmission from the browser, to php, to the database. This means they can be logged in plaintext from queries if you're not careful with your database logs. e.g. having a postgresql slow query log could catch and log the password from a login query in progress.
In Summary
Use php bcrypt if you can, it'll lessen the time that the password remains unhashed. Try to ensure your linux system has bcrypt installed in it's crypt() so that is performant. Upgrade to at least php 5.3.7+ is highly recommended as php's implementation is slightly buggy from php 5.3.0 to 5.3.6.9, and inappropriately falls back to the broken DES without warning in php 5.2.9 and lower.
If you want/need in-postgres hashing, installing bcrypt is the way to go, as the default installed hashes are old and broken (md5, etc).
Here are references for more reading on the topic:
http://codahale.com/how-to-safely-store-a-password/
http://www.postgresql.org/docs/9.2/static/pgcrypto.html
https://github.com/ircmaxell/password_compat
An application should hash its passwords using key derivation function like bcrypt or pbkdf2. Here is more information on secure password storage.
... but sometimes you still need cryptogrpahic functions in a database.
You can use pgcrypto to get access to sha256 which is a member of the sha2 family. Keep in mind sha0,sha1 md4, and md5 are very broken and should never be used for password hashes.
The following is an alright method of hashing passwords:
digest("salt"||"password"||primary_key, "sha256")
The salt should be a large randomly generated value. This salt should be protected, because the hashes cannot be broken until the salt is recovered. If you are storing the salt in the database then it can be obtained along with the password hash using sql injection. Concatenating the primary key is used to prevent 2 people from having the same password hash even if they have the same password. Of course this system could be improved, but this is much better than most systems I have seen.
Generally it is best to do hashing in your application before it hits the database. This is because querys can show up in logs, and if the database server was owned then they could enable logging to get clear text passwords.
Examples and documentation on: http://www.postgresql.org/docs/8.3/static/pgcrypto.html
UPDATE ... SET pswhash = crypt('new password', gen_salt('md5'));
SELECT pswhash = crypt('entered password', pswhash) FROM ... ;
Related
i have stored md5 password with salt (for ftp server) on an old server.
The oldest passwords have illegal chars in the salt (like '=').
example not working password:$1$y8qs4=ZJ$pjAIhiNiCazGVPVOo1EQH. on Ubu20.04 with pure ftp but works on strech with proftp
example working password:$1$eHvgbggs$A/UaFfYsmAr7yPcmpMxEK. works on both systems.
Think its because the illegal '=' in the salt.
Is there a way to tell pureftp/ubuntu to ignore illegal salts?
thanks
I'd take the following steps to send/received data between client and server. But I'm not sure if all the steps are secure enough and impossible to intercept. Please can you let me know how to patch up the security holes, if any?
Please note that:
This is all for Symmetric cryptography not the Public/Private Key method. So 'salt' and 'entropy' are the secrets between client and server.
For some reasons, I cannot/don't use SSL or TLS protocols. So there is no Certificate Authority etc.
Only standard cryptography functions and methods must be used (no inventions).
The data are not sent over a secure (HTTPS) connection.
I cannot use sessions here because they are two different applications.
First time - Sign up (user enters new username and new password)Client sideCreate a CSPRNG salt for the userSave salt on user's machineCreate an entropy in memory (base64 of a temporal value e.g. [hour of the day]+[day of the year])Post Password(base64 of the plaintext), and salt, and entropy to the serverServer sideFor the received message from the clietCheck if entropy matches (e.g. with base64 of [hour of the day]+[day of the year]) .Compute hash (SHA1) of salt and Password(base64 of the plaintext) - because hashing must always be done on the server.Save salt and the computed hash to the database
Next time - Log in (user enters his username and his password)Client sideRead salt from user's machineCompute hash (SHA1) of salt and entered password(base64 of the plaintext)Post password(base64 of the plaintext), and salt, and the computed hash to the serverServer sideRetrieve salt and 'stored hash' from the databaseFrom the received message from the clientSplit the message to 3 parts: password(base64 of the paintext); salt; 'received hash'Compare 'received hash' with 'stored hash'; if they match, the user is genuine not a hacker
Sending TemperProof queryString from user to serverClient sideRead salt from user's machineCreate an entropy in memory (base64 of a temporal value e.g. [hour of the day]+[day of the year])Compute hash (SHA1) of the queryString(base64 of the plaintext), and salt, and entropyPost querystring(base64 of the plaintext), and salt, and entropy, and the computed hash to the serverServer sideRetrieve salt from the databaseFor the received message from the clietSplit the message to 4 parts: queryString(base64 of the paintext); salt; entropy; 'received hash'Check if entropy matches (e.g. with base64 of [hour of the day]+[day of the year]) .Compute hash (SHA1) of queryString(base64 of the plaintext) and salt and entropy.Compare the computed hash with the 4th part of the splitted message ('received hash'); if they match, the queryString is genuine
Sending answer back to the user from serverServer sideCompute the answer using database queriesRetrieve salt from the databaseCreate an entropy in memory (base64 of a temporal value e.g. [hour of the day]+[day of the year])Compute hash (SHA1) of the answer(base64 of the plaintext), and salt, and entropyPost answer(base64 of the plaintext), and salt, and entropy, and the computed hash to the clientClient sideRead salt from user's machineCreate an entropy in memory (base64 of a temporal value e.g. [hour of the day]+[day of the year])Split the received message to 4 parts: answer(base64 of the paintext); salt; entropy; 'received hash'Check if entropy matches (e.g. with base64 of [hour of the day]+[day of the year]) .Compute hash (SHA1) of answer(base64 of the plaintext) and salt and entropy.Compare the computed hash with the 4th part of the splitted message ('received hash'); if they match, the answer is genuine
The followings areas are the weaknesses, I think. Can you please advise how these can be fixed and also point out the other possible holes?
A) First time - Sign up: The hacker can post rubbish to fill up the database, if he finds entropy in step 1.3
B) Next time - Log in: I cannot think of a way to add an entropy to the hashed password in step 1.2, because then I cannot compare with the one on the server database in step 2.2.2
Thanks
if you care about security at all, STOP now ...
what you need to do:
1) forget about designing your own crypto protocol ... relying on well known crypto ALSO means that you DO NOT design that kind of thing
2) think in layers ... you have the need to keep things secret while transporting them from A to B ... that means you have a transport layer ... if you want that secured, there is a name for that...
Transport Layer Security -> https://en.wikipedia.org/wiki/Transport_Layer_Security
3) when you make assumptions here like "i have 2 applications so i can not have sessions", please provide WHY you think it is that way... when you think of things like single-sign-on, you can have a lot of applications sharing one authentication method and even session data across a bunch of different plattforms ... maybe it's just you don't know that you actually can have sessions...
4) read up on the terms you use ... you misunderstood entropy ... there is no way of "checking if entropy matches" ... entropy in crypto related cases means randomness in terms of unpredictable input to a function ... if you have something like a date and the time, and even if you hash that, it might look random ... but it is very predictable by someone with a clock... if the communication can be related to the creation time of the value, then your value does not contain large amounts of entropy if it is based on the clock... again, do not design you own stuff here, and go for entropy sources that provide reliable, cryptographically secure randomness (CSPRNG ... not just PRNG)
5) ... you misunderstood/misused salt...
that is nothing to be generated on the client machine
that is nothing that needs to be kept on the client machine
7) password hashing
again ... DO NOT come up with you own stuff here ...
create a sufficently long random salt (at least hash length) for every password. use a slow hash function like PBKDF2 with a high iteration count parameter. the reason is that it becomes slow to test for passwords... your server has to calculate this once for every login attempt ... an attacker has to calculate this for every password testing attempt ... you can afford the test to take like 200ms ... for an attacker that means a lot more hardware will be needed to crack your password storage...
upd:
you want it, you get it ...
proof of concept attack on your schema by man-in-the-middle:
client: alice
server: bob
attacker/eavesdropper: eve
alice uses your sign up service and creates an account
1.1 alice creates a CSPRNG salt and stores that in a secure manner
1.2 alice gathers an arbitrary amount of entropy and encodes it with base64
1.3 alice sends Password(base64 of the plaintext), and salt, and entropy to bob
------------intercepted-------------
eve intercepts the communication between alice and bob and gains knowledge of...
2.1 ...the base 64 encoded password -> base64 decode -> the plaintext password
2.2 ...the salt
2.3 ...the entropy value
2.4 alice forwards the intercepted communication without changes to bob
--- protocol broken ---
now, bob is by no means able to distinguish between alice and eve in all further communication
upd2:
a look on your transfered (cleartext) messages:
Login:
Post password(base64 of the plaintext), and salt, and the computed hash to the server
Sending queryString from user to server:
Post querystring(base64 of the plaintext), and salt, and entropy, and the computed hash to the server
answer:
Post answer(base64 of the plaintext), and salt, and entropy, and the computed hash to the client
now for any of those messages, let's look at what information someone with malicious intent would learn from those:
all the information that is cleartext, which means, all of it ...
for the login, we gain the clear text password, means from now on an attacker can identify as a valid user
for the querystring and answer thing you want to provide a way to see if the request/answer is not tempered with.
so if an attacker now intercepts your communication how can he change the querystring without being noticed?
the attacker splits your message, and changes whatever he/she wants
then he/she computates the forged_hash as sha1(salt_from_the_original_message,tampered_querystring) and sends base64(tampered_querystring),salt_from_the_original_message,entropy_from_original_message,forged_hash to the server ...
for the answer it's the same deal:
the attacker intercepts the original answer, changes whatever in the answer and recomputes the hash based on known information (the changes, and the original salt)
The solution is to use HTTPS withTLS 1.2 and pin the certificate, there are certificate solutions that are free.
Using a hash (SHA1 in this case) to protect a password has not been good secure practice for some time. For my reference see DRAFT NIST Special Publication 800-63B Digital Authentication Guideline.
Passwords must be protected with an iterated HMAC, not a single hash. For more information on passwords see Toward Better Password Requirements by Jim Fenton, see slide 23 in particular. This seems to be at odds with the Pluralsight training, best practices have changes over time.
I have a question regarding using Login Names to protect passwords.
You salt the Login Name with a shared salt and then hash it with BCrypt.
You then take the original plain text Login Name and use it as a key to encrypt the password with AES. The result is then salted with a unique salt and finally hashed with BCrypt.
The user's Display Name is set to their User ID (integer), as opposed to their Login Name, when the account is created. The user can change it later, except it can not match anyone else's Display Name or too closely match a case-insensitive comparison to their Login Name.
My question is, if the database were compromised, would this make it significantly harder to recover the passwords than storing plaintext usernames and uniquely salted and BRcypted passwords?
This procedure appears to be very complicated. Actually you are adding a server side secret to the stored hashes (your procedure to hash passwords). That will indeed increase security, as long as the procedure stays secret. In other words, if the attacker has only read access to the database (SQL-injection) you have an advantage, as soon as he has got privileges on the server and knows the code, there is no advantage at all.
There is a much easier and safer way to get this advantage though. Just calculate a BCrypt hash with a random salt (most implementations will do that anyway), then encrypt this hash-value with a server-side key (AES for example). The key should not be derrived form other parameters, instead use a long and random enough key.
I tried to explain the reasons in my tutorial about safely storing passwords, maybe you want to have a look at it.
Edit:
Well i understand now, that you want to handle the login-name like a second password, and will not store it plaintext, instead you store only a BCrypt hash of it with a global "salt".
Lets assume that you are willing to spend 1 second of CPU time for password hashing. In your scheme you would have to split it to half a second for hashing the login-name and half a second for the password. An attacker has to brute-force the login-name first and the password in a second step.
Login-names are normally very weak passwords. While people are learning that passwords need to be strong, login-names often contain only a name with a number and are short.
You need to find the database record with the hash of the login-name, so you cannot use a random salt, instead you need to use a global "salt". This allows to build a single rainbow table to crack all usernames in one go. Important login-names like "admin" can be precalculated.
To avoid duplicates, you have to uppercase/lowercase the login-name, this reduces the search space even more.
That means you spend half a second for a very weak password (login-name), then half a second for the normal password. Compared to investing 1 second for the normal (hopefully strong) password, you are probably decreasing security, by all means i cannot see any advantage.
The authentication code would look something like this.
public static LoginResult TryLogin(string loginName, string pwd)
{
string loginHash = BCrypt.Net.BCrypt.HashPassword(loginName, SHARED_SALT);
WidgetDataContext dc = new WidgetDataContext();
var record = (from rec in dc.usp_GetUserByLoginName(loginHash)
select rec).SingleOrDefault();
if (record == null)
return new LoginResult(null, "Invalid Login Name/Password");
if (record.FailedLoginCount >= MAX_CONSECUTIVE_LOGIN_FAILURES)
return new LoginResult(null, "You have exceeded your maximum number of Login failures. Your account is locked.");
if (record.Locked) // In case account is locked for another reason
return new LoginResult(null, "Your Account is locked.");
pwd = EncryptionServices.Encrypt(pwd, loginName);
pwd = BCrypt.Net.BCrypt.HashPassword(pwd, record.Salt);
if (pwd == record.Password)
{
record.FailedLoginCount = 0;
dc.SubmitChanges();
return new LoginResult(record.UserId, "Login Successful");
}
record.FailedLoginCount++;
dc.SubmitChanges();
return new LoginResult(null, "Invalid Login Name/Password");
}
Currently I am using a particular scheme for securing passwords, and I think I have some points for improvement. The implementation is in Java, so I prefer to use SHA-2 512 as encryption form.
Currently I have a client-server model, so these things can happen:
Client wants to login, he sends his password with one time normal SHA-2 512 encryption over the network.
The server has the passwords stored in the database as for example SHA-2_512(SHA-2_512(password) + salt), with the inner SHA-2_512(password) being the 'encrypted' password it receives over the network.
Password checks are done server side and there is no way anything can leak out from the server, the only possible vulnerability would be if someone could read out the RAM I think.
I have these questions:
An attacker usually creates collision attacks when wanting to hack a password. However how are collision attacks sufficient? If the password needs to be used for other applications like Outlook.com, Facebook or whatever (which likely use another salt as they have nothing to do with my applications), how is a collision attack enough then? Don't you need the real password?
Does SHA-2 512 already use iteration? And even if so, should I change my encryption methods to automatically use a number of iterations plus how many iterations is preferred? I have also read about using a random number of iterations (in a range), how do I store the random factor determenistically?
Should I store system secrets for every iteration in the server code? See http://blog.mozilla.org/webappsec/2011/05/10/sha-512-w-per-user-salts-is-not-enough/ . I could store an array which would hold a static secret for every iteration, with the nth secret being for the nth iteration. Nobody can know the secrets, they are computed once (I guess as encrypting some random string), and then basically stored in the Server's RAM.
Currently I send the typed password from the client to the server as just SHA-2_512(password), should this process be improved, and if so, how? I cannot use salts, because the client does not have the salt available.
Regards.
TLDR: You need to send the password using an encrypted channel, such as TLS. Consider using bcrypt for password hashing.
SHA-2 512 is not an encryption algortihm, it is a message digest algorithm. An encryption algorithm requires a key and a message to encrypt. It produces ciphertext. The important thing is that an encryption algorithm has a decryption algorithm.
ciphertext = E(key, plaintext);
plaintext = D(key, ciphertext);
A message digest takes a piece of plaintext and produces a message digest. There is no corresponding reverse mechanism to take a message digest and retrieve the original message. There is also no secret key.
digest = hash(plaintext);
If an attacker is able to access a database with hashes, the attacker can retrieve the original password by brute forcing, trying lots of guesses with the hash algorithm.
digest1 = hash(guess1);
digest2 = hash(guess2); //repeat with lots of guesses
Firstly, sending a hash over a network is not secure. It needs to be sent through some secure communications mechanism such as SSL. If an attacker can intercept the hash over the communications they may be able to work out the orignal password.
A hash collision is not the same as brute forcing the password. A hash collision is caused when two different messages produce the same message digest.
digest1 = hash(plaintext1);
digest2 = hash(plaintext2);
if ( ( plaintext1 != plaintext2 ) && ( digest1 == digest2 ) )
// hash collision
SHA-512 does not have iterations designed to prevent brute-forcing. The SHA set of algorithms are designed to be efficient. The reason for adding iterations when hashing passwords is to increase the time it takes to brute force a password. The idea being the cost to perform a legitimate login attempt and perform 100 iterations is tiny compared to an attacker who has millions of passwords, each of which requires 100 iterations. Adding more iterations helps reduce the impact of improved processor speeds (which would help an attacker try more iterations quicker).
You should make the number of iterations a configurable limit that is stored against each user. So you store the password hash, salt and iteration count for each user. This means that in the future you can increase the number of iterations to take into account increased hardware power.
Sending the SHA-2 512 in plaintext is not secure. You should send it within an encrypted channel, such as SSL.
Having said all that, SHA-2 is not designed to be a password hashing algorithm. It is designed for message validation and is to be efficient. Consider using a purpose built password hashing algorithm. One example is bcrypt. It is designed to be computationally difficult and has salt and iterations built in.
Imagine a NIS user database /var/yp/input-files/passwd consisting of old crypt DES hashes. The aim is to migrate this database to md5 or sha hashes. When changing a password using yppasswd, it is generated using the same algorithm as for the existing entry (probably for legacy reasons), i.e. crypt stays crypt, md5 stays md5.
Our current plan is to write a special password change script wrapping yppasswd. Is there a more elegant way to tell yppasswd to generate changed passwords in md5 format by default?
I figured it out by now:
Download the nis-package of your distribution e.g. apt-get source nis under Debian.
Edit corresponding yp-tools-2.9/src/yppasswd.c
Replace int has_md5_passwd = 0; by int has_md5_passwd = 1; in the beginning of int main (int argc, char **argv)
yppasswd then always creates md5 hashes.
Recent Linux distributions support NIS password updates through PAM, which means you can use the local passwd program instead of yppasswd. This also means that you can configure the hash algorithm, which is still (as of yp-tools 2.12) not possible with yppasswd.
Your system should have an /etc/pam.d/passwd file, which will either contain or, more likely, refer to another file that contains a line of the form:
password sufficient pam_unix.so md5 nis
The second field might differ, and there might be other parameters at the end, but you want the password line that loads pam_unix.so and it should have at least the nis parameter as well as a hash function appropriate to your NIS server (md5, sha256, etc.)
Your distribution might have a better way of configuring this, though, so make sure your changes aren't going to be overwritten by an automated process later.
On RedHat-derived systems (including at least RHEL, CentOS, Scientific Linux, and Fedora), you need to use the authconfig program with at least the --enablenis parameter. For the above scenario with MD5 hashes, the following command line would be sufficient:
authconfig --enablenis --usemd5 --update
On Debian systems (and possibly Debian derivatives, but I can't check at the moment), the proper way to do this appears to be to copy /usr/share/pam-configs/unix to something else (say, /usr/share/pam-configs/nis), edit the new file to give it appropriate settings and a new profile name, then run pam-auth-update, disable the "Unix authentication" profile and enable your new profile.