Many users – myself included – would like the security of having everything they do on a web service encrypted. That is, they don't won't any one at the web service to be able to look at their: posts, info, tasks, etc...
This is also major complaint in this discussion of an otherwise cool service: http://news.ycombinator.com/item?id=1549115
Since this data needs to be recoverable, some sort of two-way encryption is required. But unless you're prompting the user for the encryption key on every request, this key will need to be stored on the server, and the point of encrypting the data is basically lost.
What is a way to securely encrypt user data without degrading the user experience (asking for some key on every request)?
-- UPDATE --
From #Borealid's answer, I've focused on two possibilities: challenge-response protocols, where no data (password included) is sent in the "clear", and non-challenge-response protocols, where data (password included) is sent in the "clear" (although over HTTPS).
Challenge-response protocols (specifically SRP: http://srp.stanford.edu/)
It seems that its implementation would need to rely on either a fully AJAX site or using web storage. This is so the browser can persist the challenge-response data during encryption and also the encryption key between different "pages". (I'm assuming after authentication is completed I would send them back the encrypted encryption key, which they would decrypt client-side to obtain the real encryption key.)
The problem is that I'm either:
fully AJAX, which I don't like because I love urls and don't won't a user to live exclusively on a single url, or
I have to store data encryption keys in web storage, which based on http://dev.w3.org/html5/webstorage/ will persist even after the browser is closed and could be a security vulnerability
In addition, as SRP takes more than one request ( http://srp.stanford.edu/design.html ), there needs to be some persistence on the server-side. This is just another difficulty.
Traditionally
If I'm ok transmitting passwords and data in the clear (although over HTTPS), then the client-side issues above are not present.
On registration, I'll generate a random unique encryption key for the user, and encrypt it using their password and a random salt.
In the database, I'll store the user's password hash and salt (through bcrypt), encrypted encryption key, encryption key salt, and encryption iv.
After an authentication, I'll also need to use their password to decrypt the encryption key so that they may view and enter new data. I store this encryption key only temporarily and delete it when they explicitly "log out".
The problems with this approach is that (like #Borealid points out) evil sysadmins can still look at your data when you are logged in.
I'm also not sure how to store the encryption keys when users are logged in. If they are in the same data store, a stolen database would reveal all data of those who were logged in at the time of theft.
Is there a better in-memory data store for storing these encryption keys (and challenge data during an SRP authentication)? Is this something Redis would be good for?
If the data need to be recoverable in the event of user error, you can't use something like a cookie (which could get deleted). And as you point out, server-side keys don't actually secure the user against malicious sysadmins; they only help with things like databases stolen offline.
However, if you're running a normal web service, you've already gotten pretty lucky - the user, in order to be unique and non-ephemeral, must be logged in. This means they go through some authentication step which proves their identity. In order to prove their identity, most web sites use a passed credential (a password).
So long as you don't use a challenge-response authentication protocol, which most web sites don't, you can use an encryption key derived from a combination of a server-side secret and the user's password. Store the encryption key only while the user is authenticated.
If you do this, the users are still vulnerable to sysadmins peeking while they're using the service (or stealing their passwords). You might want to go a step further. To go one up, don't send the password to the server at all. Instead, use a challenge-response protocol for authentication to your website, and encrypt the data with a derivative of the user's password via JavaScript before uploading anything.
This is foolproof security: if you try to steal the user's password, the user can see what you're doing because the code for the theft is right there in the page you sent them. Your web service never touches their data unencrypted. This is also no hindrance to the normal user experience. The user just enters their password to log in, as per normal.
This method is what is used by Lacie's storage cloud service. It's very well done.
Note: when I say "use foo to encrypt", I really mean "use foo to encrypt a secure symmetric key which is then used with a random salt to encrypt". Know your cryptography. I'm only talking about the secret, not the methodology.
None of those other solutions are going to maintain the feature set requested -- which specifically wants to preserve the user experience. If you look at the site referenced in the link, they email you a nightly past journal entry. You're not going to get that with JavaScript trickery per above because you don't have the browser to depend on. So basically this is all leading you down a path to a degraded user experience.
What you would want, or more precisely the best solution you're going to find in this space, is not so much what wuala does per above, but rather something like hush.com. The handling of user data needs to be done on the client side at all times -- this is generally accomplished via full client-side Java (like the Facebook photo uploader, etc), but HTML/JavaScript might get you there these days. JavaScript encryption is pretty poor, so you may be better off ignoring it.
OK, so now you've got client-side Java running a Journal entry encryption service. The next feature was to email past journal entries to users every night. Well, you're not going to get that in an unencrypted email obviously. This is where you're going to need to change the user experience one way or the other. The simplest solution is not to email the entry and instead to provide for instance a journal entry browser in the Java app that reminds them of some old entry once they get to the website based on a link in the daily email. A much more complex solution would be to use JavaScript encryption to decrypt the entry as an attachment inline in the email. This isn't rocket science but there is a fairly huge amount of trickery involved. This is the general path used by several web email encryption services such as IronPort. You can get a demo email by going to http://www.ironport.com/securedemo/.
As much as I'd love to see a properly encrypted version of all this, my final comment would be that journal entries are not state secrets. Given a solid privacy policy and good site security semantics, I'm sure 99% of your users will feel just fine about things. Doing all this right with true security will take an enormous amount of effort per above and at least some design/UE changes.
You should look into the MIT project CryptDB which supports querying an encrypted database using a subset of SQL. (see the forbes article, mefi thread, or Homomorphic encryption on wikipedia)
There is the Tahoe-LAFS project for cloud storage too, which conceivably could be leveraged into a fully anonymous social networking application, one day in the distant future.
If you want to perform computations on a server without even the server being able to see the data, you may be interested in knowing about fully homomorphic encryption. A fully homomorphic encryption scheme lets you perform arbitrary computations on encrypted data, even if you can't decrypt it. However, this is still a topic of research.
For now, I guess your best bet would be to encrypt all posts and assign meaningless (e.g. sequential) IDs to each one. For a more in-depth discussion of how to encrypt server-side data with today's technology, look up.
Related
I am making a social media type website, and I store user details such as emails, names and other personal details.
I will be encrypting the personal details using an Encrypt-then-MAC concept. When the user registers, a cryptographically secure string will be made to use as the private encryption key. When the user selects a password, the encryption key will be encrypted using the password.
The password will NOT be stored in the database, but will be the private key to decrypt the encryption key used to encrypt the personal details. The only person who knows the password is the user. My question is: how can I store the encryption key once decrypted?
I have thought of having a table with one column for IP and another column for the encryption key, but some people close the browser window without logging out, therefore there would not a possible way to remove the entry from the database when they have finished their session on the website.
Another way would be to store it in a cookie, but that could be intercepted when sent back to the server. I would like to know if there is a secure, nearly foolproof way to store the encryption key, client side or server side.
Thanks in advance.
EDIT:
In reply to TheGreatContini's answer -
The idea of a "zero-knowledge web application" (in your blog) is a good one, however, for zero-knowledge, even the key cannot be stored in the database, this complicates things a bit, as you would then have to use the user's password as the key. Using the password isn't as secure, as it is a bit harder to verify the password to prevent data which has been "decypted with the wrong key" from passing. There is the concept of Encrypt-then-MAC but that only verifies if the data is legit, and will assume that a hacker has messed with some data and data cannot be trusted, however, as you cannot actually verify the password (the hash would not be stored as it is "zero-knowledge"), so the password may just be wrong.
Not sure I have the answer, but a few considerations:
(1) Sessions need to be timed out. Perhaps you can do this by periodically running batch jobs that scan the database looking for sessions that have lacked activity. This requires storing in the db the date of the last action from the user.
(2) Generally keys are higher value than the content they protect because the keys have a longer lifetime than the individual data elements that the protect (because the data may change or additional data may be added). Rather than storing the key in the db, you can store the decrypted contents in the database for the length of the session. Of course, this is provided that you did (1).
Perhaps I am not adding much beyond what you already know, however may be worth considering a blog I wrote exactly about this topic. The low level details start in the section "A second line of defence for all the sensitive data." Prior to that it mainly motivates the concept. Glad to see somebody actually doing something like this.
I'm wondering whether a mechanism exists that allows client to client encryption. For example, when enabled, any information that is entered on one client can only be decrypted using a specific key.
Similar to how regular public key transactions work, but server agnostic.
A use case:
Everything on my Facebook profile is encrypted, and no body would be able to view that information (not even facebook). The users that I give the key would be able to decrypt that information.
This would allow complete control of data stored online.
The same idea can be applied for pictures uploaded to the internet.
One issue that I see is to have a practical mechanism to manage keys and a secure way to distribute keys to other users.
Has anyone done something like this before?
In case of Facebook I can imagine encrypting the data with OpenPGP keys into armored (text) format. Then you can place encrypted block to facebook or anywhere else. Other users would take the block, decrypt it on the client side and see it.
The same applies with other social networks and places where you can store some text block.
You can easily do encryption in some client application and even in Javascript (if you manage to make JavaScript load local user's keys somehow).
I've coded the mobile api using Node.js, Redis & MongoDB. However right now I am looking for a concrete way to provide security in our system. The question I have inspired by the infamous principle of Auguste Kerckhoff;
"It must not be required to be secret, and it must be able to fall into the hands of the enemy without inconvenience"
After inspecting this principle I figure out that there is no safe way to secure data after a sniffer capturing the entire data package. Of course there are alternate ways like using an API key, using encryption algorithms like MD5 & MD6,Triple DES,SHA1 etc. However this also won't work if the entire data package is captured. And there is security precaution standarts like HTTPS, SSL certificates. However again if someone with talent captures the data package it can act just like a verified user in our system.
How would it be possible to apply a security method such that even though the entire data package is captured, the system would be able to distinguish the request coming from an outer source not from our verified user.
PS: I thought that applying a custom encryption algorithm with timestamp in order to prevent this problem could be a bit chaotic.
According to Kerckhoffs's principle "A cryptosystem should be secure even if everything about the system, except the key, is public knowledge." So the way cryptosystem, works is that the key is the only thing that can be used to decipher the system. If the key is fallen to the enemy then its all over.
In practice when you communicate over the internet or try to
authenticate your email account with the password. Your password is
never sent to, nor stored on the server in plain text. If you do then,
its not secure. The best security practice is not to store the
password at all (not even encrypted), but to store the salted hash of
the encrypted password.
That is one hash for one user. It is one way, you cannot get back user info, just test if it is in the database or not. Now even if the enemy takes control of the database, it cannot access your username/passwords.
Now coming to the point, it does not matter what you transmit in the communication channel, because the communication channel is the enemy!!! It is available to other users, anyone can sniff it. It is like enemies scanning each other on the radio.
The hash that has been sent over the channel can be sniffed and be used for authentication. True, but server can differentiate between spoofed attempt and the actual attempt, using HTTPS connection. Server tracks the HTTPS sessions and would ask to revalidate if something like this happens. HTTPS blocks use of sniffed data / MITM attacks. Even if sniffer gets hash (temporary ticket), he cannot do anything malicious, username and password cannot be cracked.
Thanks to Kerckhoff, our passwords are safe.
How to implement it on node.js ?? look for passport.js package. It implements the current standard OpenAuth.
I need to make a log-in system and having basically no previous knowledge of how it's done (with security in mind) I studied it on the internet. The way I would do it now is something like this:
Server has login information in a database table - username and a password hash per user (encrypted with SHA224 for example).
When client wants to authenticate, password is encrypted with SHA224 (client-side) and sent with username to the server to verify a match in the database.
If the user ticked "Remember me" option, an authentication key is generated on the server, inserted into a database along with the IP of the client.
The authentication key is sent to the client and stored in cookies.
Now, when the client returns, authentication key from cookies is sent to the server, the server finds it in the database and checks if the IPs match as well. If it does, the user is authenticated and a new authentication key is generated and sent to the user (and stored in cookies) for next visit.
My questions are:
How does encrypting password make this any safer? The hash still can be captured on the way from client to server and misused just as well as if it was plaintext. I know that this is an elementary question but I somehow couldn't find an answer to this one.
Is this security system secure enough? (or better yet - Did I get it right?)
Why does hashing a password make the system more secure
Hashing is not equal to encryption. Encrypted data can be decrypted back into plain text. Hashed data cannot be decrypted.
By hashing your user's passwords, nobody can see what passwords are used. So if your data gets stolen, the hashes cannot be decrypted by the hacker. The same goes for the system administrator, he/she cannot 'lookup' a password. This can be an all to common scenario in shared hosting environments.
Storing passwords
The easiest way to get your password storage scheme secure is by using a standard library.
Because security tends to be a lot more complicated and with more invisible screw up possibilities than most programmers could tackle alone, using a standard library is almost always easiest and most secure (if not the only) available option.
The good thing is that you do not need to worry about the details, those details have been programmed by people with experience and reviewed by many folks on the internet.
For more information on password storage schemes, read Jeff`s blog post: You're Probably Storing Passwords Incorrectly
Whatever you do if you go for the 'I'll do it myself, thank you' approach, do not use MD5 anymore. It is a nice hashing algorithm, but broken for security purposes.
Currently, using crypt, with CRYPT_BLOWFISH is the best practice.
From my answer to: Help me make my password storage safe
As for the infamous remember me option.
Create a random token and give it to the user in the form of a cookie.
If the user presents a cookie with this token, you give them access. Key is to only accept each token once. So after it is used, replace it with a new random token.
This token is, in essence, just another password. So in order to keep it safe, you do not store the token, but a hash of it. (just as you did with the password)
Your suggestion of binding the cookie to an IP-address will unfortunately not work. Many people have dynamic IP-addresses, some even change from request to request during a single session. (this is for example caused by load-balancing proxies).
Sending passwords to the server
The only method currently usable for sending a password from a web browser to server is by using a SSL-secured connection. Anything else will not be safe, as you cannot guarantee the integrity of the solution on the client side.
Some points I want to add:
the hashing of the password is not done on the client. You cannot do it reliably. The necessary technique for computing the hash (JavaScript in your case) might not be available and you cannot trust the result. If somebody can retrieve the hashes of the passwords in your database he could just login without knowing the actual passwords.
make sure to use SSL or another secure transport for transmitting the given passwords from the client to the server. SSL is a good idea for everything after all.
you should not use a single hash algorithm for storing the passwords in the database. Have a look at HMAC. That is far better. Additionally read about salts in cryptography.
Never ever invent your own crypto
mechanisms. Use someone else's.
Crypto is beyond tricky, and unless
you're Bruce Schneier, you have an
extremely slim chance of improving
it, while having a huge chance of
screwing it royaly.
Do not encrypt passwords, hash them.
If you're using hashes, salt them.
If you don't have
to use straight hashes, use HMAC,
they're much more resistant to
precalculated attacks.
If you're
sending stuff across an unsecure
link, add a NONCE to the transmission
to prevent replay attacks. This goes
for both client->server and
server->client.
If you're using salts and nonces, make sure they have high entropy. 'secret' is not a good one. Make it random, make it long, make it use large character sets. The extra computation cost is minimal, but the security you gain from it is enormous. If you're not sure how, use a random password generator, and then use ent to measure entropy.
Do NOT use a
timestamp as a nonce, unless you have
a very specific need and really know
what you're doing.
Use session
protection. SSL isn't perfect but
it's helluva better than nothing.
If you're using SSL, make sure to disable weak protocols. SSL session starts with 'offerings' of lists of ciphers both sides can do. If you let clients use a weak one, an attacker will definitely use that.
I have an application where I need to store 3rd party credentials to services like Amazon S3, FTP, SFTP, etc..
I know that it is possible to access some of those systems without passwords, but that has its own issues. If our customers gave us access to their S3 buckets via ACL we would still need to verify which bucket belongs to which user, same goes for SFTP and ssh key auth.
We will try our best to allow non-password alternatives where possible, but sometimes (FTP) it just won't be possible. Therefor I am looking for advice on how to store this sensitive data in our database (MySql) or elsewhere.
In the past I have read about people using TrueCrypt partitions that automatically unmount, but that would probably require decent intrusion detection. For now I'm interested in simple approaches that lead to reasonable security and can improved upon in the future.
Any advice on the subject would be highly appriciated!
There are a range variety of possibilities and since in my opinion you provide not enough info about the context, i will try to give you an overview from my point of view. I assume that here the most important aspect is confidentiality of your data and and authentication of the users. Integrity and availability of data is much less important.
If you want basic security, you can let MySQL handle it by means of username/password combinations and set access rights on the given account. However, since the access control mechanism of mysql is not fine-grained (you can set access control rules per table only, not per row) this will probably yield a bad database design.
If you want to have a non-password approach, you can give users client-certificates and let them prove their identity by presenting their client certificates (use TLS for that) or let them sign something (note their are dangers because you create a so called signing oracle).
Another approach is to encrypt your data in the database. You can do that by deriving a symmetric key from the password and encrypt the credentials with this data. The catch here is of course that your key derivation protocol should be good and this is not easy to accomplish (so if you choose this, i advice you to take existing key derivation protocols or use a streamcipher). Take a look here for a list of streamcipher http://en.wikipedia.org/wiki/Stream_cipher .
If you care very much for security you can start thinking about fancy solutions like authentication with smartcards, or a time synchronized tamper resistant device for generating acccess codes. However, note that these fancy solutions do not give you free security, implementing such systems if hard and costly (due to development and deployment) however, if done correctly they provide the best security.
Have the user supply a (strong) password when they set up an account (before they provide their passwords). Then encrypt all data for that account within your database using a key derived from a strong hash (SHA256 or something like that) of the user's password. That way if your servers get compromised, no data will be revealed because it is encrypted with the user's password (well, a hash of the user's password) and that password is not stored anywhere on your server.
You need to investigate the use of keystores. TruCrypt is an example of such a keystore, but this is a personal keystore, not intended for service level credentials.
You won't be able to avoid storing their passwords in a format that someone can get access to, the goal is to minimize who can access the information. Putting in the same MySQL as application data is asking for disaster.