I have to handle some sensitive data in my application, such as passwords, credit card information, etc.
What are possible security risks I could have and how can I avoid them?
Don't store Credit Card Information (in some jurisdictions, you might be breaking the law by doing so, or at least falling foul of a commercial agreement)
You don't say where your sensitive data is stored, but encypting it is the usual approach. There are two forms symmetric and asymmetric. Symmetric means you use the same key for encrypting and decrypting. Asymmetric consists of a public/private key pair.
Passwords: store only a salted hash (i.e. un-reversible) of your passwords, and compare with a similarly salted hash of an entered password.
Be aware that you really shouldn't store credit card info in any shape or form on a web server.
Bit of info on doing this in a web environment, which is my background:
If a website (or any application) needs to store card info, it should comply with the PCI DSS (Payment Cards Industry Data Security Standard). Amongst other things, it requires that the data be held encrypted on a separate server that isn't publicly accessible (IE: isn't hosting the site) and has a separate firewall between it and the webserver. Penalties for not complying are potentially very large in the event of any fraudulent activity following a security breach, and can include them ceasing working with you - it pretty much reserves the right for the them to chargeback any losses from the fraud to you (from my interpretation as a non legal person)
More on it here: https://www.pcisecuritystandards.org/security_standards/pci_dss.shtml
Obviously, this may be expensive compared to shared hosting, as you immediately need two servers and a load of networking gear. Which is why people don't often do this.
I would be inclined to perform some form of reversible encryption on the information as it's being stored, something like:
$card = myEncryptionFunction($input);
A little more information on the nature of your application wouldn't hurt though.
I'd be using reversible encryption on the database data. Make sure this data doesn't seep into log-files too, log derived information instead. Consider how yoǘ'll handle different environments - normally you want to not use production data in your test environments. So even though you may consider copying production data back to test systems, you should probably generate fake data for the sensitive parts.
It's been already said that you shouldn't store CC especially CVV2 information in your database, avoid where possible.
If you store CC + CVV2 then consider using asymmetric encryption and store your private key in another server. Otherwise an attacker who can access the data 99% can access the key and the whole encryption would be pointless.
Passwords should be stored as one way hashed.
After all these you need to ensure that your application is secure against vulnerabilities such as SQL Injeciton, remote code execution etc.
Don't forget Even when an attacker can't read previous data they can plant a backdoor for the next data.
Related
I'd like to use Windows.Security.Credentials.PasswordVault in my desktop app (WPF-based) to securely store a user's password. I managed to access this Windows 10 API using this MSDN article.
I did some experiments and it appears that any data written to PasswordVault from one desktop app (not a native UWP app) can be read from any other desktop app. Even packaging my desktop app with Desktop Bridge technology and thus having a Package Identity does not fix this vulnerability.
Any ideas how to fix that and be able storing the app's data secure from other apps?
UPDATE: It appeared that PasswordVault adds no extra security over DPAPI. The case is closed with a negative result.
(this is from what I can understand of your post)
There is no real way of preventing data access between desktop apps when using these kind of API's http://www.hanselman.com/blog/SavingAndRetrievingBrowserAndOtherPasswords.aspx tells more about it. You'd probably just want to decrypt your information.
memory access restriction is difficult, code executed by the user is always retrievable by the user so it would be difficult to restrict this.
have you considered using the Windows Data Protection API :
https://msdn.microsoft.com/en-us/library/ms995355.aspx
grabbed straight from the source
DPAPI is an easy-to-use service that will benefit developers who must provide protection for sensitive application data, such as passwords and private keys
WDPAPI uses keys generated by the operating system and Triple DES to encrypt/decrypt your data. Which means your application doesn't have to generate these keys, which is always nice.
You could also use the Rfc2898DeriveBytes class, this uses a pseudo-random number generator to decrypt your password. It's safer than most decrypters since there is no practical way to go back from the result back to the password. This is only really useful for verifying the input password and not retrieving it back again. I have never actually used this myself so I would not be able to help you.
https://msdn.microsoft.com/en-us/library/system.security.cryptography.rfc2898derivebytes(v=vs.110).aspx
see also this post which gives a way better explanation than I can.
How to securely save username/password (local)?
If I misunderstood the question in some way, tell me, I will try to update the answer.
NOTE that modern/metro apps do not have this problem, although they still are accessible in other ways.
The hard truth is that storing a password in a desktop application, 100% securely is simply not possible. However, you can get close to 100%.
Regarding your original approach, PasswordVault uses the Credential Locker service which is built into windows to securely store data. Credential Locker is bound to the user's profile. Therefore, storing your data via PasswordVault is essentially equivalent to the master password approach to protecting data, which I talk about in detail further down. Only difference is that the master password in that case is the user's credentials. This allows applications running during the user's session to access the data.
Note: To be clear, I'm strictly talking about storing it in a way that allows you access to the plain text. That is to say, storing it in an encrypted database of any sort, or encrypting it yourself and storing the ciphertext somewhere. This kind of functionality is necessary in programs like password managers, but not in programs that just require some sort of authentication. If this is not a necessity then I strongly recommend hashing the password, ideally per the instructions laid out in this answer by zaph. (Some more information in this excellent post by Thomas Pornin).
If it is a necessity, things get a bit more complicated: If you want to prevent other programs (or users I suppose) from being able to view the plaintext password, then your only real option is to encrypt it. Storing the ciphertext within PasswordVault is optional since, if you use good encryption, your only weak point is someone discovering your key. Therefore the ciphertext itself can be stored anywhere. That brings us to the key itself.
Depending on how many passwords you're actually trying to store for each program instance, you might not have to worry about generating and securely storing a key at all. If you want to store multiple passwords, then you can simply ask the user to input one master password, perform some salting and hashing on that, and use the result as the encryption key for all other passwords. When it is time for decryption, then ask the user to input it again. If you are storing multiple passwords then I strongly urge you to go with this approach. It is the most secure approach possible. For the rest of my post however, I will roll with the assumption that this is not a viable option.
First off I urge you not to have the same key for every installation. Create a new one for every instance of your program, based on securely generated random data. Resist the temptation to "avoid having to store the key" by having it be generated on the fly every time it is needed, based on information about the system. That is just as secure as hardcoding string superSecretKey = "12345"; into your program. It won't take attackers long to figure out the process.
Now, storing it is the real tricky part. A general rule of infosec is the following:
Nothing is secure once you have physical access
So, ideally, nobody would. Storing the encryption keys on a properly secured remote server minimizes the chances of it being recovered by attackers. Entire books have been written regarding server-side security, so I will not discuss this here.
Another good option is to use an HSM (Hardware Security Module). These nifty little devices are built for the job. Accessing the keys stored in an HSM is pretty much impossible. However, this option is only viable if you know for sure that every user's computer has one of these, such as in an enterprise environment.
.Net provides a solution of sorts, via the configuration system. You can store your key in an encrypted section of your app.config. This is often used for protecting connection strings. There are plenty of resources out there on how to do this. I recommend this fantastic blog post, which will tell you most of what you need to know.
The reason I said earlier not to go with simply generating the key on the fly is because, like storing it as a variable in your code, you rely exclusively on obfuscation to keep it secure. The thing about this approach is that it usually doesn't. However, sometimes you have no other option. Enter White Box cryptography.
White box cryptography is essentially obfuscation taken to the extreme. It is meant to be effective even in a white-box scenario, where the attacker both has access to and can modify the bytecode. It is the epitome of security through obscurity. As opposed to mere constant hiding (infosec speak for the string superSecretKey approach) or generating the key when it is needed, white box cryptography essentially relies on generating the cipher itself on the fly.
Entire papers have been written on it, It is difficult to pull off writing a proper implementation, and your mileage may vary. You should only consider this if you really really really want to do this as securely as possible.
Obfuscation however is still obfuscation. All it can really do is slow the attackers down. The final solution I have to offer might seem backwards, but it works: Do not hide the encryption key digitally. Hide it physically. Have the user insert a usb drive when it is time for encryption, (securely) generate a random key, then write it to the usb drive. Then, whenever it is time for decryption, the user only has to put the drive back in, and your program reads the key off that.
This is a bit similar to the master password approach, in that it leaves it up to the user to keep the key safe. However, it has some notable advantages. For instance, this approach allows for a massive encryption key. A key that can fit in a mere 1 megabyte file can take literally billions of years to break via a brute force attack. Plus, if the key ever gets discovered, the user has only themselves to blame.
In summary, see if you can avoid having to store an encryption key. If you can't, avoid storing it locally at all costs. Otherwise, your only option is to make it as hard for hackers to figure it out as possible. No matter how you choose to do that, make sure that every key is different, so even if attackers do find one, the other users' keys are safe.
Only alternative is to encrypt password with your own private key stored somewhere in your code. (Someone can easily disassemble your code and get the key) and then store encrypted password inside PasswordVault, however the only security you have is any app will not have access to password.
This is dual security, in case of compromised machines, attacker can get access to PasswordVault but not your password as they will need one more private key to decrypt the password and that will be hidden somewhere in your code.
To make it more secure, if you leave your private key on your server and expose an API to encrypt and decrypt password before storing in Vault, will make it most secure. I think this is the reason people have moved on to OAuth (storing OAuth token in PasswordVault) etc rather then storing password in vault.
Ideally, I would recommend not storing password, instead get some token from server and save it and use that token for authentication. And store that token in PasswordVault.
It is always possible to push the security, with miscellaneous encryption and storage strategies. Making something harder is only making the data retrieval longer, never impossible. Hence you need to consider the most appropriate level of protection considering execution cost x time (human and machine) and development cost x time aspects.
If I consider strictly your request, I would simply add a layer (class, interface) to cipher your passwords. Best with asymmetrical encryption (and not RSA). Supposing the other softs are not accessing your program data (program, files OR process), this is sufficient. You can use SSH.NET (https://github.com/sshnet/SSH.NET) to achieve this quickly.
If you would like to push the security and give a certain level of protection against binary reverse-engineering (including the private key retrieval), I recommend a small (process limited) encrypted VM (like Docker, https://blogs.msdn.microsoft.com/mvpawardprogram/2015/12/15/getting-started-with-net-and-docker/) based solution such as Denuvo (https://www.denuvo.com/). The encryption is unique per customer and machine based. You'll have to encapsulated you c# program into a c/c++ program (which acts like a container) that will do all the in-memory ciphering-deciphering.
You can implement your own strategy, depending on the kind of investment and warranty you require.
In case your program is a backend program, you can pick the best strategy (the only I really recommend) of all which is to store the private key at the client side, public key at backend side and have local deciphering, all transmitted password would be hence encrypted. I would like to remark that password and keys are actually different strategies to achieve the same goal: checking if the program talks to the right person without knowing the person's identity; I mean this: instead of storing passwords, better store directly public keys.
Revisiting this rather helpful issue and adding a bit of additional information which might be helpful.
My task was to extend a Win32 application that uses passwords to authenticate with an online service with a "save password" functionality. The idea was to protect the password using Windows Hello (UserConsentVerifier). I was under the impression that Windows surely has something comparable to the macOS keychain.
If you use the Windows Credential Manager APIs (CredReadA, CredWriteA), another application can simply enumerate the credentials and if it knows what to look for (the target name), it will be able to read the credential.
I also explored using DPAPI where you are in charge of storing the encrypted blob yourself, typically in a file. Again, there seems to be no way (except obfuscation) to prevent another application from finding and reading that file. Supplying additional entropy to CryptProtectData and CryptUnprotectData again poses the question of where to store the entropy (typically I assume it would be hard-coded and perhaps obfuscated in the application: this is security by obscurity).
As it turns out, neither DPAPI (CryptProtectData, CryptUnprotectData) nor Windows Credential Manager APIs (CredRead, CredWrite) can prevent another application running under the same user from reading a secret.
What I was actually looking for was something like the macOS keychain, which allows applications to store secrets, define ACLs on those secrets, enforce biometric authentication on accessing the secret, and critically, prevents other applications from reading the secrets.
As it turns out, Windows has a PasswordVault which claims to isolate apps from each other, but its only available to UWP apps:
Represents a Credential Locker of credentials. The contents of the locker are specific to the app or service. Apps and services don't have access to credentials associated with other apps or services.
Is there a way for a Win32 Desktop application to access this functionality? I realize that if a user can be brought to install and run a random app, that app could probably mimic the original application and just prompt the user to enter the secret, but still, it's a little disappointing that there is no app-level separation by default.
I have a (university) project where I basically write and read text out of an NFC tag with Android devices in order to store one's balance in the card (which can be used on the cafeteria, for example).
Right now, I'm using NTAG213 doing the below code:
ndef.connect();
NdefRecord mimeRecord = NdefRecord.createMime("text/plain", messageEncrypted.getBytes(Charset.forName("US-ASCII")));
ndef.writeNdefMessage(new NdefMessage(mimeRecord));
ndef.close();
As you can notice, I'm using application level encryption to encrypt the message (messageEncrypted) before writing it to the tag (AES-256 encrypt with 'com.scottyab:aescrypt:0.0.1' library - with a very big password key that uses also the tag UID as part of it).
So far so good - only I can understand the data on the tag.
On my research, I've found that when it comes to security Ultralight C > NTAG213.
Question 1) When using application level encryption, why (is it?) is MIFARE Ultralight C safer then NTAG213?
Question 2) I'm pretty sure I can guarantee security using AES encryption, but I don't want people (besides me) messing with the stored data (formatting tag or writing info there). I see that the only way of preventing that (please, correct me if I'm wrong) is to set a password for the tag. However, both NTAG213 and Ultralight C have only an 32-bit password. Is it good enough? Is there another way of preventing someone (besides me) writing data?
Question 3) Which other security measures can I use on such tags to enforce security (tag and application layer)?
Question 4) When you compare tag security (MIFARE DESFire > Ultralight > NTAG213 > MIFARE Classic), what is really being compared? The ease of one cracking the (native tag's) encryption or the ease of one store (anything) on the tag without permission?
Question 5) I see a bunch of other techs (MIFARE DESFire, ICODE SLIX, Infineon Cipurse) that are more secure, which makes me wonder if the tech I'm using (NTAG213 or Ultralight C) is good enough for storing someones balance. Would you (and that's a personal opinion) say that NTAG213 with application level encryption and 32-bit password good enough for this type of application? And how long would it take someone to actually break its security?
When using application level encryption, why is Ultralight C safer then NTAG213? Is that even true?
First of all, "safer" depends on what your actual protection goals are. Since you want to store a balance (cash money!) on the card, you would probably want to (at least) protect towards the following goals:
Users must not be able to print their own money by setting an arbitrary balance on their card.
Users must not be able to duplicate their card and, consequently, their money balance.
Users must not be able to print their own money by restoring (roll-back) their card to a previous balance after payment.
Users must not be able to print their own money by replaying a top-up procedure.
Users must not be able to evade payment by tearing their card during a payment transaction.
Users must not be able to generate an arbitrary (and potentially higher) balance on their card by tearing their card during a top-up procedure.
In addition, you might not want to trust operators (the persons accepting payments and performing top-up) as well. In a system where one group of operators performs only top-up and the other performs only payment transactions, the latter group should probably not be allowed to ever "create" money. Particularly, you have to make yourself very clear about whether you fully trust the (Android) devices that you use in the field to perform these operations and whether you trust operators (e.g. that they don't perform any attacks against these devices).
Moreover, there might be privacy aspects that you need to consider (e.g. if the balance is freely readable, if users are identifiable, etc.)
So let's look into what you "application level encryption" adds in terms of security:
Since users do not know the encryption key, they are probably not able to generate an arbitrary balance on their card. However, this heavily depends on the format of your balance (in unencrypted form). Users can make arbitrary modifications to the cipher text with result in "random" modifications of the plain text. Consequently, users may be able to modify the balance value despite encryption. Digital signature/message authentication codes are they path that you would probably want to take to overcome this.
Since the encryption key (assuming encryption would be sufficient, which it probably is not) depends on the UID of the tag, you may be safe against cloning of cards (+ balance). However, be aware that the UID is just a freely readable identifier. It is by no means authenticated itself and may be clonable as well. See Serials on NFC Tags - truly unique? cloneable?.
The encrypted value does not protect you from users restoring their balance to a previously recorded value after payment. This type of vulnerability has been found before (particularly in MIFARE Ultralight based systems), see, for instance, Benninger, C., Sobell, M. (2012): NFC for free rides and rooms (on your phone). In: Presentation at EUSecWest 2012.
Since you write the complete value during a top-up procedure (i.e. there is no specific "increment balance" command), you are probably safe against users replaying a top-up (except for the roll-back aspect of this).
Effects of tearing are probably rather limited if your system only allows for attended payment/top-up.
So let's see what additional features NTAG213 would have that you could use to secure your system:
UID is unique on genuine tags. This does not help much, see Serials on NFC Tags - truly unique? cloneable?.
Originality signature: same as above, the originality signature is also just a static, freely readable value. Consequently, it's just as well susceptible to cloning.
The one-way counter might be a tool to help you in protecting against roll-back (by including the counter value into the signature). This still would not prevent cloning onto a tag platform that allows to generate arbitrary counter values. Moreover, the counter is not easily controllable and will change its value if the user tries to read the tag. Consequently, it's questionable if an implementation based on that value would be reliable.
Unlike MIFARE Ultralight, NTAG213 does not have a usable one-time programmable area (since that's already used by the capability container). Consequently, you can't implement a one-time deductible balance based on that.
The password protection feature could help you in authenticating tags (by performing password verification) and in protecting the value stored on the tag (by making the value only readable/writable after password verification). However, the password is transmitted in clear text (may be subject to sniffing, particularly in (but not limited to) unattended scenarios) and there is no cryptographic binding between the password and the actual read/write.
MIFARE Ultralight C would add the following:
The OTP bytes can be used. If it's an option to make the tags one-time usable (i.e. they start with a specific balance that can only be deducted from and not topped-up), then using the OTP bytes to represent the balance would be an option. Note that there's still lots of things that you could do wrong with that, e.g. Beccaro, M., Collura, M. (2013): OTP circumventing in MIFARE ULTRALIGHT: Who says free rides?. In: Presentation at DEFCON 21
The authentication is much improved. The 3DES authentication scheme seems to be sufficiently secure to prevent sniffing the key. However, read/write commands are also not cryptographically bound to the authentication step. Consequently, an attacker might be able to let a genuine payment terminal + genuine tag perform authentication, but redirect read/write to somewhere else. This might (particularly) be a problem in an unattended scenario.
I'm pretty sure I can guarantee security using AES encryption.
See above. This is probably not true.
I don't want people messing with the stored data. I see that the only way of preventing that is to set a password for the tag.
A password/authentication key may help, but be aware of the limitations due to authentication being decoupled from read/write on these tag platforms.
Both NTAG213 and Ultralight C have only a 32-bit password.
This is not true. NTAG213 has a 32-bit password. MIFARE Ultralight C uses a more sophisticated mutual 2K-3DES authentication mechanism with a 112-bit key.
When you compare tag security, what is really being compared?
Authentication mechanisms (algorithms, key sizes)
Communication security (e.g. is the communication itself encrypted/authenticated using a session key derived from the authentication step?)
Access control (e.g. are there separate keys for top-up and payment?)
Are there dedicated mechanisms for balance management (e.g. value fields, dedicated increment/decrement operations)? And, consequently, are ther mechanisms to protect agains tearing attacks?
And probably more ...
I see a bunch of other techs that are more secure, which makes me wonder if the tech I'm using is good enough for storing someones balance.
You specific system is flawed in many ways. In my opinion MIFARE Ultralight/NTAG203/NTAG21x are definitely not a good choice for a an offline system storing cash money on the cards.
MIFARE Ultralight C may be suitable with some precautions. I would definitely refrain from using that in unattended scenarios, and I would probably use an online system tracking the balance and monitoring for inconsistencies.
Anything that uses symmetric cryptography and storing the cryptographic keys in a terminal would certainly require precautions against malicious operators. It's probably fairly easy for an operator (with some knowledge) to extract keys from an app and generate their own money.
I guess your question is too broad and not for all subquestions this section of SO is the most appropriate one.
By focusing on cryptographic strength you miss something: if the low level security of the token can be easily attacked, nobody needs to crack your key.
A simple dump and later restore (after some payments) corresponds to printing money
If the token contains the money directly (instead of only identifying the wallet, which is stored on a background system), you need a far more secure system to avoid financial damage. This involves dynamically encrypted communication but continues with substantial further topics.
I am considering encryption options for a new Sybase project. I am thinking that Sybase encryption is the wrong strategy because a) dba's can get in, and b) if and when we migrate to SQL Server or Oracle I don't want to deal with different encryption strategies.
Therefore I'm thinking to encrypt the sensitive data (symmetric encryption) in my Java code before storing it in the DB.
Now, the encrypted fields better not have their encryption key changed, ever, except in a very controlled environment, which for me effectively means never. So it's going to be a permanent password.
The question is, where should I keep this password in a way that it is accessible from the program but not accessible to anyone else. If it's in a properties file, any developer with access to our Git repo could see it.
We could hard code it in the source code, but good lawd, that's a bad practice.
We could generate it in source, like the 10th Fibonacci or 3!+8! that would be hard to locate, but it's still rather exposed.
We could have the sa's maintain it in the environment, but then where do they file it for future reference?
So many poor choices. Are there any good ones?
Simply using some secret code to create the key on the fly is both an insecure method and produces a poor key. The DB keep needs to be a random byte array. Keep in mind that the key needs to be in memory when used which will be most of the time for the DB.
WRT using the DB encryption, examine closely if the algorithm is fully specified and compatibility to another DB. There is also the possibility that the entire DB will need to be run-off and then added to a new DB, in that case using the internal DB encryption will be transparent.
You really need to consider needing to be able to change the encryption key in the future, what will you do if it is ever compromised? There are solutions to this dilemma. There may be a substantial performance penalty performing the encryption outside the DB, there is a substantial setup time for each new encryption operation. Also since not all columns will be encrypted (a good guess) that information is not shared by the DB and the outside encryption code, that coupling is not good for design nor maintenance.
Do not connect the DB server to the Internet, make it separate and connected with a non-networked connection such as direct Ethernet. This also limited the number of admin users of the only system that contains the encryption key.
Another important part of the solution is to restrict admin access to the server. This includes requiring two-factor authentication as well as severely limiting the number of administrators. You need to control the second-factor to physical serial-numbered devices owned by the organization so that they can be positively retrieved on personnel changes and not copied. Personally I favor RSA SecureID (or similar) hardware devices, there is positive control.
Finally in answer to the question, keep the key in a file on the DB server secured as above, that is with no Internet access and restricted admin access.
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
PHP 2-way encryption: I need to store passwords that can be retrieved
I know that the best practice for storing user passwords is to store only an irreversible hash of the password.
However, I am developing an application where I will need to store a user's login information for another web service -- I'll need to periodically log them in and perform some maintenance tasks. Unfortunately, the service doesn't offer authorization tokens so I (very apprehensively) have to store the passwords in a way that I can access their plain-text values. I don't own or control the service to which I am authenticating, and the only method is to 'borrow' a users username and password and authenticate.
I am planning to AES_ENCRYPT the passwords in the DB, which means that if somebody is somehow able to access the DB they won't be able to get the plaintext. However my code will need to have access to the key to unencrypt them, thus if the entire server is compromised this is no protection and the passwords will be revealed.
Aside from the above-described encryption, are there any best practices or steps I can take to do this as safely as possible?
EDIT
I know that whatever I do, ultimately the passwords must be accessible in plaintext and so a compromised server means the passwords will be revealed, but I am wondering what steps I can do to mitigate my risk. E.G. encrypting the DB protects me in the situation where the DB is compromised but not the entire server. Other similar mitigating steps would be much appreciated.
However, I am developing an application where I will need to store a user's login information for another web service -- I'll need to periodically log them in and perform some maintenance tasks.
OK... I read through the answers and the comments, and about all I can say is I hope you have crack legal team. It sounds to me like the service you are offering is predicated on user trust. It's good that it's a user-controlled switch, and not something being helpfully done behind their backs, but I think you want a really iron clad service agreement on this.
That said, there's a lot of security paranoia you can invoke. You'll have to figure out how much you want to go through based on the harm to your product, your company and users if a break in occurs. Here's thoughts:
Data storage - store the passwords far away from where an attacker can get in. Highly access controlled files, a database on a back end machine, etc. Make any attacker have go to through layers of defense just to get to the place the data is stored. Similarly have network protection like firewalls and up to date security patches. No one thing works in isolation here.
Encryption - any encryption technique is a delaying tactic - one the attacker has your data, they will eventually crack your encryption given an infinite amount of time. So mostly you're aiming to slow them down long enough for the rest of the system to discover you've been hacked, alert your users, and give the users time to change passwords or disable accounts. IMO - either symmetric or assymetric cryptography will work - so long as you store the key securely. Being a PKI person myself, I'd lean towards assymmetric crypto just because I understand it better and know of a number of COTS hardware solutions that make it possible to store my private key extremely securely.
Key storage - your encryption is only as good as your key storage. If the key is sitting right next to the encrypted data, then it stands to reason that the attacker doesn't need to break your crypto, they just use the key. HSM (hardware security modules) are the high end choice for key storage - at the upper ranges these are secure boxes that are tamper proof which both hold your keys and perform crypto for you. At the low end, a USB token or Smart Card could perform the same function. A critical part of this is that ultimately, it's best if you make an admin activate key access on server startup. Otherwise, you end up with a chicken and egg scenario as you try to figure out how to securely store the ultimate password.
Intrusion detection - have a good system in place that has a good chance of raising alarms if you should get hacked. If your password data is compromised, you want to get the word to your users well ahead of any threat.
Audit logging - have really good records of who did what on the system - particularly in the vicinity of your passwords. While you could create a pretty awesome system, the threat of privileged users doing something bad (or dumb) is just as bad as external threats. The typical high end auditing systems track high privilege user behavior in a way that can't be viewed or tampered with by the high privilege user - instead, there's a second "auditor" account that deals only with audit logs and nothing else.
This is a highlight of the high points of system security. My general point is - if you are serious about protecting user passwords, you can't afford to just think about the data. Just encrypting the passwords is not likely to be enough to really protect users and safeguard trust.
The standard way to approach this is to consider the cost of explotation vs. the cost of protection. If both costs are too high for the value of the feature, then you have a good indication that you shouldn't bother doing it...
As you said, your code will eventually need the key and so if the server is compromised, so will be the passwords. There is no way around it.
What you can do is have a very minimal proxy whose only job will be to have the passwords, listen to the requests from your main application, connect to the service in question, and return the response to your application. If that very simple proxy is all that is running on a server then it will be much less likely to be compromised than a complicated application running on a server with many services.
I'm already using salted hashing to store passwords in my database, which means that I should be immune to rainbow table attacks.
I had a thought, though: what if someone does get hold of my database? It contains the users' email addresses. I can't really hash these, because I'll be using them to send notification emails, etc..
Should I encrypt them?
Bruce Schneier has a good response to this kind of problem.
Cryptography is not the solution to your security problems. It might be part of the solution, or it might be part of the problem. In many situations, cryptography starts out by making the problem worse, and it isn't at all clear that using cryptography is an improvement.
Essentially encrypting your emails in the database 'just in case' is not really making the database more secure. Where are the keys stored for the database? What file permissions are used for these keys? Is the database accesable publically? Why? What kind of account restrictions are in place for these accounts? Where is the machine stored, who has physical access to this box? What about remote login/ssh access etc. etc. etc.
So I guess you can encrypt the emails if you want, but if that is the extent of the security of the system then it really isn't doing much, and would actually make the job of maintaining the database harder.
Of course this could be part of an extensive security policy for your system - if so then great!
I'm not saying that it is a bad idea - But why have a lock on the door from Deadlocks'R'us which cost $5000 when they can cut through the plywood around the door? Or come in through the window which you left open? Or even worse they find the key which was left under the doormat. Security of a system is only as good as the weakest link. If they have root access then they can pretty much do what they want.
Steve Morgan makes a good point that even if they cannot understand the email addresses, they can still do a lot of harm (which could be mitigated if they only had SELECT access)
Its also important to know what your reasons are for storing the email address at all. I might have gone a bit overboard with this answer, but my point is do you really need to store an email address for an account? The most secure data is data that doesn't exist.
I realize this is a dead topic, but I agree with Arjan's logic behind this. There are a few things I'd like to point out:
Someone can retrieve data from your database without retrieving your source code (i.e. SQL injection, third-party db's). With this in mind, it's reasonable to consider using an encryption with a key. Albeit, this is only an added measure of security, not the security...this is for someone who wants to keep the email more private than plaintext,
In the off chance something is overlooked during an update, or an attacker manages to retrieve the emails.
IMO: If you plan on encrypting an email, store a salted hash of it as well. Then you can use the hash for validation, and spare the overhead of constantly using encryption to find a massive string of data. Then have a separate private function to retrieve and decrypt your emails when you need to use one.
In common with most security requirements, you need to understand the level of threat.
What damage can be done if the email addresses are compromised?
What's the chance of it happening?
The damage done if email addresses are REPLACED could be much greater than if they're EXPOSED. Especially if you're, for example, using the email address to verify password resets to a secure system.
The chance of the passwords being either replaced or exposed is much reduced if you hash them, but it depends what other controls you have in place.
I would say it depends on the application of your database.
The biggest problem is, where do you store the encryption key? Because if the hacker has excess to anything more than your DB, all your efforts are probably wasted. (Remember, your application will need that encryption key to decrypt and encrypt so eventually the hacker will find the encryption key and used encryption scheme).
Pro:
A leak of your DB only will not expose the e-mail addresses.
Cons:
Encryption means performance loss.
Allot of database actions will be harder if not impossible.
Don't accidentally conflate encryption with obfuscation. We commonly obfuscate emails to prevent spam. Lots of web sites will have "webmaster _at_ mysite.com" to slow down crawlers from parsing the email address as a potential spam target. That should be done in the HTML templates -- there's no value to doing this in persistent database storage.
We don't encrypt anything unless we need to keep it secret during transmission. When and where will your data being transmitted?
The SQL statements are transmitted from client to server; is that on the same box or over a secure connection?
If your server is compromised, you have an unintentional transmission. If you're worried about this, then you should perhaps be securing your server. You have external threats as well as internal threats. Are ALL users (external and internal) properly authenticated and authorized?
During backups you have an intentional transmission to backup media; is this done using a secure backup strategy that encrypts as it goes?
Both SQL Server and Oracle (and I believe also others DBs) support encryption of data at the database level. If you want to encrypt something why don't simply abstract the access to the data that could be encrypted on the database server side and left the user choose if use the encrypted data (in this case the SQL command will be different) or not. If the user want to user encrypted data then it can configure the database server and all the maintenance work connected with key management is made using standard DBA tool, made from the DB vendor and not from you.
A copy of my answer at What is the best and safest way to store user email addresses in the database?, just for the sake of the search...
In general I agree with others saying it's not worth the effort. However, I disagree that anyone who can access your database can probably also get your keys. That's certainly not true for SQL Injection, and may not be true for backup copies that are somehow lost or forgotten about. And I feel an email address is a personal detail, so I wouldn't care about spam but about the personal consequences when the addresses are revealed.
Of course, when you're afraid of SQL Injection then you should make sure such injection is prohibited. And backup copies should be encrypted themselves.
Still, for some online communities the members might definitely not want others to know that they are a member (like related to mental healthcare, financial help, medical and sexual advice, adult entertainment, politics, ...). In those cases, storing as few personal details as possible and encrypting those that are required (note that database-level encryption does not prevent the details from showing using SQL Injection), might not be such a bad idea. Again: treat an email address as such personal detail.
For many sites the above is probably not the case, and you should focus on prohibiting SELECT * FROM through SQL Injection, and making sure visitors cannot somehow get to someone else's personal profile or order information by changing the URL.
It's worth to encrypt data in Databases, it's not making it a bit more difficult but way more difficult when its encrypted in the right way so stop philosophy and encrypt the sensitive data ;)
#Roo
I somewhat agree to what you are saying but isn't it worth encrypting the data just to make it a little more difficult for someone to get it?
With your reasoning, it would be useless to have locks or alarms in your house, because they can also easily be compromised.
My reply:
I would say that if you have sensitive data that you don't want to fall in the wrong hands, you should probably do it as hard as you can for a hacker to get it, even if it's not 100% fool proof.
I miss one important answer here.
When you have European users, you have to comply with the GDPR rules.
Email addresses are considered personal data meaning Art.5 does apply on email addresses.
processed in a manner that ensures appropriate security of the
personal data, including protection against unauthorised or unlawful
processing and against accidental loss, destruction or damage, using
appropriate technical or organisational measures (‘integrity and
confidentiality’).
Of course this does not say that you must encrypt email addresses. But by encrypting the data you do protect it from snooping employees. And protect yourself as a developer from requests to make a manual change in the database.
You really have to weigh your worst case senario of someone obtaining those email addresses, the likelihood of someone obtaining them, and your extra effort/time needed to impliement the change.