I'm looking for a way to encrypt the entire DB and keep the ability to search for data although it's encrypted.
I have seen a lot of questions regarding encryption of at rest data in Mongo, but none of it got an answer that can help one complete a full flow for their application.
I hope to present here my findings and get feedback and more ideas (I still have some questions).
Encryption options:
1.mongoose-encryption.
Complete solution! Can encrypt all fo the db with minimal work for you!.
2. Procona mongodb - I didn't had a chance to test it, I've spent hours trying to install and get it to run, without luck (this is probably just me though..).
3. Create get and send methods to encrypt and decrypt your data in the Module level.
My requirements for at rest data encryption are:
Application layer does not need to be involved in the encryption- decryption process. Should be like we don't even have the data encrypted (for the most part).
We can perform search and lookups on encrypted data.
I don't know how to do that but hopefully search for partial words and phrases in encrypted text fields.
Of course that all data is encrypted expect for Object IDs.
My approach:
I want to try and use mongoose-encryption to use all the benefits of this amazing plugin.
I also want to add to the schema the Hash of the Real value in the encrypted field so I could preform find operations on encrypted field.
The problem:
I can't seem to find the correct mongoose Hook to temper with the non-encrypted data before mongoose-encryptions hides it. So I can't generate my Hash.
This doesn't work:
Users.pre('save', () => {
this.hashedName = hash(this.name)
console.log(":(")
});
Also as mentioned above, searching for partials and phrases in encrypted data.
With my approach we could find someone named "Danielle" but we can't search in Hash for users with a name that starts with "Dani".
Please give me your opinion as well for my approach. I know that this is a topic without easy to find solutions.
If you want to encrypt the data on disk, encrypt the entire disk and encrypt the swap. If someone gets a copy of the database (e.g. you forgot to put auth on the database and someone connects to the database and dumps the data) the plaintext is exposed.
If you want the database to store encrypted data only, use client side encryption. This requires key management on the client side but makes it so that someone dumping your database doesn't get the plaintext.
I have a web app that uses known username and password combinations to login to external servers. There are multiple username/password combinations used for different services. Right now, they are essentially "hard-coded" into the website code, but, I would like to move this information off the code base for better security.
My initial thought is to store this data in the database which is used to support the website. I want to store it in a way that it is not easily "hackable" (i.e. I'm not going to store it as plain text or as a MD5 hash). Should I follow the same format that I use to store the website user's passwords, where I use a random number generator to create SALT for each password and then store the password as hashed combination of the password and SALT, or would this be overkill?
Generally, storing passwords in the application code is always a bad idea. Moving it outside the code has many advantages including security.
Now storing it either in DB or Configuration Files is a choice you have to take depending on your application.
For full security you should never store passwords in retrievable form. But to login to a external server as in your case, you need to get the actual plain text password, so one way hash will not work for you.
In our product we deal with such situation by using 2 Way SSL Certificates. It is very secure and there is no need to store the passwords.
But if you really need to store the passwords, then I will suggest to use configuration file and let your application read it. You can encrypt the passwords stored in the configuration files (Encrypting the passwords stored in the configuration file will again bring you back to the same question of how to protect the key). The access to the configuration file should be restricted (in Unix, 600 File Permission).
Alternatively, if your web application is Java, then you can consider using JNDI.
After more research, I've decided at this point to follow the ideas here:
Encrypt a Column of Data - SQL Server | Microsoft Docs
...and encrypt/decrypt on the DB inside a Stored Procedure.
This is just a question to put out there as I am unsure if there is a benefit to this or not. However I am thinking about encrypting the files that are uploaded to my site (As they could potentially contain users names and addresses along with some other personal data). However it occurred to me that the only way hackers could get hold of these files would be to gain control of the server and if they did that then they would have access to the encrypted files and also the database connection strings etc. Once they have those then they could easily decrypt the files.
The traffic itself is protected with SSL and I am not interested in using public key encryption as that is not something that would work for the users. So is it worth the extra effort and hassle of encrypting the uploaded files?
Introduction
I want to create a Java web application for storing and backing up user files, similar to Dropbox. One of the interesting Dropbox feature is that it can detect whether a certain file already exists on server. For example, if one user upload a file onto server, another user who tries to upload the same file will not need to upload the same file content. Server will only need mark that he has the same file. This helps to save the bandwidth/space and increases the speed in many ways.
The most basic solution to this problem is to use a file hash string, e.g. sha1, md5, etc., to identify the file. The client software check whether a certain hash exists on server or not. If it exists, then it can skip the uploading process and mark that user has the same file.
Problem
The web application is implemented based on REST architecture so that user can easily write their own client software to upload their files. For security reasons, the SSL is enabled for all transactions. But my most security concern is about users faking that they have a file without actually owning it if I use sha1 or any other standard hash alogorithms. This cannot be prevented by SSL or encryption. If a user manage to get the hash string, e.g. md5 and sha1 of many files can be found by googling, he can mark that he has the file using REST service on the web application.
So one of the possible solution is that the server requests a set of certain random bytes from the file as well as the hash of the whole file. Here is example steps:
Client checks whether a certain hash exists on server or not. Then, server returns the required positions of random bytes if the file already exists.
Client sends random bytes as per request if the server has the file. Client software will not be able to response to it without having the actual file.
In this way, it can save the bandwidth as well as ensure that user owns the file they want to upload.
Question
I am no expert in Security over the web so I have no idea whether this is a good idea or not. I have read some articles about implementing their own fancy process might lead to the reduction in security strength because the security cannot be tested and the extra information may provide a cracking method.
Does anyone has any comment on the process?
Will it reduce the sucurity?
Does anyone have an idea to solve this problem differently?
I understand that there might not be an exactly answer to this question but I would like to hear if anyone has encounter the same problem and has any good solution to it.
Rather than asking the client to upload some random bytes of the file's contents, it may be better to ask the client to upload the hash of a random region the file. That way you can use a wider range of sizes that you ask the client to verify.
Better yet, though, may be to send the client a random number and require the client to compute an HMAC of the entire file's contents using that number as the key. This is more computationally-expensive since the server must compute the HMAC too, but it verifies that the client has the entire file, not just a small portion of it.
One unavoidable side effect of this hash feature, even with a verification scheme, is that it reveals that a copy of the file already exists somewhere on the server. That by itself may be sensitive information.
For the most stringent privacy protection, you should forego this feature and make each user upload their own copy of the file. You can use hash comparison on the server to avoid storing multiple copies of the file, transparently to the clients.
I want to create a website where a user enters content (say a couple of sentences) which eventually gets stored in a backend database (maybe MySQL). But before the content leaves the client side, I want it to get encrypted using something on client like maybe javascript.
The data will travel over the web encrypted, but more importantly, will also be permanently stored in the backend database encrypted.
Is JavaScript appropriate to use for this? Would 256 bit encryption take too long?
Also, how do you query an encrypted database later on if you want to pull down the content that a user may have submitted over the past 2 months?
I'm looking for tips, suggestions and any pointers you guys may have in how to go about learning about and accomplishing this.
Thanks!
You shouldn't implement the encryption for the communication between the client and the server yourself, use SSL (https) for that. As for encrypting data in the database, you can always use MySQL's built-in methods, such as AES_ENCRYPT and AES_DECRYPT, see reference manual for details.
Look at http://www.farfarfar.com/scripts/encrypt/ (encrypt/decrypt).
Tried text/text/XXTEA with success.
However, that's about as far you can go with JS encryption.
As long as you're not using SSL/https, the main disadvantage is:
A fuzz to protect the private key, (it can be done though, like a form field, not submitted, but requires user to enter value on each session.)
About searching/extracting data on encypted data, I belive nothing is going to work.
Well, to point out something: if it's client side encrypted, it'll likely also be easily decrypted, seeing as with languages like javascript, they're being handed the source code of your encryption scheme. Plenty of encryption schemes out there, AES, Blowfish, etc, but if the data is traveling on an encrypted connection, I suppose the encryption of the data only adds very little security, maybe packet sniffers locally installed or something to that effect.
I would also suggestion you look into using compression as well, I myself have used LZMA, Huffman, and even base64 encoding with javascript to at least obscure the content from casual observers. Point being, no matter how good the encryption, you're handing over the process to the client, and they can just view your source and be able to easily reverse the encryption, given that they know the encryption scheme and aren't to lazy to do a bit of Googling or searching on Wikipedia. I personally prefer compression because it also reduces the size of the data being sent, and unless one is trained in analysis of encryption and compression, it is isn't easy to figure out the compression being used from sniffing packets.
EDIT: But if you want high levels of security, I would suggest against using just compression, and instead, using an asymmetric encryption method like RSA or Diffie-Hellman encryption on data traveling between the server and client, as the private key will never be shared.
For client side, javascript or better, a Java application, would work (at least you can't right click -> view source with a Java app in two seconds). 256 bit encryption wouldn't take too long if you're just wanting to save a few sentences like you said. A modern processor with blast through that in milliseconds. Then, when it arrives at the server, encrypt it using AES or another powerful algorithm (note that the US government uses AES 256 for their Top Secret documents) before sending it to the database. (Also want to add that you can also store a hash (MD5 is a function readily available in PHP, you can opt for something like Whirlpool as well, but you'll have to find a library for that) of the data on a separate database, and check against it when you access it to ensure that your database hasn't been compromised)
For querying, I suggest PHP. Read the data, decrypt it, then send to the client (in still encrypted form), or if you want, and have the client decrypt it also, if you don't trust the networks the data is being sent through.
Here's a good source on Javascript and AES: http://point-at-infinity.org/jsaes/
My two saved links using RSA in javascript: http://www.ohdave.com/rsa/ and http://www.hanewin.net/encrypt/rsa/rsa.htm
As for the data being sent over the network encrypted, this is the kind of thing that HTTPS was made for. Nothing is transmitted in the clear. It's encrypted securely enough for most banks to trust it. Don't reinvent the wheel here; it'll never be as round as what a bunch of people who make wheels for a living have come up with.
Have your form submit to an https:// url (rather than http://), and the script/app at that URL talk to the database, encrypting and decrypting the data as needed. Javascript won't easily talk directly to MySQL anyway, and if you do all the encryption client-side, anyone who can get the page can still decrypt it (since by necessity the client would need to be given the decryption key as well).
Check out this blogpost: http://www.ravellosystems.com/blog/all-you-need-to-know-to-configure-ssl-offloading/
It takes you through all the steps you need to do to configure your webs server to serve content under an encrypted channel (a.k.a SSL termination).