I am sending objects to a server in java, however I want to ensure that it has not been tampered with, in addition to the already provided security. I understand how to take do use a message digest, however, should the checksum ideally be placed in the object? and if so wouldnt this change the checksum at the end. How do you go about this? How does it work with TCP headers etc it is starting to confuse me qutie a bit.
Thanks
Let's say your checksum is 32 bytes long. You calculate your checksum from your data and append the checksum to the end of it. When you receive the data, you know that the checksum occupies the last 32 bytes and the actual data is before the checksum.
Checksum of the message cannot be part of that message - it would be almost impossible to calculate the checksum.
The checksum must be computed from the message + some secret key. (Without the secret key, attacker could modify the message and easily compute its checksum.) Only problem I see here is that attacker can repeat the message, so if you receive two same messages, you don't know whether the the later one is from the attacker.
Ad TCP headers - why are you bothering about it? Just send the message and the checksum. TCP is just transport protocol; TCP or IP checksums are not for security, they are for detecting transport errors.
I believe that TCP & IP calculate their checksums with the checksum fields filled with zeros. Then the receiver calculates the checksums assuming the fields are zeros, and compares with the actual checksum.
Certainly you could do something similar, if you didn't want to take the simple solution of just appending the checksum at the end.
However note that this doesn't provide much protection against tampering, as an attacker can just replace the checksum with a checksum of the changed data. You need public key signatures (with an appropriate trust relationship) to protect against tampering.
When using tcp/ip, it is reasonable certain that the stream is not corrupted in transit "by accident", so don't bother about that.
If you are worried about security, note that Md5 is not secure, anyone can compute a md5 checksum, you need some sort of secret key or pki-solution for the signing.
Note that "signing" is just authentication, the actual message can still be read by someone else. You need encryption + authentication if you also want the content to be secret.
A simple solution is to send your data over a ssl-socket.
I would recommend turning to bouncycastle for good encryption support.
If the messages are transported in some other way (and might need to be handled by other systems, stored on files, ftp:ed, mq:ed, xml-ed etc etc,) you can serialise the message to a byte[], and sign them using a signer in org.bouncycastle.crypto.signers and calculate and append a signature as a byte[] after the actual message.
In that case, you would need to use a message format so that you can separate and extract both the data and the signature, and then recalulate the signature (using the public key of the sender) and make sure that the received and calculated signatures match.
There are several standards for this, S/MIME, pgp, PKCS#7RFC 3369 , and they are probably better than anything you or me can come up with, so investigate them.
If you are using RMI, you could google "secure rmi" - use a SSL socket factory for instance. If you just want authentication (cleartext+signature), you might even be able to write a authenticating socket factory...
If you have no clue about security, the simplest solution might be to set up a secure tunnel using a VPN or ssh, and send your messages using that.
Related
TL;DR
Do I have to sign every single message, or is there a more efficient way of verifying the origin of messages?
I am developing a simple crypto protocol for fun. Of course I know that in any serious project, I should just use some industry standard, like OpenSSL, but this is for learning and experimenting.
The idea is that the communicating terminals exchange RSA public keys, then using these keys they exchange an AES key securely, so that the AES key can be used to encrypt every message from this point. I have already implemented all of this, and it works fine.
The problem is: a potential attacker cannot read anything thanks to AES, but could still for example cause errors or attempt to hijack the communication or do some other nasty stuff by being man-in-the-middle (for instance she/he could copy an encrypted message and disrupt things by sending it again and again). What I need is digital signature so I can confirm that messages are coming from the valid source and I'm in luck, because I already have a working RSA implementation.
I know how digital signatures work (taking the hash of the message and encrypting it using the private key, etc.), but the only way I can think of making this work is to sign every single message, then check whether the signature is valid or not on the receiving side. However, I'm concerned that this will slow down my protocol. The whole purpose of using AES to secure the communication (or any symmetric key encryption) is that it's a lot faster than RSA (or any public key encryption). Wouldn't doing this defeat the purpose of AES (or any symmetric key encryption)? So the question is: do I have to sign every single message? Or is there a more efficient way of doing this? How does for example OpenSSL handle this?
TL;DR Use authenticated encryption.
In symmetric encryption it is possible to produce a Message Authentication Code (MAC) that enables you to check whether a message that you sent was (maliciously) manipulated. A man-in-the-middle attacker has only a negligible advantage of forging an authentication tag for a message that you haven't tagged.
There are many ways of doing this, but it's generally seen that a MAC should authenticate the ciphertext and not the plaintext (Should we MAC-then-encrypt or encrypt-then-MAC?). Popular MAC algorithms are HMAC (e.g. HMAC-SHA256), CMAC/OMAC1 or GMAC. There are also some distinct authenticated modes such as GCM, EAX, OCB, SIV, CWC, etc. Those combine a mode to achieve confidentiality and a mode for authenticity without the need of different keys for both.
But this is not enough because this only enables the receiver to detect tampering or forgeries of messages. An attacker may still mount other attacks such as replay or delay attacks. Therefore you need to send nonces (such as a message counter) and time stamps along. The receiver would have to keep a record of previously sent messages (by storing the nonces) and not accept any messages that are sent too late judging by its internal clock.
In order to prevent the attacker from changing the nonces and time stamps at will, those also have to be authenticated. Most authenticated modes are actually Authenticated Encryption with Associated Data which can authenticate additional non-secret data such as nonces and time stamps.
The use of authenticated encryption can make a purely symmetric communication relatively tamper-proof under the assumption that the key was exchanged confidentially and was also verified through conventional digital-signatures such as RSA-PSS or Ed25519 (EdDSA).
I've coded the mobile api using Node.js, Redis & MongoDB. However right now I am looking for a concrete way to provide security in our system. The question I have inspired by the infamous principle of Auguste Kerckhoff;
"It must not be required to be secret, and it must be able to fall into the hands of the enemy without inconvenience"
After inspecting this principle I figure out that there is no safe way to secure data after a sniffer capturing the entire data package. Of course there are alternate ways like using an API key, using encryption algorithms like MD5 & MD6,Triple DES,SHA1 etc. However this also won't work if the entire data package is captured. And there is security precaution standarts like HTTPS, SSL certificates. However again if someone with talent captures the data package it can act just like a verified user in our system.
How would it be possible to apply a security method such that even though the entire data package is captured, the system would be able to distinguish the request coming from an outer source not from our verified user.
PS: I thought that applying a custom encryption algorithm with timestamp in order to prevent this problem could be a bit chaotic.
According to Kerckhoffs's principle "A cryptosystem should be secure even if everything about the system, except the key, is public knowledge." So the way cryptosystem, works is that the key is the only thing that can be used to decipher the system. If the key is fallen to the enemy then its all over.
In practice when you communicate over the internet or try to
authenticate your email account with the password. Your password is
never sent to, nor stored on the server in plain text. If you do then,
its not secure. The best security practice is not to store the
password at all (not even encrypted), but to store the salted hash of
the encrypted password.
That is one hash for one user. It is one way, you cannot get back user info, just test if it is in the database or not. Now even if the enemy takes control of the database, it cannot access your username/passwords.
Now coming to the point, it does not matter what you transmit in the communication channel, because the communication channel is the enemy!!! It is available to other users, anyone can sniff it. It is like enemies scanning each other on the radio.
The hash that has been sent over the channel can be sniffed and be used for authentication. True, but server can differentiate between spoofed attempt and the actual attempt, using HTTPS connection. Server tracks the HTTPS sessions and would ask to revalidate if something like this happens. HTTPS blocks use of sniffed data / MITM attacks. Even if sniffer gets hash (temporary ticket), he cannot do anything malicious, username and password cannot be cracked.
Thanks to Kerckhoff, our passwords are safe.
How to implement it on node.js ?? look for passport.js package. It implements the current standard OpenAuth.
I have been asked to develop a highly secure B2B File Transfer system between three companies.
VPN is not an option and they prefer to use common ports like 80,443, etc, so no extra firewall configuration shall be done.
i found solutions like oftp2 and as2 to be sufficient enough. although, i have some questions before i can decide:
is not https file transfer secure enough. so i can use asp.net/C# to do the task.
what about existing tools like SFTP, rsync and other *nix tools.
what about using SOAP?
my main concern is to avoid any possible clear data exposing to the outer world.
all ideas are appreciated.
thanks in advance.
if you use a block cipher like AES to encrypt the data and send the result using RSA encryption that will do the job. For the RSA you encrypt using their "Public key" which you get them to send to you out of band (Courier service) then they decrypt with their private key. This is totally secure providing both companies keep their private key secret. You have a key pair for each of the 3 companies. The extra AES layer is if you are really paranoid and really really want to make sure even if someone got the private keys they still can't read the data. Also you should sign all messages: send a hash of the rest of the message encrypted (AES) with your private key then the recipient can decrypt with your public key, and hash the data themselves and if their hash is not the same as your one that was attached after it was decrypted then it was not from you. This prevents man in the middle, domain in the middle etc interceptions. This would only allow someone to interfere if they got both the public and private key and the AES password... at that point the estimated crack time is well over 2 billion years with 2048 bit RSA so I think you're safe.
Technically you can always do a scp/rsync over ssh, if port 22 is among the white-listed port. If not, you can run a ssh daemon on 80/443 etc.
To answer your question, yes https/SFTP are secure enough, so is rsync if done over a encrypted channel (refer http://troy.jdmz.net/rsync/index.html)
Another thing you can explore is stunnel ( http://www.stunnel.org/ )
I can think of more than one ways to go about it. Totally depends on your servers' OS and other restrictions you may have.
The main issue with SSL is certificate validation. By default all certificates matching the target domain which are signed by any of a plethora of CAs is considered valid. If you are paranoid, you should check the certificate used on the connection directly against the a certificate stored in your configuration.
Using a DHE handshake to achieve perfect forward privacy would also be nice, but the built in SSL API in .net doesn't expose a way to enforce that. So you might or might not get DHE depending on the version of windows and .net.
Another good choice is tunneling something over SSH. For example SCP is an existing file copying utility that does this.
OK, you don't want to expose the file contents, with files to be exchanged between three parties, to anyone else.
There are two things to consider:
1) Protect the transport. Here, the files are sent over an encrypted link. So, you're basically putting the normal bits into a tunnel that is encrypted to protect anyone from snooping over the link. This is usually done using SFTP for company-to-company communications and keys are exchanged and authenticated out-of-band before any transfers occur.
2) Protect the files. Here, each file is encrypted independently and then transported to the destination. You encrypt the files of the file before they leave your network and then they are decrypted once they arrive at their destination. This is usually done using PGP for company-to-company communications and the PGP keys are exchanged and authenticated out-of-band before any transfers occur.
If you protect the transport, you're just sending the data through a protected pipe, linking the companies. Once the file is received, it's not encrypted (it's only encrypted through the pipe). If you protect the file, you are block-encrypting files themselves, so it's more of a process to encrypt and decrypt the files; only the actual process/system that has the PGP keys at the receiving end can decrypt the file.
So, what do you want to do? That's a risk decision. If you're only concerned about someone intercepting the file contents that is not company A or B (or C), you need to protect the transport (SFTP, et al). If you're concerned about protecting each file independently and making sure that only specific processes at the receiving end can decrypt the file, you want to protect the files. If the data is very sensitive, and under high risk, you may want to do both.
Some very good points have been made in security issues of developing your own file transfer programs. There are software security, network security, and user authentication security issues involved here. Understanding all the various encryption algorithms and security rules take years to master and is a time consuming endeavor for the development team to just keep up with all of the intricate changes in digital security standards and laws.
Another option is that there are several very good and affordable managed file transfer (MFTP) solutions that have already developed and addressed all of these security issues. They also have mastered the workflow of file transfer management to make this process much much easier on the IT staff. One of these MFTP solutions that I've used for the past few years is Linoma Software's GoAnywhere product. It has saved our team months of time and headache, allowing us to focus on our core business.
I hope this helps...
I want to create a website where a user enters content (say a couple of sentences) which eventually gets stored in a backend database (maybe MySQL). But before the content leaves the client side, I want it to get encrypted using something on client like maybe javascript.
The data will travel over the web encrypted, but more importantly, will also be permanently stored in the backend database encrypted.
Is JavaScript appropriate to use for this? Would 256 bit encryption take too long?
Also, how do you query an encrypted database later on if you want to pull down the content that a user may have submitted over the past 2 months?
I'm looking for tips, suggestions and any pointers you guys may have in how to go about learning about and accomplishing this.
Thanks!
You shouldn't implement the encryption for the communication between the client and the server yourself, use SSL (https) for that. As for encrypting data in the database, you can always use MySQL's built-in methods, such as AES_ENCRYPT and AES_DECRYPT, see reference manual for details.
Look at http://www.farfarfar.com/scripts/encrypt/ (encrypt/decrypt).
Tried text/text/XXTEA with success.
However, that's about as far you can go with JS encryption.
As long as you're not using SSL/https, the main disadvantage is:
A fuzz to protect the private key, (it can be done though, like a form field, not submitted, but requires user to enter value on each session.)
About searching/extracting data on encypted data, I belive nothing is going to work.
Well, to point out something: if it's client side encrypted, it'll likely also be easily decrypted, seeing as with languages like javascript, they're being handed the source code of your encryption scheme. Plenty of encryption schemes out there, AES, Blowfish, etc, but if the data is traveling on an encrypted connection, I suppose the encryption of the data only adds very little security, maybe packet sniffers locally installed or something to that effect.
I would also suggestion you look into using compression as well, I myself have used LZMA, Huffman, and even base64 encoding with javascript to at least obscure the content from casual observers. Point being, no matter how good the encryption, you're handing over the process to the client, and they can just view your source and be able to easily reverse the encryption, given that they know the encryption scheme and aren't to lazy to do a bit of Googling or searching on Wikipedia. I personally prefer compression because it also reduces the size of the data being sent, and unless one is trained in analysis of encryption and compression, it is isn't easy to figure out the compression being used from sniffing packets.
EDIT: But if you want high levels of security, I would suggest against using just compression, and instead, using an asymmetric encryption method like RSA or Diffie-Hellman encryption on data traveling between the server and client, as the private key will never be shared.
For client side, javascript or better, a Java application, would work (at least you can't right click -> view source with a Java app in two seconds). 256 bit encryption wouldn't take too long if you're just wanting to save a few sentences like you said. A modern processor with blast through that in milliseconds. Then, when it arrives at the server, encrypt it using AES or another powerful algorithm (note that the US government uses AES 256 for their Top Secret documents) before sending it to the database. (Also want to add that you can also store a hash (MD5 is a function readily available in PHP, you can opt for something like Whirlpool as well, but you'll have to find a library for that) of the data on a separate database, and check against it when you access it to ensure that your database hasn't been compromised)
For querying, I suggest PHP. Read the data, decrypt it, then send to the client (in still encrypted form), or if you want, and have the client decrypt it also, if you don't trust the networks the data is being sent through.
Here's a good source on Javascript and AES: http://point-at-infinity.org/jsaes/
My two saved links using RSA in javascript: http://www.ohdave.com/rsa/ and http://www.hanewin.net/encrypt/rsa/rsa.htm
As for the data being sent over the network encrypted, this is the kind of thing that HTTPS was made for. Nothing is transmitted in the clear. It's encrypted securely enough for most banks to trust it. Don't reinvent the wheel here; it'll never be as round as what a bunch of people who make wheels for a living have come up with.
Have your form submit to an https:// url (rather than http://), and the script/app at that URL talk to the database, encrypting and decrypting the data as needed. Javascript won't easily talk directly to MySQL anyway, and if you do all the encryption client-side, anyone who can get the page can still decrypt it (since by necessity the client would need to be given the decryption key as well).
Check out this blogpost: http://www.ravellosystems.com/blog/all-you-need-to-know-to-configure-ssl-offloading/
It takes you through all the steps you need to do to configure your webs server to serve content under an encrypted channel (a.k.a SSL termination).
I was given advice that I am suspicious about so I'm looking for support here to go back and challenge the advice.
I was advised to use Diffie-Hellman to get both sides to agree on a secret key, use the secret key to generate an AES key, and then use AES to encrypt/decrypt passwords that are being transmitted. Pretty much like the sample code here
When using this scheme, the length of the encrypted password is the same as the length of the unencrypted password. Should I be worried about this?
Before, I was using RSA, encrypting the passwords with the receiver's public key. This was resulting in an encrypted length of 256 no matter what the password length. Isn't that better?
You can just pad to whatever length with any data. It doesn't have to be random. As long as it's all encrypted. I think though that is the least of your worries.
Note if you use Diffie-Hellman you still need to authenticate the parameters sent, which you probably need to do with RSA.
The alternatives are:
Use RSA to exchange an encrypted secret key that you then use to encrypt your data.
Use Diffie-Hellman to exchange a secret key and then use RSA to sign values sent to authenticate the transaction.
If you do all this, then you have to also worry about whether exchanges have been replayed to make you reuse keys etc.
To be honest if you need to ask this question then you probably are not qualified to write a crypto protocol. They are extremely hard to get right and not for the faint hearted.
Suggest you use SSL/TLS for your exchange if you need to stream a lot of data. PGP/PKCS#7 if you just need to send a single message.
First off: Don't invent your own authentication protocol. Period. If you do, you WILL get it wrong even if you're using strong encryption. There are a number of existing well documented authentication protocols that have been vetted by cryptographers and thus are thought to be secure. Don't be tempted to "simplify" them, they've already been simplified.
Second: IMHO you should never send passwords on the wire for authentication (I'm not aware of any authentication protocol which does, including the hideously insecure NTLMv1 protocol)[1].
If you're dead set on going down the "roll my own authentication scheme" path, here's how I'd make the scheme you described above more secure (Caveat: I'm not a cryptographer - I believe that there are serious weaknesses in what I'm describing here):
Instead of sending the password directly, send a one-way-function (also known as a OWF, often implemented as a cryptographic hash like SHA256 or stronger) of the password.
In other words, have the server send the client a salt value, add the salt to the password, compute the OWF of the password+salt value and send the OWF result to the server. On the server, add the salt to the password and also perform the OWF calculation. If the results are the same, the password is valid, if they're not it's invalid.
And finally have whatever you do reviewed by a real cryptographer. They will find problems in your implementation and you're going to have to fix them. They're likely to suggest that you abandon your effort in favor of an existing published protocol.
[1] AFAIK, the only time you should send the password on the wire is when you're changing the password and even then, you should pad the length to a multiple of the block size (include the length in the cybertext so that when you decrypt it you can distinguish between the password and the padding).
If you can help it, don't send passwords over the wire at all. Instead, use a scheme like SRP, which authenticates both parties with one password.