I have been asked to develop a highly secure B2B File Transfer system between three companies.
VPN is not an option and they prefer to use common ports like 80,443, etc, so no extra firewall configuration shall be done.
i found solutions like oftp2 and as2 to be sufficient enough. although, i have some questions before i can decide:
is not https file transfer secure enough. so i can use asp.net/C# to do the task.
what about existing tools like SFTP, rsync and other *nix tools.
what about using SOAP?
my main concern is to avoid any possible clear data exposing to the outer world.
all ideas are appreciated.
thanks in advance.
if you use a block cipher like AES to encrypt the data and send the result using RSA encryption that will do the job. For the RSA you encrypt using their "Public key" which you get them to send to you out of band (Courier service) then they decrypt with their private key. This is totally secure providing both companies keep their private key secret. You have a key pair for each of the 3 companies. The extra AES layer is if you are really paranoid and really really want to make sure even if someone got the private keys they still can't read the data. Also you should sign all messages: send a hash of the rest of the message encrypted (AES) with your private key then the recipient can decrypt with your public key, and hash the data themselves and if their hash is not the same as your one that was attached after it was decrypted then it was not from you. This prevents man in the middle, domain in the middle etc interceptions. This would only allow someone to interfere if they got both the public and private key and the AES password... at that point the estimated crack time is well over 2 billion years with 2048 bit RSA so I think you're safe.
Technically you can always do a scp/rsync over ssh, if port 22 is among the white-listed port. If not, you can run a ssh daemon on 80/443 etc.
To answer your question, yes https/SFTP are secure enough, so is rsync if done over a encrypted channel (refer http://troy.jdmz.net/rsync/index.html)
Another thing you can explore is stunnel ( http://www.stunnel.org/ )
I can think of more than one ways to go about it. Totally depends on your servers' OS and other restrictions you may have.
The main issue with SSL is certificate validation. By default all certificates matching the target domain which are signed by any of a plethora of CAs is considered valid. If you are paranoid, you should check the certificate used on the connection directly against the a certificate stored in your configuration.
Using a DHE handshake to achieve perfect forward privacy would also be nice, but the built in SSL API in .net doesn't expose a way to enforce that. So you might or might not get DHE depending on the version of windows and .net.
Another good choice is tunneling something over SSH. For example SCP is an existing file copying utility that does this.
OK, you don't want to expose the file contents, with files to be exchanged between three parties, to anyone else.
There are two things to consider:
1) Protect the transport. Here, the files are sent over an encrypted link. So, you're basically putting the normal bits into a tunnel that is encrypted to protect anyone from snooping over the link. This is usually done using SFTP for company-to-company communications and keys are exchanged and authenticated out-of-band before any transfers occur.
2) Protect the files. Here, each file is encrypted independently and then transported to the destination. You encrypt the files of the file before they leave your network and then they are decrypted once they arrive at their destination. This is usually done using PGP for company-to-company communications and the PGP keys are exchanged and authenticated out-of-band before any transfers occur.
If you protect the transport, you're just sending the data through a protected pipe, linking the companies. Once the file is received, it's not encrypted (it's only encrypted through the pipe). If you protect the file, you are block-encrypting files themselves, so it's more of a process to encrypt and decrypt the files; only the actual process/system that has the PGP keys at the receiving end can decrypt the file.
So, what do you want to do? That's a risk decision. If you're only concerned about someone intercepting the file contents that is not company A or B (or C), you need to protect the transport (SFTP, et al). If you're concerned about protecting each file independently and making sure that only specific processes at the receiving end can decrypt the file, you want to protect the files. If the data is very sensitive, and under high risk, you may want to do both.
Some very good points have been made in security issues of developing your own file transfer programs. There are software security, network security, and user authentication security issues involved here. Understanding all the various encryption algorithms and security rules take years to master and is a time consuming endeavor for the development team to just keep up with all of the intricate changes in digital security standards and laws.
Another option is that there are several very good and affordable managed file transfer (MFTP) solutions that have already developed and addressed all of these security issues. They also have mastered the workflow of file transfer management to make this process much much easier on the IT staff. One of these MFTP solutions that I've used for the past few years is Linoma Software's GoAnywhere product. It has saved our team months of time and headache, allowing us to focus on our core business.
I hope this helps...
Related
My temmate proposed me to encrypt login/password before sending them to backend.
We use https so I don't see any reason why I should do it and in what circumstances.
Https itself, by definition, is all about encrypting data and protecting us from "man in the middle" attack.
As for me it is not the reason when someone say that we should protect our client from viruses and sniffers, which he has on its computer, and which can decrypt SSL traffic using kindof RSA algorithms. It is up to the owner of the computer. Antiviruses can cope with that pretty well.
One reason when it could be applied is when we store such data on browsers/client's side (for example in local storage), and I think it is architectural risks. All data should be persisted on backend side and browser should be stateless in a way.
Question: in what cases, if any, we do need to encrypt our data before transferring them over https?
Thanks in advance!
You should apply protection to assets (data in this case) when you want to protect it against something. It makes sense to encrypt if you have a threat in mind, a usecase where this data would be compromised, which is prevented by the proposed encryption.
By default, it's a standard and accepted thing to send usernames and passwords without further encryption over https. If you can come up with ways why that's not enough for you in your specific case, you may apply further encryption. For example you might want to protect it against man-in-the-middle attackers, say a corporate proxy that has its root cert installed on clients (very common in corporate settings).
Note though that encrypting it is not straightforward at all, key management is relatively difficult. How will the client have the key, how will it store it, how will you revoke it if compromised, and so on.
Usually it's not worth it to go down this route. In some very specific cases, it might be.
I'm trying to get a hobby/educational project off the ground that involves user accounts. Naturally, it will be critical for users to be able to log in securely. The information I will be storing and transporting for users is not intended to be personal or personally identifiable, but just on principle I'd like password transport to be as secure as possible.
I understand that the best way to get the user's password safely to the back end from the front is by using HTTPS. However, I don't want to pay for a CA for this learning/experiment project, and I also don't want to get browser warnings for self-signed certs, because though this is a hobby project, I would ultimately like to share it with the world (like a portfolio piece, something to share with friends, for people to actually find useful and fun).
Granted, you'd be right to say that if I want it to be a "portfolio piece," I should use best practices such as HTTPS, but I still don't want to pay an annual CA fee for what ultimately is a hobby/learning project.
I'm considering the asymmetric cryptography solution found here: https://github.com/travist/jsencrypt It's quite like the one suggested by ArthuruhtrA, and seems promising. It uses public/private key, so I could encrypt using public key on the front end, and transport encrypted to be decrypted on the back end with a secret/private key (where proper salting/hashing practices would occur before storage to a database). Does this seem reasonably safe, if untraditional?
Is there any other, better way to get user information (e.g. password) safely to the back end without browser warnings and without spending money needlessly?
If you use encrypted password from JavaScript it cannot be well secure since it is only client side customisation, i.e. it is not à service provided by the browser.
I would advise you to use TLS with a free certificate. You should have à look at letsencrypt.org.
I don't know what the best answer to this is. Obviously you've considered using a self-signed cert. I may be wrong, but depending on the OS and browser, you may be able to tell it to trust certs you have signed.
Another solution might be to use the asymmetric cryptography principle behind HTTPS, but without actually using SSL: Include your server's public key in your webpage. Use javascript to encrypt the data with it. Your server would then be able to decrypt it using its private key, without a middleman being able to do so.
Hope that helps! If I come up with something else I'll add it here.
Edit:
On second thought, this solution would be super vulnerable to a man-in-the-middle attack. Sorry.
I am creating a chat service program that follows the server/client paradigm. That chat program exists as both a chat server and a chat client, and a user can either host the chatroom(and it will connect his client to that server), or he can join an existing one.
Clients connect via a direct IP address that the other user will tell them, such as gained from whatismyip.com, and a specified port number.
During any time in this chat program, one user can send a file to another user. This is initiated by asking the server to set up a handshake between the two users, with user A passing his IP through the server to user B, and user B calling the new service that user A created for file transfer. This eliminates the original chat server, and the users are connected via a direct IP using nettcp protocol.
Over this file transfer, the files are encrypted with AES after initially sending the AES private key via RSA encryption.
I want to know what kind of ways somebody can initiate a man in the middle attack here. Obviously I see the flaw I have in passing the IP address through the server to the other user, but right now I don't see any other way as I cannot have the server retrieve the IPV4 of the sender.
Is the way a man in the middle attack works, is that he can see that these two users are transferring files, and somehow pull the data stream to himself from both ends? Can he do this on an already ongoing file transfer session?
I'm trying to understand the way MITM attacks work so I can see if I can protect my program from such attacks... but if the only way to reliably do so is to use a certificate authority(of which I'm still learning about), please go ahead and tell me that.
After doing more searching, I found this great link explaining the different types of MITM attacks and how they work and are executed in great detail.
http://www.windowsecurity.com/articles/Understanding-Man-in-the-Middle-Attacks-ARP-Part1.html
There are a total of four parts.
Is the way a man in the middle attack works, is that he can see that
these two users are transferring files, and somehow pull the data
stream to himself from both ends? Can he do this on an already ongoing
file transfer session?
You need to define a threat model. The usual suspects are message insertion, deletion, tampering and reordering. Sometimes the attacker only needs to tamper with a message so you do the wrong thing. For example, he/she may need to flip a bit so "transfer $100 from A to B" changes to "transfer $900 from A to B". In this case, the attacker did not need to be in the middle or decrypt the message.
I'm trying to understand the way MITM attacks work so I can see if I
can protect my program from such attacks... but if the only way to
reliably do so is to use a certificate authority(of which I'm still
learning about), please go ahead and tell me that.
Rather than attempting to design a hardened protocol, perhaps you could use a protocol that already exists that addresses your concerns.
That protocol would be Z-Real-time Transport Protocol (ZRTP). The protocol is specified in RFC 6189, ZRTP: Media Path Key Agreement for Unicast Secure RTP.
ZRTP is a key exchange protocol that includes Short Authentication Strings (SAS) to keep out the MitM. Essentially, the SAS is a voice authentication that only needs to be performed once. You can omit the SAS check, though its not recommended. If you omit the check and the bad guy is not attacking, then everything is OK for current and future sessions.
Once you establish your first secure channel without adversarial tampering, all future sessions will be secure because of the way key agreement for the current session depends on earlier sessions. And the earliest session (first session) is known to be secure.
ZRTP also provides forward secrecy, so a compromise of the current session does not affect security of past sessions.
ZRTP does not require certification authorities or other (un)trusted third parties.
Dr. Matthew Green has a blog about ZRTP on his Cryptography Engineering site at Let's talk about ZRTP.
To answer your question about MitM, there's too much for a Stack Overflow answer. A great free book is Peter Guttman's Engineering Security. MitM is sometimes a goal of an attacker, but it not his/her only vector. Guttman's book looks at a number of threats, how humans act and react, why the attackers succeed, and how to design around many of the problems.
Are there any protocols which would allow 2 visitors to communicate securely through my website, without the possibility of me reading their messages?
Ask Google for End-to-End encryption like PGP/GPG. For a client-side browser-based implementation you might want to check out GPG encryption in JavaScript.
I just googled it and cannot tell if its really secure (not sending your private key to anyone). I just want to give you a point to start with.
EDIT: Looks like it does send the clients private key to your server to perform a server-based encryption. This is not what you want. But I am sure that a JavaScript implementation of GPG is possible even though I don't know if somebody has done it yet.
Yes; for example, this is what would happen if your server was a link in a communication protected by SSL/TLS.
The participants use a public-key encryption scheme to agree upon a secret, symmetric key; that is then used to encrypt their communication.
It's also possible for the participants to simply encrypt their messages with the public key of the intended receiver. That way, only the intended receiver can decrypt the message. This is not a very advanced scheme and probably vulnerable. (Among others, if an eavesdropper can guess exactly what is sent, he can encrypt that message with the intended recipient's public key and see if the result matches with what is being sent).
There is a lot of literature available on cryptographic protocols; for starters, here's a Wikipedia article on Key Agreement Protocols.
If we're talking about not possible then the second part to S.L. Barth's answer will achieve this with the exception that the key exchange must be done by some other means. This can be the phone or email or even another website but if it's done via your website then it's open to a man-in-the-middle attack. You can tell your users to do this, you just can't actually help them do it.
There is probably a javascript library somewhere that will implement GPG encryption so that all you have to do is make sure that every message is encrypted in the browser before it's sent to your web server. You can store the messages as long as you like, they're encrypted. Only the user with the correct private key will be able to decrypt it.
SSL and TLS as they are used by websites everywhere are vulnerable to man-in-the-middle attacks. The reason we don't hear much about these sorts of attacks is that most of the people in the middle are trustworthy so the attacks simply don't happen. The recent revoking of the CA certificates of DigiNotar and others was precisely because the Iranian Government were caught acting as a man-in-the-middle and decrypting their own citizen's SSL traffic.
If you're happy with preventing casual snooping by curious sysadmins, the key exchange can be done through your website as well.
One more thing: Security is hard.
Even if you do this with well-known encryption techniques, the chances of there being a flaw in the implementation will be very close to 1. This doesn't mean that those curious sysadmins will be able to accidentally read messages but it does mean that a determined and skilled adversary will be able to find a way in. As soon as you can afford it you should hire an expert to redesign or at the very least examine your protocol and implementation.
In general, such a secure link between your users without you being able to read and/or modify their messages is only possible if they have some way of identifying each other (or at least in one direction).
This might be a shared secret (like a passphrase) or a public key known to one (or certified by a CA known to one), where the other one has the corresponding private key.
On this one can build a secure protocol (using a key exchange and then symmetric encryption with MACs in both directions), like TLS does. (Another way, used often for instant messaging, is OTR, the Off-the-Record messaging protocol.)
Without a way to identify the other end point, you end up with a way of allowing man-in-the-middle attacks. SSL/TLS without certificates, or with certificates where the man-in-the-middle knows the corresponding private key, is insecure, as is every other similar encryption scheme.
Another issue is that you said visitors of my website. This looks like you would implement client-side cryptography in JavaScript, delivered from your website. Don't do this ... if the visitors do not trust you not to read their data, they also should not trust you to feed them non-malicious JavaScript, which might implement something else than you are claiming it does, again allowing a MITM, or even directly sending a copy of the data to you.
More details about this are discussed in Javascript Cryptography Considered Harmful (from a slightly different perspective).
I am looking for something like https, but backwards. The user generates their own private key (in advance) and then (only later) provides the web application with the associated public key. This part of the exchange should (if necessary) occur out-of-band. Communication is then encrypted/decrypted with these keys.
I've thought of some strange JavaScript approaches to implement this (From the client perspective: form submissions are encrypted on their way out while (on ajax response) web content is decrypted. I recognize this is horrible, but you can't deny that it would be a fun hack. However, I wondered if there was already something out there... something commonly implemented in browsers and web/application servers.
Primarily this is to address compromised security when (unknowingly) communicating through a rogue access point that may be intercepting https connections and issuing its own certificates. Recently (in my own network) I recreated this and (with due horror) soon saw my gmail password in plain text! I have a web application going that only I and a few others use, but where security (from a learning stand point) needs to be top notch.
I should add, the solution does not need to be practical
Also, if there is something intrinsically wrong with my thought process, I would greatly appreciate it if someone set me on the right track or directed me to the proper literature. Science is not about finding better answers; science is about forming better questions.
Thank you for your time,
O∴D
This is already done. They're called TLS client certificates. SSL doesn't have to be one-way; it can be two-party mutual authentication.
What you do is have the client generate a private key. The client then sends a CSR (Certificate Signing Request) to the server, who signs the public key therein and returns it to the client. The private key is never sent over the network. If the AP intercepts and modifies the key, the client will know.
However, this does not stop a rogue AP from requesting a certificate on behalf of a client. You need an out-of-band channel to verify identity. There is no way to stop a man in the middle from impersonating a client without some way to get around that MITM.
If a rogue access point can sniff packets, it can also change packets (an ‘active’ man-in-the-middle attack). So any security measure a client-side script could possibly provide would be easily circumvented by nobbling the script itself on the way to the client.
HTTPS—and the unauthorised-certificate warning you get when a MitM is trying to fool you—is as good as it gets.
SSL and there for HTTPS allows for client certificates. on the server side you can use these environment variables to verify a certificate. If you only have 1 server and a bunch of clients then a full PKI isn't necessary. Instead you can have a list of valid client certificates in the database. Here is more info on the topic.
Implementing anything like this in JavaScript is a bad idea.
I don't see, why you are using assymetric encryption here. For one, it is slow, and secondly, it is vulnerable to man in the middle anyhow.
Usually, you use an asymmetric encryption to have a relatively secure session negotiation, including an exchange of keys for a symmetric encryption, valid for the session.
Since you use a secure channel for the negociation, I don't really understand why you even send around public keys, which themselves are only valid for one session.
Asymmetric encryption makes sense, if you have shared secret, that allows verifying a public key. Having this shared secret is signifficantly easier, if you don't change the key for every session, and if the key is generated in a central place (i.e. the server and not for all clients).
Also, as the rook already pointed out, JavaScript is a bad idea. You have to write everything from scratch, starting with basic arithmetic operations, since Number won't get you very far, if you want to work with keys in an order of magnitude, that provides reasonable security.
greetz
back2dos