I apologize for my lack of knowledge on how the intricacies of the web work ahead of time.
I run a fairly large deal site (lets call it dealsite.com) and we send a lot of traffic to Amazon.com. Is there anyway for me to hide from Amazon that the users are are coming from dealsite.com? I do not want Amazon to know that we (dealsite.com) are the ones sending the traffic.
Maybe strip certain cookies?
Send outbound traffic through a proxy?
I am not doing anything illegal and these are real users not bots.
By using the noreferrer tag on your links, you can prevent Amazon from learning their traffic is coming from your site, and you don't need to set up a proxy, vpn, or cookie redirects.
HTTP generally sends the referring page along with its request for the new page as part of the HTTP referer section of the request header, and that's how sites track where their visitors come from. So for example, a user would click through to Amazon.com from Dealsite.com, and the request would include an HTTP referer telling Amazon.com that the user was linked from Dealsite.com.
To prevent web sites like Amazon from learning that their traffic came from your site, prevent your links from sending the HTTP referer. In HTML5, just add rel="noreferrer" to your links, and then referral information will not be sent to the site that was linked. The noreferrer link type is only suppported in new browsers, so I suggest using the knu's noreferrer polyfill to make sure it works on older browsers too.
So far this will prevent referrer information from being sent from 99.9% of your users - the only users that will send referral information will be users that are both using old browsers and have JavaScript disabled. To make it 100%, you could require users have JavaScript enabled to be able to click on those particular links.
Disclaimer: This is not the thorough idea you're looking for. I ran out of space in the comments so posted it as an answer. A couple of possible solutions come to my mind.
Proxy servers: Multiple distributed proxy servers to be specific. You can round robin your users through these servers and and hit Amazon so that the inbound traffic to Amazon from dualist.com keeps revolving. Disadvantage is that this will be slow depending on where the proxy server resides. So not the most ideal solution for an Ecommerce site but it works. And the major advantage is that implementation will be very simple.
VPN tunneling: Extremely similar to proxy server. VPN tunnel to another server and send redirect to Amazon from there. You'll get a new (non dealsite.com) IP from the VPN server of this network and your original IP will be masked
Redirects from user (Still in works) For this one I was thinking of if you could store the info you need from dealsite.com in a cookie and then instruct the host to redirect to Amazon by itself. Hence the inbound traffic to Amazon will be from the users IP and not dealsite.coms. If you need to get back to the dealsite session from Amazon, you could use the previously saved cookie to do so.
Ill add to this answer if I find something better.
Edit 1 A few hours more hours researching brought me to the Tor project. This might be useful but be wary, Many security experts advise against using Tor. See here
I know that browsers often prevent cross domain http requests to servers due to security measures (which can be avoided by CORS or JSONP), but what about a server making an http request to another server? Can that be blocked by security restrictions?
I guess what I'm asking is that since the server is making the request and not the browser, would I still need to deal with things such as CORS and/or JSONP, or are those work arounds specifically geared towards browser-level security?
A computer is free to send whatever requests it wants.
In the case of CORS, that's one piece of software (the browser) restricting less trusted code (Javascript) running on the same computer. But if you have full access to the computer you can do anything.
It is a browser specific measure designed to deal with the fact that people often run untrusted code in their browser and sensibly want to restrict it. More specifically, the Same Origin Policy causes the restriction and CORS is a way around it for participating servers due to the need for legitimate cross site AJAX.
Blocked by whose security restrictions? Of course it could be, but not by the user. A server making an HTTP request to another web server is no different than your browser making the same request.
I want to implement https on only a selection of my web-pages. I have purchased my SSL certificates etc and got them working. Despite this, due to speed demands i cannot afford to place them on every single page.
Instead i want my server to serve up http or https depending on the page being viewed. An example where this has been done is ‘99designs’
The problem in slightly more detail:
When my visitors first visit my site they only have access to non-sensitive information and therefore i want them to be presented with simple http.
Then once they login they are granted access to more sensitive information, e.g. profile information for which HTTPS is used to deliver.
Despite being logged in, if the user goes back to a non-sensitive page such as the homepage then i want it delivered using HTTP.
One common solution seems to be using the .htaccess file. The problem is that my site is relatively large meaning that to use this would require me to write a rule for every page (several hundred) to determine whether it should be server up using http or https.
And then there is the problem of defining user generated content pages.
Please help,
Many thanks,
David
You've not mentioned anything about the architecture you are using. Assuming that the SSL termination is on the webserver, then you should set up separate virtual hosts with completely seperate and non-overlapping document trees, and for preference, use a path schema which does not overlap (to avoid little accidents).
APIs with terrible security are common place. Case in point - this story on TechCrunch.
It begs the question, how do you balance security with performance when it comes to SSL? Obviously, sensitive information such as usernames and password should be sent over SSL. What about subsequent calls that perhaps use an API key? At what point is it okay to use an unencrypted connection when it comes to API calls that require proof of identity?
If you allow mixed content, then a man-in-the-middle, can rewrite mixed content to inject JS to steal sensitive information already in the page.
With cafés and the like providing free wireless access, man-in-the-middle attacks are not all that difficult.
https://www.eff.org/pages/how-deploy-https-correctly gives a good explanation:
When hosting an application over
HTTPS, there can be no mixed content;
that is, all content in the page must
be fetched via HTTPS. It is common to
see partial HTTPS support on sites, in
which the main pages are fetched via
HTTPS but some or all of the media
elements, stylesheets, and JavaScript
in the page are fetched via HTTP.
This is unsafe because although the
main page load is protected against
active and passive network attack,
none of the other resources are. If a
page loads some JavaScript or CSS code
via HTTP, an attacker can provide a
false, malicious code file and take
over the page’s DOM once it loads.
Then, the user would be back to a
situation of having no security. This
is why all mainstream browsers warn
users about pages that load mixed
content. Nor is it safe to reference
images via HTTP: What if the attacker
swapped the Save Message and Delete
Message icons in a webmail app?
You must serve the entire application
domain over HTTPS. Redirect HTTP
requests with HTTP 301 or 302
responses to the equivalent HTTPS
resource.
The problem is that without understanding the performance of your application it is just wrong to try and optimize the application without metrics. This is what leads to decisions by devs to leave an API unecrypted simply thinking it's eeking out another 10ms's of performance. Simply put the best way to balance security concerns versus performance is to worry about security first, get some load from real customers(not whiteboard stick figures being obsessed over by some architect) and get real metrics from your code when you suspect performance might be an issue. I have a weird feeling that it won't be security related.
You need to gather some evidence about the alleged performance issues of SSL before you leap. You might get quite a surprise.
If I was setting up a server, and had the SSL certificate(s), why wouldn't I use HTTPS for the entire site instead of just for purchases/logins? I would think it would make more sense just to encrypt the entire site, and protect the user entirely. It would prevent problems such as deciding what has to be secured because everything would be, and it's not really an inconvenience to the user.
If I was already using an HTTPS for part of the site, why wouldn't I want to use it for the entire site?
This is a related question: Why is https only used for login?, but the answers are not satisfactory. The answers assume you've not been able to apply https to the entire site.
In addition to the other reasons (especially performance related) you can only host a single domain per IP address* when using HTTPS.
A single server can support multiple domains in HTTP because the Server HTTP header lets the server know which domain to respond with.
With HTTPS, the server must offer its certificate to the client during the initial TLS handshake (which is before HTTP starts). This means that the Server header hasn't been sent yet so there is no way for the server to know which domain is being requested and which certificate (www.foo.com, or www.bar.com) to respond with.
*Footnote: Technically, you can host multiple domains if you host them on different ports, but that is generally not an option. You can also host multiple domains if your SSL certificate is has a wild-card. For example, you could host both foo.example.com and bar.example.com with the certificate * .example.com
I can think of a couple reasons.
Some browsers may not support SSL.
SSL may decrease performance somewhat. If users are downloading large, public files, there may be a system burden to encrypt these each time.
SSL/TLS isn't used nearly often enough. HTTPS must be used for the entire session, at no point can a Session ID be sent over HTTP. If you are only useing https for logging in then you are in clear violation of The OWASP top 10 for 2010 "A3: Broken Authentication and Session Management".
Why not send every snail-mail post in a tamper-proof opaque envelope by Registered Mail? Someone from the Post Office would always have personal custody of it, so you could be pretty sure that no one is snooping on your mail. Obviously, the answer is that while some mail is worth the expense, most mail isn't. I don't care if anyone reads my "Glad you got out of jail!" postcard to Uncle Joe.
Encryption isn't free, and it doesn't always help.
If a session (such as shopping, banking, etc.) is going to wind up using HTTPS, there's no good reason not to make the whole session HTTPS as early as possible.
My opinion is that HTTPS should be used only when unavoidably necessary, either because the request or the response needs to be safeguarded from intermediate snooping. As an example, go look at the Yahoo! homepage. Even though you're logged in, most of your interaction will be over HTTP. You authenticate over HTTPS and get cookies that prove your identity, so you don't need HTTPS to read news stories.
The biggest reason, beyond system load, is that it breaks name-based virtual hosting. With SSL, it's one site - one IP address. This is pretty expensive, as well as harder to administer.
For high latency links the initial TLS handshake requires additional round trips to validate the certificate chain (including sending any intermediate certificates), agree on cipher suites and establish a session. Once a session is established subsequent requests may utilize session caching to reduce the number of round trips but even in this best case there is still more round trips than a normal HTTP connection requires. Even if encryption operations were free round trips are not and can be quite noticable over slower network links especially if the site does not leverage http pipelining. For broadband users within a well connected segment of the network this is not an issue. If you do business internationally requring https can easily cause noticable delays.
There are additional considerations such as server maintenance of session state requiring potentially significantly more memory and of course data encryption operations. Any small sites practically need not worry about either given server capability vs cost of todays hardware. Any large site would easily be able to afford CPU /w AES offload or add-on cards to provide similar functionality.
All of these issues are becoming more and more of a non-issue as time marches on and the capabilities of hardware and the network improve. In most cases I doubt there is any tangable difference today.
There may be operational considerations such as administrative restrictions on https traffic (think intermediate content filters..et al) possibly some corporate or governmental regulations. Some corporate environment require data decryption at the perimeter to prevent information leakage ... interference with hotspot and similiar web based access systems not capable of injecting messages in https transactions. At the end of the day in my view reasons for not going https by default are likely to be quite small.
https is more resource-hungry than the normal http.
It demands more from both the servers and the clients.
If whole session is encrypted then you won't be able to use caching for static resources like images and js on proxy level eg ISP.
You should use HTTPS everywhere, but you will lose the following:
You should definitely not use SSL Compression or HTTP Compression over SSL, due to BREACH and CRIME attacks. So no compression if your response contains session or csrf identifiers. You can mitigate this by putting your static resources (images, js, css) on a cookie-less domain, and use compression there. You can also use HTML minification.
One SSL cert, one IP address, unless using SNI, which doesn't work on all browsers (old android, blackberry 6, etc).
You shouldn't host any external content on your pages that don't come over SSL.
You lose the outbound HTTP Referer header when browser goes to an HTTP page, which may or may not be a problem for you.
Well, the obvious reason is performance: all of the data will have to be encrypted by the server before transmission and then decrypted by the client upon receipt, which is a waste of time if there's no sensitive data. It may also affect how much of your site is cached.
It's also potentially confusing for end users if all the addresses use https:// rather than the familiar http://. Also, see this answer:
Why not always use https when including a js file?
https requires the server to encrypt and decrypt client requests and responses. The performance impact will add up if the server is serving lots of clients. That's why most current implementations of https is limited to password authentication only. But with increasing computing power this may change, after all Gmail is using SSL for the entire site.
In addition to WhirlWind's response, you should consider the cost and applicability of SSL certificates, access issues (it's possible, though unlikely, that a client may not be able to communicate via the SSL port), etc.
Using SSL isn't a guaranteed blanket of security. This type of protection needs to be built into the architecture of the application, rather than trying to rely on some magic bullet.
I was told that on one project at our company, they found that the bandwidth taken up by SSL messages was significantly more than for plain messages. I believe someone told me it was an astounding 12 times as much data. I have not verified this myself and it sounds very high, but if there is some sort of header added to each page and most pages have a small amount of content, that may not be so far out.
That said, the hassle of going back and forth between http and https and keeping track of which pages are which seems like too much effort to me. I only once tried to build a site that mixed them and we ended up abandoning the plan when we got tripped up by complex things like pop-up windows created by Javascript getting the wrong protocol attached to them and that sort of thing. We ended up just making the whole site https as less trouble. I guess in simple cases where you just have a login screen and a payment screen that need to be protected and they're simple pages, it wouldn't be a big deal to mix-and-match.
I wouldn't worry much about the burden on the client to decrypt. Normally the client is going to be spending a lot more time waiting for data to come over the wire than it takes to process it. Until users routinely have gigabit/sec internet connections, client processing power is probably pretty irrelevant. The CPU power requried by the server to encrypt pages is a different issue. There might well be issues of it not being able to keep up with hundreds or thousands of users.
One other small point (maybe someone can verify), If a user types data into a form item such as a text box and then for some reason refreshes the page or the server crashes out for a second, the data the user entered is lost using HTTPS but is preserved using HTTP.
Note: I'm not sure if this is browser specific but it certainly happens with my Firefox browser.
windows Server 2012 with IIS 8.0 now offers SNI which is Server Name Indication which allows multiple SSL Web Applications in IIS to be hosted on one IP Address.