I am using IIS as a revese proxy for some backend services. ARR or Helicon ISAPI_Rewrite. Everything works fine, except that chunked responses from the service do not go through IIS (the connection to the endpoint that happens to keep the connection open and sends some data back to the client in chunks) never answers. The service itself without a reverse proxy works well, i.e. client receives updates. This is similar to CouchDB /_changes endpoint with 'continous' mode.
In Chrome, the connection is marked as "pending".
I tried to disable all the caching on IIS, but no success.
The whole setup mimics sending SSE events, actually. Implementation of SSE also does not work obviously.
Is there any way to fix that?
Related
I use a service called glitch.me to host my Node.js project, and they only allow you to open one port internally. At the moment, my Node.js server can only handle HTTP requests, but glitch.me proxies my traffic for me, effectively allowing the client to communicate indirectly using HTTPS. My project heavily relies on the use of WebSockets, and they work perfectly when accessing it directly via example.glitch.me (the url provided by glitch.me). However, I need to connect this project to my custom domain (example.xyz), all traffic of which is proxied through Cloudflare. However, when attempting to establish a WebSocket connection through my custom domain (in other words: via Cloudflare), the socket connection is never established.
Now, I realise that this sounds very specific, but I think it's possible to generalise here:
WebSockets work:
client => glitch.me => my backend
client => A => my backend
Webockets don't work:
client => cloudflare => glitch.me => my backend
client => A => B => my backend
Note: HTTP/HTTPS still works for both domains.
I added logging on the server-side every time a WebSocket connection was established, and sure enough the server receives the connections! However, the client is never notified and in Chrome Developer tools it shows that the request never completed. Remember, everything is working perfectly when the traffic is not routed through Cloudflare.
I have full understanding for the fact that it may be impossible for you to resolve my issue with the information given to you, but I'm not asking you to do so. My question is simply this: where does the problem lie? Does the response from my backend get stuck somewhere along the way and doesn't reach all the way through? I have contacted both glitch.me and Cloudflare and they have both confirmed that they fully support WebSocket traffic.
What is the most probable cause of my problem? From there I could probably fix it!
Edit: I uses the ws package on npm on the backend and standard JavaScript WebSockets on the frontend.
I have moved an old web app from being hosted on a very old custom implementation of a web host using Cassini to .net core 2.2 hosted in IIS.
Since the old servers were a self implementation, it wasn't rejecting some requests that IIS is rejecting.
To be specific, the problem is with POST requests with no body and no Content-Length header, with a Transfer-Encoding header of "chunked". There is absolutely no reason not to use simple GET requests there, but this is the situation.
The requests are sent from an old plugin that some of the clients are using, and we can't make them upgrade it to a newer version that is sending valid GET requests. Currently IIS is rejecting the requests with a 502 code. Therefor, we need to maintain an old version of the server just for these clients.
I am looking for a way to configure IIS to forward these requests to the .net core web app despite being invalid and to handle them there (when tested on Kestrel without IIS, the results are returned as expected).
Thanks
The problem was with IIS that acted as a reverse proxy since I used out of process hosting. Once I switched to in process hosting the request went directly to the server and the problem was solved.
https://weblog.west-wind.com/posts/2019/Mar/16/ASPNET-Core-Hosting-on-IIS-with-ASPNET-Core-22
My errors are becoming more and more ridiculous every minute. I now have the ultimate problem with my reverse proxy.
An IIS8 reverse proxy should send all requests coming for
(.*)\.nsf(.*)
(.*)\.ibmxspres(.*)
to the IBM Domino http server - which it does, at least for basic requests that every third-grader deems valid.
But now, guess what happens when I open the following "URL" (a) on the IBM Domino HTTP directly and (b) via IIS8 Reverse proxy:
http://www.mydomain.com/xsp/.ibmxspres/.mini/css/2Ojcore.css&2Ojdojo.css&2OldefaultTheme.css&2OldojoTheme.css&#Da&#Ib&#Th&#Ti&2TgxspSF.css&Wdojoroot/dojo/resources/dojo.css&Wdojoroot/dojox/grid/enhanced/resources/claro/EnhancedGrid.css&Wdojoroot/dojox/grid/enhanced/resources/tundra/EnhancedGrid.css&Wdojoroot/dojox/grid/enhanced/resources/EnhancedGrid_rtl.css&Wdojoroot/dojox/grid/resources/tundraGrid.css&Wdojoroot/dojox/grid/resources/claroGrid.css&Wdojoroot/dojox/grid/resources/Grid.css.css
Is this an URL that every HTTP server should be able to handle?
If yes, why isn't IIS routing them through, although they match the rule?
If no, why is IBM using such non-standard HTTP requests, and can I change this behaviour somewhere?
In my setup, I have 2 layers of transparent proxies. When a client makes an SSL request, I wish to have the first proxy it meets simply forward the traffic to another one without attempting to do the handshake with the client.
The setup seems funny, but it is justified in my case - the 2nd proxy registers itself to the first one (through some other service) only occassionally. It tells the first: "I'm interested in some traffic that looks like___". In most cases, the 1st proxy simply does the work.
Can an httpProxy (in node-proxy) proxy SSL requests? Must I use an httpsProxy (which will then do the handshake with the client)?
You could do all of this with the existing httpsProxy if you wanted to. Unless you are wanting to use a non-Node proxy or proxy to a different server, I can't see what you would gain by having two.
Simply add the required the logging/signing logic to the existing httpsProxy.
Typically, I use https on the proxy to both restrict the number of open ports and to remove the need to do https on all of the Node servers running. You can also add Basic Auth using http-basic library too.
See my example code: https://github.com/TotallyInformation/node-proxy-https-example/blob/master/proxy.js
EDIT 2012-05-15: Hmm, after some thought, I wonder if you shouldn't be looking at something like stunnel to do what you want rather than Node?
(For reference, I've already made some of those points in my answer to your similar question on ServerFault.)
If you are after a MITM proxy (that is, a proxy that can look inside the SSL content by using its own certificates, which can work provided the clients are configured to trust them), it will hardly be fully transparent, since you will at least have to configure its clients to trust its certificates.
In addition, unless all your client use the server name indication extension, the proxy itself will be unable to determine reliably which host to issue its certificate for (something that a normal HTTPS proxy would have been able to know by looking at the CONNECT request issued by the client).
If you're not after a MITM proxy, then you might as well let the initial connection through via your router. If you want to record that traffic, your router might be able to log the encrypted packets.
Having your router catch the SSL/TLS packets to send them transparently to a proxy that will merely end up relaying that traffic untouched anyway to the target server doesn't make much sense. (By nature, the transparent proxy will imply the client isn't configured to know about it, so it won't even send its CONNECT method with which you could have had the requested host and port. Here, you'll really have nothing more than what the router can do.)
EDIT: Once again, you simply won't be able to use an HTTP proxy to analyse the content of the connection transparently. Even when using a normal proxy, an HTTPS connection is relayed straight through to the target server. The SSL/TLS connection itself is established between the original client and the target server. The point of using SSL/TLS is to protect this connection, and to make the client notice if something is trying to look inside the connection.
Plain HTTP transparent proxy servers work because (a) the traffic can be seen (in particular, the request line and the HTTP Host header are visible so that the proxy can know which request to make itself) and (b) the traffic can be altered transparently so that the initial client doesn't notice that the request wasn't direct and works as if it was.
Neither of these conditions are true with HTTPS. HTTPS connections that go through an HTTP proxy are simply tunnel, after explicit request from the client, which has sent a CONNECT command and was configured to make use of such a proxy.
To do something close to what you're after, you'd need an SSL/TLS server that accepts the SSL/TLS connection and deciphers it (perhaps something like STunnel) before your HTTP proxy. However, this won't be transparent, because it won't be able to generate the right certificates.
Sorry if it is a duplicate, as I am not a security nor network expert I may have missed the correct lingo to find information.
I am working on an application to intercept and modify HTTP requests and responses between a web browser and a web server (see how to intercept and modify HTTP responses on server side? for the background). I decided to implement a reverse proxy in ASP.Net which forwards client requests to the back-end HTTP server, translates links and headers from the response to the properly "proxified" URL, and sends the response to the client after having extracted relevant information from the response.
It is working as expected, except for the authentication part: the web server uses NTLM authentication by default, and just forwarding requests and responses through the reverse proxy does not allow the user to be authenticated on the remote application. Both the reverse proxy and the web application are on the same physical machine and are executed in the same IIS server (Windows server 2008/IIS 7 if that matters). I tried both enabling and disabling authentication on the reverse proxy app with no luck.
I have looked for information about it, and it seems to be related to the "double-hop problem", which I do not understand. My question is: is there a way to authenticate the user on the remote application through the reverse proxy using NTLM? If there is none, are there alternative authentication methods I could use?
Even if you don't have a solution to my problem, just pointing me to relevant information about it to help me get out of the confusion would be great!
I found what the problem was (and it is NTLM): in order to have the browser asks the user for its credentials, the response must have a 401 status code. My reverse proxy was forwarding the response to the browser, so IIS was adding a standard HTML code to explain the requested page cannot be accessed thus preventing the browser from asking credentials.
The problem was solved by removing the response content when the status code is a 401.
With all due respect I have for the one that answered that some years ago, I must admit this is plainly false. The problem was indeed solved AFTER removing the response content when the status code is a 401, but it had none to do with the initial problem..
The truth is that windows authentication was made to authenticate people over local windows networks, where no proxy server is present or even needed.
The main problem with NTLM authentication is that this protocol does not authenticate the HTTP session but the underlying TCP connection, and as far as I know there is no way to access it from asp code.
Every proxy server I tried broke NTLM authentication.
Windows authentication is comfortable for an user because he won't ever need to enter your password to whatever application may lie in your intranet, frightening for a security guy because there is an auto-login without even a prompt if the site domain is trusted by IE, shocking for a network administrator because it melts the application, transport and network layer into some "windows ball of mug" instead of just plain http traffic.
NTLM won't work if the TCP packets are not forwarded exactly as the reverse proxy received > them. And that's why many reverse proxy doesn't work with NTLM authentication. (like nginx) > They forward HTTP requests correcty but not the TCP packets.
Nginx has the functionality to work with NTLM authentication. Keepalive needs to be enabled which is only available trough the http_upstream_module. Additionally in the location block you need to specify that you will be using HTTP/1.1 and that the "Connection" header field should be cleared for each proxied request. Nginx config should look something like:
upstream http_backend {
server 1.1.1.1:80;
keepalive 16;
}
server {
...
location / {
proxy_pass http://http_backend/;
proxy_http_version 1.1;
proxy_set_header Connection "";
...
}
}
I scratched my head for quite some time with this issue but the above works for me. Note that if you need to proxy HTTPS traffic, a separate upstream block is deemed necessary. To clarify a bit more, "keepalive 16;" specifies the number of simultaneous connections to the upstream your proxy is allowed to keep. Adjust the number as per the expected number of simultaneous visitors on the site.
Although this is an old post, I just want to report that it works for me quite well with an Apache2.2 reverse proxy and the keepalive=on option. Obviously, this keeps the connection between the proxy and the SharePoint host open and "pinned" to the client<>proxy connection. I don't exactly know the mechanisms behind this, but it works fairly well.
But: Sometimes, my users encounter the issue that they're logged in as another user. So there seems to be some mixing-up through sessions. I will have to give this some further testing.
Solution for everything (in case you have a valid, signed SSL certificate): Switch IIS to Basic Auth. This works absolutely fine, and even Windows (i.e. Office with SharePoint connection, all WebClient-based processes etc.) won't complain at all.
But they will when you're just using http without SSL/TLS, and also with self-signed certificates.
I confirm that it works with "keep-alive=on" on apache2.2
I examined frames with Wireshark, and I know why it doesn't work. NTLM won't work if the TCP packets are not forwarded exactly as the reverse proxy received them. That's why many reverse proxies, like nginx, don't work with NTLM authentication. Reverse proxies forward HTTP requests correctly but not the TCP packets.
NTLM requires a TCP reverse proxy.