IIS Reverse Proxy Re-Encoding URLs Containing Percent Sign (%) - iis

I am trying to set up a reverse proxy for Jenkins using IIS 7.5, Application Request Routing 3.0 (ARR), and URL Rewrite 2.0.
I have the proxy mostly working, but am running into issues with URLs that contain the percent symbol (%).
No matter what I try, the proxy insists on either de-encoding or re-encoding the percent sign in the rewritten URL.
This is how I want the URLs rewritten:
http://my.proxy/a%2Fb -> http://my.host:8080/a%2Fb
This is how the URLs are actually being rewritten:
http://my.proxy/a%2Fb -> http://my.host:8080/a/b
- or -
http://my.proxy/a%2Fb -> http://my.host:8080/a%252Fb
How can I get IIS\ARR\Rewrite to stop re-encoding my rewritten URLs?
Things I've tried:
A normal reverse-proxy (rewrites the URL as http://my.host:8080/a/b):
<rule name="ReverseProxyInboundRule1" stopProcessing="true">
<match url="(.*)" ignoreCase="true" />
<action type="Rewrite" url="http://my.host:8080/{R:1}" />
</rule>
Using the UNENCODED_URL server variable (rewrites the URL as http://my.host:8080/a%252Fb):
<rule name="ReverseProxyInboundRule1" stopProcessing="true">
<match url="(.*)" ignoreCase="false" />
<conditions logicalGrouping="MatchAll">
<add input="{UNENCODED_URL}" pattern="/(.*)" />
</conditions>
<action type="Rewrite" url="http://my.host:8080/{C:1}" />
</rule>
Just entering the URL in straight (as a test - also rewrites the URL as http://my.host:8080/a%252Fb):
<rule name="ReverseProxyInboundRule1" stopProcessing="true">
<match url="(.*)" ignoreCase="false" />
<action type="Rewrite" url="http://my.host:8080/a%2Fb" />
</rule>
All the ideas in Scott Hanselman's excellent "Experiments in Wackiness: Allowing percents, angle-brackets, and other naughty things in the ASP.NET/IIS Request URL"
<httpRuntime requestValidationMode="2.0" requestPathInvalidCharacters="*,:,&,\" relaxedUrlToFileSystemMapping="true" />
<security>
<requestFiltering allowDoubleEscaping="true" />
</security>'
Note: I ran into this behavior when my IIS reverse proxy ran afoul of Jenkins' built-in reverse proxy checking system which attempts to do an HTTP redirect to a URL of this form.

Joseph, that is a great summary of all the ways I've tried to resolve the exact same issue, having IIS with SSL routing traffic to my Gerrit instance. When I found your post I hoped that maybe someone figured out a magic way to configure it but I guess it's not possible. I have tried one more thing, I've written a custom rewrite provider for IIS so that I can undecode the percent signs before routing is done, but then I realized that the encoding takes place later and this is pointless (I forgot about your step nr 3 that shows it very good).
I couldn't however get rid of IIS like out did, so I have figured a workaround. I have implemented a simple service that acts as additional proxy between IIS and Gerrit. When you configure IIS like in step 2, requests that are forwarded will get %25 in place of percent characters in the urls. Instead of reaching Gerrit, IIS forwards the requests to the proxy service. The service changes all occurrences of %25 to % (decodes percents) and forwards it to Gerrit. Nothing needs to be done with the response. For those who want to go this way you can start from my simple implementation of the proxy in C#:
https://gist.github.com/gralin/b5edfd908a41fc7268a7757698af1e66

I was able to fix this issue using the second approach and setting useOriginalURLEncoding="false":
<rules useOriginalURLEncoding="false">
<rule name="ReverseProxyInboundRule1" stopProcessing="true">
<match url="(.*)" ignoreCase="false" />
<conditions logicalGrouping="MatchAll">
<add input="{UNENCODED_URL}" pattern="/(.*)" />
</conditions>
<action type="Rewrite" url="http://my.host:8080/{C:1}" />
</rule>
</rules>
See also the official blog post for background information. The wording of useOriginalURLEncoding is a bit unfortunate.

I have thought using proxy with url contains % symbol has the problem, but after that i have found out it wasn't. The issue that the proxy URL too long.
I use datatable with option server-side: true and type: GET. Then when loading content from server with the proxy url too long, there is a problem. I have improve the size of url request and the issue has been fixed.
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxQueryString="4000" maxUrl="2000" />
</requestFiltering>
</security>
<rewrite>...</rewrite>
...
</system.webServer>
But keep in mind that allowing long query string and url is a security risk, more over, it’s a bad design.

Related

URL Rewrite causes "This page can’t be displayed"

I have implemented URL Rewriting at the server level as I wanted to redirect all HTTP and HTTPS requests that matches a certain rule to my actual site, and redirection should only take place if the users are hitting my actual site. The rules works fine initially. However, triggering CTRL+R repeatedly on my actual site seems to render my site unaccessible. The error "This page can't be displayed" is then returned to the user. This test was done on IE 11 browser on Windows x64, and my web server is IIS 8.5 on Windows Server 2012 R2. The HTTP response code returned for redirect is configured as 307.
When I turn on Failed Request Routing on my IIS server, I see a warning message on REWRITE_DISABLED_KERNEL_CACHE in the Failed Request Logs. This was the time when the page returns "This page can't be displayed".
Disabling my URL Rewrite rule would immediately render both my HTTP and HTTPS sites accessible once again, and I've verified that redirect no longer works. Enabling the same rule thereafter but on my HTTPS site only would work.
As follows is my redirection rule
<system.webServer>
...
<rewrite>
<globalRules>
<clear />
<rule name="HTTPS to HTTP" enabled="true" stopProcessing="true">
<match url="^(downloads?/?)?$" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="false">
<add input="{REQUEST_URI}" pattern="http://.*?/downloads/" negate="true" />
</conditions>
<action type="Redirect" url="http://{HTTP_HOST}/downloads/" appendQueryString="false" redirectType="Temporary" />
</rule>
</globalRules>
<outboundRules>
</outboundRules>
</rewrite>
...
</system.webServer>
Basically, if the request hits any of the following example URLs, I will redirect them:
1) http://fqdn/download
2) http://fqdn/download/
3) https://fqdn/downloads
4) https://fqdn/downloads/
5) https://fqdn/download
6) https://fqdn/download/
I will not redirect, if the request hits my site directly:
http://fqdn/downloads/
When I hit my actual site, I realise that the redirection rule is still being applied. So I suspect there could be 2 different issues over here.
1) An infinite redirection rule being applied when requests are sent to http://fqdn/downloads/
2) Some unknown problem with REWRITE_DISABLED_KERNEL_CACHE
You're in trouble with infinite redirects because of your wrong assumptation about REQUEST_URI and the lack of HTTPS check.
{REQUEST_URI} contains URL's path, including the query string with a leading slash (never been welldocumented), never contains uri scheme or hostname. So, you have a false positive.
http(s)://<host>:<port>/<path>?<querystring>
Here's a self-explanatory rule.
<rule name="Force Http downloads page" stopProcessing="true">
<!-- If the url starts with download or downloads with an optional trailing slash -->
<match url="^downloads?/?$" />
<!-- Redirect -->
<action type="Redirect" url="http://{HTTP_HOST}/downloads/" appendQueryString="false" redirectType="Temporary" />
<!-- When -->
<conditions logicalGrouping="MatchAny">
<!-- REQUEST_URI does not start with "/downloads/" -->
<add input="{REQUEST_URI}" pattern="^/downloads/" negate="true" />
<!-- Or -->
<!-- HTTPS is not off -->
<add input="{HTTPS}" pattern="^off$" negate="true" />
</conditions>
</rule>
Hope it helps.

IIS URL Redirect failing

I need to redirect URLs of the form:
http://server1.name.here/path1/path2/?num=123456
to:
http://server2.name.here/path3/path2/?num=123456
using the IIS URL Rewrite Module 2.0 on IIS 8.5. I've used a DNS alias to handle the server name redirection.
I've been using the user interface to configure the path rewrite but despite trying several variations and extensive research I cannot get my rewrite to work, which is slightly embarrassing as I feel this should be a simple rewrite. The web.config produced by the URL rewite user interface is:
<rewrite>
<rules>
<rule name="path rewrite" patternSyntax="ECMAScript" stopProcessing="false">
<match url="path1/path2/" ignoreCase="true" />
<conditions>
<add input="{QUERY_STRING}" pattern="num=[0-9]+" />
</conditions>
<action type="Rewrite" url="path3/path2/" appendQueryString="true" logRewrittenUrl="true" />
</rule>
</rules>
</rewrite>
When I try to open server1.name.here/path1/path2/?num=123456 I get a "403 - Forbidden: Access is denied." and the logs show no evidence that my URL is being rewritten or even flagged by the rewrite rule.
I'm sure I'm missing something obvious, can anyone enlighten me as to what I've got wrong?
Many Thanks Eden

Adding Rewrite rule to web.config causes error in IIS console

Hi I have added a http to https redirect to my web.config
<rewrite>
<rules>
<rule name="HTTP/S to HTTPS Redirect" enabled="true" stopProcessing="true">
<match url="(.*)" />
<conditions logicalGrouping="MatchAny">
<add input="{HTTPS}" pattern="off" ignoreCase="true" />
</conditions>
<action type="Redirect" url="https://{HTTP_HOST}/" redirectType="Found" />
</rule>
</rules>
</rewrite>
It is within the as required, in fact it is exactly the same as another iis I have running on another box.
However on this server when I save the config and then check the IIS control panel I get an
"There was an error while performing this operation ... Details: Filename \?D:\site\web.config Error:
and there is no error. When I remove the rewrite from the config everything is fine.
The only difference between this server setup and the one that works is that the broken server website is not in the root or is it the default website.
Has anyone encountered this type of error before?
Thanks
Jon
Very strange, even though i had everything set up in IIS and during the add/remove roles to include the http redirect it still needed this extension adding
http://www.microsoft.com/web/gallery/install.aspx?appid=urlrewrite2
the urlrewrite. I installed that extension and it all started working. Shame the error message couldnt ell me that i was missing the feature.

Remove multiple forward slashes

I've noticed that with .NET MVC sites, you're able to hit URLs with multiple forward slashes, for example:
http://www.example.com//category
http://www.example.com//category//product
The URL loads fine and everything works, however, I've been asked to stop this from happening.
I've been trying to use IIS URL Rewrites to get it working:
<rewrite>
<rules>
<rule name="Remove multiple slashes" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{UNENCODED_URL}" matchType="Pattern" pattern="^(.*)//(.*)$" />
</conditions>
<action type="Redirect" redirectType="Permanent" url="{C:1}/{C:2}" />
</rule>
</rules>
</rewrite>
However, the results seem very temperamental. Sometimes the product URL will redirect, sometimes it won't and the same thing happens with the category. It's almost like the URL is being cached by the application.
Does anyone know if I can disable any caching that's in place, or if there is another way to get around this multiple slash issue?
Any help is much appreciated.
In the end, I have resorted to using a code behind redirect to get it working.
The issues I was having using the IIS URL Rewrites was due to the way IIS caches the redirects. When I disabled caching completely, as WouterH suggested, it worked. However, I'm not comfortable disabling caching in this way as it could introduce performance issues.
My fix was to use a code behind redirect in the Global.asax.cs file:
protected void Application_BeginRequest(object sender, EventArgs e)
{
string requestUrl = Request.ServerVariables["REQUEST_URI"];
string rewriteUrl = Request.ServerVariables["UNENCODED_URL"];
if (rewriteUrl.Contains("//") && !requestUrl.Contains("//"))
Response.RedirectPermanent(requestUrl);
}
I would have liked to use IIS URL Rewrite to get this working, unfortunately I didn't have the time to continue down that line.
Interestingly, the below method did work, however, the HTTP_X_REWRITE_URL is added by the Helicon ISAPI Rewrite which I'm running locally, but is not available on our production server.
<rewrite>
<rules>
<rule name="Remove multiple slashes" stopProcessing="true">
<match url=".*" />
<action type="Redirect" url="{REQUEST_URI}" />
<conditions>
<add input="{HTTP_X_REWRITE_URL}" pattern="([^/]*)/{2,}([^/]*)" />
</conditions>
</rule>
</rules>
</rewrite>
URL Rewrite Module
As IIS automatically normalizes url's with double slashes, you can try to redirect to the normalized url like this:
<rewrite>
<rules>
<rule name="Remove multiple slashes" stopProcessing="true">
<match url=".*" />
<action type="Redirect" url="{REQUEST_URI}" />
<conditions>
<add input="{UNENCODED_URL}" pattern="(.*?)[/]{2,}$" />
</conditions>
</rule>
</rules>
</rewrite>
You can also try to disable caching of the URL rewrite module:
Disabling internal caching of rewrite rules
To disable caching of inbound rewrite rules in URL Rewrite Module run
the following command from an elevated command prompt:
reg add HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\InetStp\Rewrite /v
RewriteCacheEnabled /t REG_DWORD /d 0
I have a small feeling that you'll have to restart the webserver after this change :)
Other option #1: non-cacheable server variable
This idea just popped into my mind:
use a non-cacheable server variable in the rule, f.e. try with HTTP_USER_AGENT
<rewrite>
<rules>
<rule name="Remove multiple slashes" stopProcessing="true">
<match url=".*" />
<action type="Redirect" url="{REQUEST_URI}" />
<conditions>
<add input="{UNENCODED_URL}" pattern="(.*?)[/]{2,}$" />
<add input="{HTTP_USER_AGENT}" pattern=".*" />
</conditions>
</rule>
</rules>
</rewrite>
You can explore other server variables here
Other option #2: clear browser cache
During web development, make sure you use Ctrl+F5 to refresh your page, or clear your browser cache after making changes like updating rewrite rules etc. Otherwise you can spend hours of watching to the same problem while it was just your browser that needed to refresh its cache.
Other option #3: IIRF
If you really can't get it to work with the IIS URL Rewrite Module, you can give the open-source ISAPI module IIRF a try. It accepts rules similar to mod_rewrite for Apache.
Try following
<rule name="RemoveMultipleSlashes" patternSyntax="ECMAScript" stopProcessing="true">
<match url=".*" />
<action type="Redirect" url="{REQUEST_URI}" />
<conditions>
<add input="{UNENCODED_URL}" pattern="([^/]*)/{2,}([^/]*)" />
</conditions>
</rule>

IIS UrlRewrite problem

I've been using the UrlRewrite IIS plugin for about a month on our production site.
I created a single redirect rule using the supplied template/wizard, the resulting config enrty is as follows:
<rewrite>
<rules>
<rule name="CanonicalHostNameRule1" enabled="true">
<match url="(.*)" />
<conditions>
<add input="{HTTP_HOST}" pattern="^www\.mycompany\.com$" negate="true" />
</conditions>
<action type="Redirect" url="http://www.mycompany.com/{R:1}" />
</rule>
</rules>
</rewrite>
It's been running fine until this morning, when the site started erroring with "too many redirects". As far as I know, nothing in the configuration or infrastructure changed.
I disabled the rule, and the site became functional again (though clearly without any redirecting).
I then re-enabled the rule, and now all is running as expected. I didn't make any changes to the rule other than to temporarily disable it.
Any ideas? Is the plugin buggy?
I'd recommend setting this up:
http://learn.iis.net/page.aspx/467/using-failed-request-tracing-to-trace-rewrite-rules/
This may help you track down the problem if you start getting the "too many directs" error again.
Try this other code, i have on my web and run perfect:
<rule name="Canonical Host Name" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{HTTP_HOST}" pattern="^yourdomain\.com$" />
</conditions>
<action type="Redirect" url="http://www.yourdomain.com/{R:0}" redirectType="Permanent" />
</rule>
The explanation is simple:
Match any URL received to process
The condition is that have anydomaintext.extension (your domain and extension) without prefix
Redirect to same domain with full prefix and put all url.
Other tries was R:1 but quit some of the main url and not run.
The sample from Ruslani:
http://blogs.iis.net/ruslany/archive/2009/04/08/10-url-rewriting-tips-and-tricks.aspx
I tried to use adding www but finally use the sample above.
The fix below worked for me. I discovered my rewrite rule was at out-of-date. The other domain had changed their URL policy and were now redirecting all traffic from otherdomain.com to www.otherdomain.com
<action type="Rewrite" url="http://otherdomain.com/abc/{R:1}" />
to
<action type="Rewrite" url="http://www.otherdomain.com/abc/{R:1}" />
Do you see the difference? By adding the 'www' I preempted the other domain redirection. I basically just complied with their new policy.

Resources