IIS ARR replace encode slash %2F in query string - iis

I have an gitlab ce installed on VM Ubuntu-GLS inside the LAN. To give it access from internet am using IIS ARR URL Rewrite module at my primary web server. Gitlab for getting some files from repository via API requires to split catalog with forward slash encoded as %2F.
If I sending request to gitlab API like:
curl --header "PRIVATE-TOKEN: <myPrivToken>" "http://gitlab.mysite.com/api/v4/projects/8/repository/files/DemoAM%2F.project?ref=master"
Where is DemoAM is folder in root repository. ARR replaces %2F on forward slash, and gitlab gets request like (i hid some non important fields from gitlab log):
{
"status": 404,
"method": "GET",
"path": "/api/v4/projects/8/repository/files/DemoAM/.project",
"host": "Ubuntu-GLS",
"remote_ip": "192.168.0.1:58496, 192.168.0.102, 127.0.0.1"....
where %2F deplaced with /
But if I sending request directly to local VM/ like:
curl --header "PRIVATE-TOKEN: <myPrivToken>" "http://Ubuntu-GLS/api/v4/projects/8/repository/files/DemoAM%2F.project?ref=master"
gitlab gets correct request like:
{
"status": 200,
"method": "GET",
"path": "/api/v4/projects/8/repository/files/DemoAM%2F.project",
"host": "Ubuntu-GLS",
"remote_ip": "192.168.0.102, 127.0.0.1"....
where if %2F provided as is, and it's correct
I've find a very similar question here
rewrite urls with slashes ( %2F ) in query string
but i can't using it's answer because I can't change gitlab request rules.
Also, without result i've tried to set up useOriginalURLEncoding to false as was recommended here: Windows IIS ARR Reverse Proxy Encoding Issue.
So, is the other way(s) to solve this problem? I expect to solution where rest string {R:1} will passthrow as is, without replacing encoded symbols
I use rules like this:
<rule name="ReverseProxyInboundRule1" enabled="true" stopProcessing="true">
<match url="(.*)" />
<conditions logicalGrouping="MatchAny">
<add input="{CACHE_URL}" pattern="^(https?)://gitlab\.mysite\.com" />
</conditions>
<action type="Rewrite" url="{C:1}://Ubuntu-GLS/{R:1}" appendQueryString="true" />
<serverVariables>
<set name="HTTP_X_ORIGINAL_ACCEPT_ENCODING" value="{HTTP_ACCEPT_ENCODING}" />
<set name="HTTP_ACCEPT_ENCODING" value="" />
</serverVariables>
</rule>
<outboundRules>
<rule name="ReverseProxyOutboundRule1" preCondition="ResponseIsHtml1" enabled="true">
<match filterByTags="A, Form, Img" pattern="^http(s)?://Ubuntu-GLS/(.*)" />
<action type="Rewrite" value="http{R:1}://gitlab.mysite.com/{R:2}" />
</rule>
<preConditions>
<preCondition name="ResponseIsHtml1">
<add input="{RESPONSE_CONTENT_TYPE}" pattern="^text/html" />
</preCondition>
<preCondition name="NeedsRestoringAcceptEncoding">
<add input="{HTTP_X_ORIGINAL_ACCEPT_ENCODING}" pattern=".*" />
</preCondition>
</preConditions>
</outboundRules>

Change
<action type="Rewrite" url="{C:1}://Ubuntu-GLS/{R:1}" appendQueryString="true" />
to
<action type="Rewrite" url="{C:1}://Ubuntu-GLS{UNENCODED_URL}" appendQueryString="false" />
and also change request filtering to allow double escaping.
Reference
https://blogs.iis.net/iisteam/url-rewrite-v2-1

Related

Keycloak behind IIS Reverse Proxy

I just cant get my reverse proxy to work properly and I hope that someone can help me with that.
A few Informations:
1 VM Windows Server 2019,IIS 10 (with ARR and URL Rewrite Module),"keycloak" App under the "Default Website", proxy-address-forwarding="true" were added in the standalone.xml
keycloak is locally available at "localhost:8080". The default website is available from intern network (Port 80 is open, 8080 is closed). I want to use the IIS as a reverse proxy for keycloak. It should look like the following URL.
Request: "http://server_fqdn/keycloak" -> IIS Reverse Proxy -> localhost:8080
At first I tried it without "/keycloak" and it worked perfectly. After I added "/keycloak" it just shows 404 Errors. I saw that it tries to open "http://server_fqdn/auth" instead of "http://server_fqdn/keycloak/auth". If I enter "/keycloak/auth" manually it works. My first thought was to just write another Blank Inbound Rule which redirects "http://server_fqdn/auth" to "http://server_fqdn/keycloak/auth". It works, but now there is another problem. If i want to enter the admin console I get an error which says "Invalid parameter: redirect_uri".
It stops at the following URL (not complete, but the necessary part is there)
http://server_fqdn/keycloak/auth/realms/master/protocol/openid-connect/auth?client_id=security-admin-console&redirect_uri=http%3A%2F%2Fserver_fqdn%2Fkeycloak%2Fauth%2Fadmin%2Fmaster%2Fconsole%2F&state=.....
If i remove %2Fkeycloak after "redirect_uri" it works and i get the keycloak login screen. Maybe someone can help me here.
Inbound Reverse Proxy Rule:
<rule name="ReverseProxyInboundRule1" enabled="true" stopProcessing="true">
<match url="^keycloak/(.*)" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="false" />
<action type="Rewrite" url="http://localhost:8080/{R:1}" />
<serverVariables>
</serverVariables>
</rule>
Outbound Reverse Proxy Rule:
<outboundRules>
<clear />
<rule name="ReverseProxyOutboundRule1" preCondition="ResponseIsHtml1" enabled="true">
<match filterByTags="A, Area, Base, Form, Head, IFrame, Img, Input, Link, Script" pattern="^http://localhost:8080/(.*)" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="true">
</conditions>
<action type="Rewrite" value="http://server_fqdn/keycloak/{R:1}" />
</rule>
<preConditions>
<preCondition name="ResponseIsHtml1">
<add input="{RESPONSE_CONTENT_TYPE}" pattern="^text/html" />
</preCondition>
</preConditions>
</outboundRules>
Inbound Reverse Proxy Rule:
<rule name="keycloak" enabled="true" stopProcessing="true">
<match url="^auth/(.*)" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="false" />
<action type="Redirect" url="http://server_fqdn/keycloak/{R:0}" redirectType="Permanent" />
</rule>

Remove Namespace Prefix From MediaWiki URL with IIS10 URL Rewrite

I have a MediaWiki installation hosted in IIS10. I am attempting to use the URL Rewrite module to remove the namespace prefix from some MediaWiki article URLs.
In MediaWiki, articles outside the "Main" namespace show as foo.com/wiki/Namespace:Article_Title
I only need to remove the 'Bar:' Namespace prefix, others can remain intact
Restriction: I am unable to move these articles into the Main namespace
The desired result would present content from foo.com/wiki/Bar:Article_Title but display as foo.com/wiki/Article_Title.
Existing Inbound URL Rewrite
I have one URL rewrite already in place, per Microsoft's doc for MediaWiki on IIS, which removes the query string from the url. Here is my rewrite rule based on that guidance:
<rule name="Clean-WikiArticlePages" stopProcessing="false">
<match url="^wiki/(.*)$" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="false" />
<action type="Rewrite" url="/w/index.php?title={UrlEncode:{R:1}}" />
</rule>
What I've tried so far
I've tried both an Inbound and Outbound rule, to no effect.
Inbound rule:
<rule name="Clean-Namespace" enabled="true">
<match url="^wiki/(Meetings:)(.*)" />
<action type="Rewrite" url="/wiki/{R:2}" />
</rule>
Outbound rule:
<outboundRules>
<rule name="Remove-Namespace" preCondition="IsHTML" enabled="true">
<match filterByTags="A" pattern="^wiki/Bar:(.*)" />
<action type="Rewrite" value="/wiki/{R:1}" />
</rule>
<preConditions>
<preCondition name="IsHTML">
<add input="{RESPONSE_CONTENT_TYPE}" pattern="^text/html" />
</preCondition>
</preConditions>
</outboundRules>
I think it is possible to achieve this if you only want to rewrite for Bar.
You want the URL displayed as foo.com/wiki/Article_Title and load content from /wiki/Bar:Article_Title?
Then you should redirect all /wiki/Bar:Article_Title request to foo.com/wiki/Article_Title. Then rewrite foo.com/wiki/Article_Title back to /wiki/Bar:Article_Title request.
But please keep in mind this only work for static namespace. Because the rewritten request unable to get the namespace of original request before redirection.
Not sure if below rules achieve your requirement
When you access
foo.com/wiki/Bar:Article_Title ------redirect----->foo.com/wiki/Article_Title
-------rewrite------> foo.com/wiki/Bar:Article_Title-------rewrite---->foo.com/w/index.php?title=bar%3AArticle_Title
<rule name="redirect" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{REQUEST_URI}" pattern="^/wiki/bar:Article_Title$" />
</conditions>
<action type="Redirect" url="wiki/Article_Title" redirectType="Temporary" />
</rule>
<rule name="rewrite">
<match url="^wiki/Article_Title$" />
<action type="Rewrite" url="wiki/bar:Article_Title" />
</rule>
<rule name="Clean-WikiArticlePages" enabled="true" stopProcessing="true">
<match url="^wiki/(.*)$" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="false" />
<action type="Rewrite" url="/w/index.php?title={UrlEncode:{R:1}}" />
</rule>

IIS Rewrite rule, url modify himself

I have write a rewrite rule to redirect an incoming request to the correct server.
Here my web.config :
<rules>
<rule name="ToMonceau">
<match url="test/(.*)" />
<action type="Rewrite" url="http://10.5.5.83/{R:1}" />
<serverVariables>
<set name="HTTP_X_ORIGINAL_ACCEPT_ENCODING" value="{HTTP_ACCEPT_ENCODING}" />
<set name="HTTP_ACCEPT_ENCODING" value="" />
</serverVariables>
</rule>
</rules>
<outboundRules>
<rule name="AddPrefix" preCondition="IsText" enabled="true">
<match filterByTags="A, Img, Link, Script" pattern="(http://10.5.5.83/mantis/)?(/mantis/)?(.*)" />
<conditions>
<add input="{URL}" pattern="(test/mantis)/(.*)" />
</conditions>
<action type="Rewrite" value="./{R:3}" />
</rule>
<rule name="RestoreAcceptEncofing" preCondition="NeedsRestoringAcceptEncoding">
<match serverVariable="HTTP_ACCEPT_ENCODING" pattern="^(.*)" />
<action type="Rewrite" value="{HTTP_X_ORIGINAL_ACCEPT_ENCODING}" />
</rule>
<preConditions>
<preCondition name="IsText">
<add input="{RESPONSE_CONTENT_TYPE}" pattern="^text/(.+)" />
</preCondition>
<preCondition name="NeedsRestoringAcceptEncoding">
<add input="{HTTP_X_ORIGINAL_ACCEPT_ENCODING}" pattern=".*" />
</preCondition>
</preConditions>
</outboundRules>
My problem when I go to http://localhost/test/mantis (my server hosts a mantis here), the url change automatically to http://localhost/mantis/, I have to put again the "test/" and it change to http://localhost/mantis/login_page.php. I reput again "test/" and this time the login's page of mantis shown.
The problem continue when I try to log in, the url changes continously by removing the "test/" part.
If I go directly to my mantis without the redirect (http://10.5.5.83/mantis) everything is working like a charm.
What am I missing in my rule to do it correctly ?
I take the exemple here with mantis but I have the same problem on every sites host by different server.
Thanks in advance and sorry if my english is not perfect.
Go to IIS manager and click on URL rewrite:
Keep pattern .*
Action type should be Rewrite
Rewrite URL: http://yourdesiredurl/{R:0}

JQuery script getting cut off when accessed via reverse proxy

We have a reverse proxy server running on wwwdev.example.com, and we have the full site running on dev.example.com. If I go to dev.example.com/scripts/jquery-version.js, I get the full jQuery-version.js file returned, ending like this:
if ( typeof define === "function" && define.amd && define.amd.jQuery ) {
define( "jquery", [], function () { return jQuery; } );
}
})( window );
If I go to wwwdev.example.com/scripts/jquery-version.js, the file is cut off before it completes:
if ( typeof define === "function" && define.amd && define.amd.jQuery )
The only relevant inbound rule follows:
<rule name="ReverseProxyInboundRule1" enabled="true" stopProcessing="true">
<match url="(.*)" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="false">
<add input="{HTTP_HOST}" pattern="^(.+?\.)?wwwdev\.example\.com" />
</conditions>
<serverVariables>
<set name="HTTP_ACCEPT_ENCODING" value="" />
<set name="HTTP_X_PROXY_REQUEST" value="" />
</serverVariables>
<action type="Rewrite" url="http://{C:1}dev.example.com/{R:1}" logRewrittenUrl="true" />
</rule>
The only relevant outbound rule follows:
<rule name="Rewrite Links in JS" preCondition="IsJavascript" enabled="true">
<match filterByTags="None" pattern=""/(.+)"" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="true" />
<action type="Rewrite" value=""http://wwwdev.example.com/{R:1}"" />
</rule>
<preConditions>
<preCondition name="IsJavascript">
<add input="{RESPONSE_CONTENT_TYPE}" pattern="^application/x-javascript" />
</preCondition>
</preConditions>
Static compression is on; dynamic compression is off. I've tried turning static compression off as well, but that doesn't seem to affect this result.
When I capture with Fiddler, I can see the full file returned, but it doesn't make it through the browser (IE or Chrome). I do get a Content-Length mismatch reported through Fiddler (Response Header indicated 266,885 bytes, but server sent 266,961 bytes), which I would assume is relevant, but I'm not sure what's causing the issue.
Any ideas would be GREATLY appreciated - this has been driving me nuts.
Answer found here:
Under the Application Request Routing Cache feature of the IIS server node, under Server Proxy Settings, Response buffer threshold (KB) needs to be set to 4096 (to match the Response buffer (KB)).

IIS Rewrite Rule is not setting HTTP_COOKIE

I'm working with a client who has a mobile site setup through a third party. Currently, we check through IIS whether a user agent matches any of your standard mobile agents in which case we redirect the user to the mobile version m.whatever.com.
One of the rules we have requires us to set the cookie to a value of 0 when the user wants to see the mobile site again.
<rules>
<rule name="if httpcookie is , set it to 0" stopProcessing="true">
<match url="^(.*)$" />
<conditions>
<add input="{HTTP_COOKIE}" pattern="mobileoptout=1" />
</conditions>
<serverVariables>
<set name="HTTP_COOKIE" value="mobileoptout=0" />
</serverVariables>
<action type="None" />
</rule>
</rules>
As per the above, we're matching on the URL and the cookie value. I have tested these independently and they work as expected. However, at the end of this rule, the value for the cookie MobileOptOut is still 1 and not 0.
I've searched and tried all the examples available on numerous sites but have been completely unable to understand why the value of the cookie is not being changed.
The domain for the cookie is [whatever.com] which is the same as www.whatever.com and based on previous tests it can read from the cookie to validate a condition.
Any ideas?
Including an additional attempt which does not work either:
<rules>
<rule name="set cookie">
<match url="(.*)" />
<serverVariables>
<set name="HTTP_COOKIE" value="optout=1" />
<set name="{HTTP_COOKIE}" value="optout=2" />
</serverVariables>
<action type="None" />
</rule>
</rules>
Thanks to #cheesemacfly I looked into using the outbound rules, which for some reason I completely overlooked.
The solution uses a single inbound rule to check two conditions.
<rule name="MobileOptOut=1 Stop Processing Rules" stopProcessing="true">
<match url="(.*)" />
<conditions logicalGrouping="MatchAny" trackAllCaptures="false">
<add input="{QUERY_STRING}" pattern="mobileoptout=1" />
<add input="{HTTP_COOKIE}" pattern="MobileOptOut=1" />
</conditions>
<action type="None" />
</rule>
This will handle the first request with the querystring and all subsequent requests with the cookie that will be created after this rule finishes.
The outbound rule is as follows:
<outboundRules>
<rule name="if querystring=1" preCondition="If mobileoptout query = 1">
<match serverVariable="RESPONSE_Set_Cookie" pattern="." />
<action type="Rewrite" value="mobileoptout=1; Domain=site.local; Path=/;" />
</rule>
<preConditions>
<preCondition name="If mobileoptout query = 1">
<add input="{QUERY_STRING}" pattern="mobileoptout=1" />
</preCondition>
</preConditions>
</outboundRules>
This will check that first condition of if querystring is mobileoptout=1. If true, it will set a cookie in the response named 'mobileoptout' with a value of 1 in the root domain that will expire with the session.
This is exactly what I was overlooking.
First, to make sure your rule can work, you have to setup the allowed server variables.
To do so, in the iis manager, under the URL Rewrite section, click on View Server Variables...:
You then can add a new variable by clicking Add... on the right.
In your case, you want to add HTTP_COOKIE:
From here, your rule (as following) should be working:
<rules>
<rule name="if httpcookie is , set it to 0" stopProcessing="true">
<match url="^(.*)$" />
<conditions>
<add input="{HTTP_COOKIE}" pattern="mobileoptout=1" />
</conditions>
<serverVariables>
<set name="HTTP_COOKIE" value="mobileoptout=0" />
</serverVariables>
<action type="None" />
</rule>
</rules>
Note that the cookie is rewritten only for the request.
Meaning the client cookies won't be changed but the request hitting the page will have the new cookie values when the rule is triggered.

Resources