Custom field in IIS log has "-" for value for certain Paths - url-rewrite-module

I have configured the following rewrite rule in an ASP.NET application's "web.config" hosted on IIS:
<rewrite>
<rules>
<rule name="setappname">
<match url=".*" />
<serverVariables>
<set name="CONTAINER_APP_NAME" value="desiredValue" />
</serverVariables>
</rule>
</rules>
</rewrite>
And in "applicationHost.config", I have the following snippets:
<sites>
<site name="mysite" id="1" serverAutoStart="true">
<application path="/" applicationPool=".NET v4.5">
<virtualDirectory path="/" physicalPath="c:\mysite" />
</application>
<bindings>
<binding protocol="http" bindingInformation="*:80:" />
</bindings>
<logFile directory="c:\iislog" period="MaxSize" truncateSize="4294967295">
<customFields>
<add logFieldName="x-forwarded-for" sourceName="X-Forwarded-For" sourceType="RequestHeader" />
<add logFieldName="container-app" sourceName="CONTAINER_APP_NAME" sourceType="ServerVariable" />
</customFields>
</logFile>
<applicationDefaults preloadEnabled="true" />
</site>
</sites>
AND
<system.webServer>
<rewrite>
<allowedServerVariables>
<add name="CONTAINER_APP_NAME" />
</allowedServerVariables>
</rewrite>
</system.webServer>
This works fine (I see the 2 custom fields in the logs) except when the Path ends with "/" (e.g.: / or /APath/). In those cases, the value of the container-app field (using Server Variable) is always "-". For instance:
$ curl --silent --output /dev/null -H "X-Forwarded-For:10.3.2.12" http://localhost/APath/
Yields:
2019-12-02 20:47:32 172.29.152.165 GET /APath/ - 80 - 192.168.7.4 curl/7.67.0 - 200 0 0 121 10.3.2.12,+::1 -
Whereas:
$ curl --silent --output /dev/null -H "X-Forwarded-For:10.3.2.12" http://localhost/home.aspx
Yields:
2019-12-02 20:50:17 172.29.152.165 GET /home.aspx - 80 - 192.168.7.4 curl/7.67.0 - 200 0 0 63 10.3.2.12,+::1 desiredValue
I even enabled the Failed Request Tracing to see if perhaps the rewrite rule isn't picking up those paths, but I can confirm that the rule matches the path and the server variable is set to the desired value.
I wonder if there is anything else I can try to troubleshoot this. Why such paths aren't logged properly?

I think I found the issue and posting it here for others.
By looking at the Failed Request Traces, I can see that IIS creates child requests for directories' default documents (URIs ending with "/"). Apparently, by design, rewrite rules don't apply to child requests (e.g.: https://forums.iis.net/t/1152699.aspx).
To solve this, I created a rewrite rule to change such requests to be explicit requests to documents so the other rewrite rule gets applied at the main process level:
<rewrite>
<rules>
<rule name="setExplictDoc">
<match url="(.*(APath)/$)" />
<action type="Rewrite" url="{R:0}Default.aspx" />
</rule>
<rule name="setappname">
<match url=".*" />
<serverVariables>
<set name="CONTAINER_APP_NAME" value="desiredValue" />
</serverVariables>
</rule>
</rules>
</rewrite>
The idea came from https://support.microsoft.com/en-ca/help/3050055/iis-digest-authentication-does-not-permit-pass-though-authentication-f

Related

Why IIS rewrite rule returns error when set server variable?

I added a new application to IIS and added a web.config file to rewrite:
https://mylocaldomain.com/ogc?a=2&b=3&c=3...
to
https://myactualdomain.com/ogc?a=2&b=3&c=3...
But http://myactualdomain.com requires Basic authentication requests.
So that my Web.config is like following:
<configuration>
<system.webServer>
<rewrite>
<rules>
<rule name="wms" stopProcessing="true">
<match url="^ogc?(.*)" />
<action type="Rewrite" url="http://myactualdomain.com/ogc?(.*)" />
<serverVariables>
<set name="HTTP_Authorization" value="Basic abcdefgasdas" />
</serverVariables>
</rule>
</rules>
<outboundRules>
<clear />
<rule name="wms">
<match serverVariable="RESPONSE_Access_Control_Allow_Origin" pattern=".*" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="true">
<add input="{HTTP_ORIGIN}" pattern="(.*)" />
</conditions>
<action type="Rewrite" value="{C:0}" />
</rule>
</outboundRules>
</rewrite>
</system.webServer>
</configuration>
This settigns returns error:
500 - Internal server error.
There is a problem with the resource you are looking for, and it cannot be displayed.
If I remove following lines, there is no error but a username and password popup appearing.
<serverVariables>
<set name="HTTP_Authorization" value="Basic abcdefgasdas" />
</serverVariables>
If I set any kind of serverVariables, 500 error occured. How can I solve this issue?
By default, the distributed rewrite rules (i.e. the rules that are defined for specific sites or web applications) cannot set or change any IIS server variable, unless the server variable is added to the "Allowed" list.
Select the "View Server Variables..." action from the "Actions" pane:
Use the "Add..." action to add the server variables HTTP_COOKIE and ORIGINAL_URI to the "Allowed Server Variables" list:
If the problem still exists, please use FRT to see the cause of this 500 error.

IIS10 url rewrite

I have a simple question which is driving me crazy from the last week: I splitted a big application in "Default Web Site" into single applications grouped by directory so everyone could have his own git repository. So the main root directory is quite empty, just few scripts linked from outasides statically.
All the user point to the root directory, and I would like redirect this pointing to the stable application in /subdir_stable and not showing the user this action. Every application in subfolders have an own prefix beginning from his local and works in every subfolder I put.
So I need two type of action:
- users point to myhostname.local
- action1: get redirect to myhostname.local/subdir_stable
- action2: keep having their urls as myhostname.local
I have achieved the action1 using the url rewrite with this web.config:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<rewrite>
<rewriteMaps>
<rewriteMap name="/spot_backend">
<add key="/" value="/spot_backend" />
</rewriteMap>
<rewriteMap name="/">
<add key="/" value="/spot_backend" />
</rewriteMap>
</rewriteMaps>
<rules>
<rule name="Rewrite rule1 for /">
<match url=".*" />
<conditions>
<add input="{/:{REQUEST_URI}}" pattern="(.+)" />
</conditions>
<action type="Rewrite" url="{C:1}" appendQueryString="false" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>
I don't how to achieve the action2. Any hints?

Setting up UpSource behind IIS Reverse Proxy for HTTPS ~ TypeError: Failed to fetch

I am trying to set up UpSource, along with YouTrack, TeamCity and Hub, over https on a single server using a reverse IIS Proxy.
The situation is as follows:
The http version of UpSource is located at http://server.company.com:8081/upsource and works fine. I want it to be accessible via https://server.company.com/upsource. However, while it is possible to access UpSource via the https address, the connection is immediately interrupted and the following error message comes up:
Backend is not available
TypeError: Failed to fetch
I find this error to be weird and confusing, considering that the backend appears to be available and running since http://server.company.com:8081/upsource works perfectly.
As for my configuration, I set it up mostly following the steps as outlined in the documentation, making amends where needed to account for the fact that we have four JetBrains services running on a single server and over the same IIS Reverse Proxy.
The current web.config for the IIS Proxy reads as follows:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<rewrite>
<rules>
<clear />
<rule name="Reverse Proxy to TeamCity" stopProcessing="true">
<match url="^teamcity(.*)" />
<action type="Rewrite" url="http://server.company.com{R:1}" />
</rule>
<rule name="Reverse Proxy to Hub" stopProcessing="true">
<match url="^hub(.*)" />
<action type="Rewrite" url="http://server.company.com:8082/hub{R:1}" />
<serverVariables>
<set name="HTTP_X_FORWARDED_HOST" value="{HTTP_HOST}" />
<set name="HTTP_X_FORWARDED_SCHEMA" value="https" />
<set name="HTTP_X_FORWARDED_PROTO" value="https" />
</serverVariables>
</rule>
<rule name="Reverse Proxy to YouTrack" stopProcessing="true">
<match url="^youtrack(.*)" />
<action type="Rewrite" url="http://server.company.com:8080/youtrack{R:1}" />
<serverVariables>
<set name="HTTP_X_FORWARDED_HOST" value="{HTTP_HOST}" />
<set name="HTTP_X_FORWARDED_SCHEMA" value="https" />
<set name="HTTP_X_FORWARDED_PROTO" value="https" />
</serverVariables>
</rule>
<rule name="Reverse Proxy to UpSource" stopProcessing="true">
<match url="^upsource(.*)" />
<action type="Rewrite" url="http://server.company.com:8081/upsource{R:1}" />
<serverVariables>
<set name="HTTP_X_FORWARDED_HOST" value="{HTTP_HOST}" />
<set name="HTTP_X_FORWARDED_SCHEMA" value="https" />
<set name="HTTP_X_FORWARDED_PROTO" value="https" />
</serverVariables>
</rule>
<rule name="Reverse Proxy to Collaboration General" stopProcessing="true">
<match url="(.*)" />
<action type="Rewrite" url="http://server.company.com/{R:1}" />
</rule>
</rules>
</rewrite>
<security>
<requestFiltering>
<requestLimits maxUrl="6144" maxQueryString="4096" />
</requestFiltering>
</security>
</system.webServer>
</configuration>
As I mentioned, this already works fine for TeamCity and Hub. However, for UpSource, there seems to be something still missing, which is likely related to the "TypeError: Failed to fetch". I've tried looking that up, but could not find any helpful information thus far .
If anyone has any ideas how to resolve this, I'd be more than happy to get additional input on this
Okay, so I figured out how to do this:
The above configuration of the web.config is actually correct. However, the following steps need to be performed in addition, and in the correct order:
NOTE: All commands beginning with hub.bat need to be performed on the hub.bat file in [Hub Installation Directory]\bin and all commands beginning with upsource.bat need to be performed on the upsource.bat file in [UpSource Installation Directory]\bin.
upsource.bat stop
hub.bat stop
hub.bat configure --listen-port 8082 --base-url https://server.company.com/hub
upsource.bat configure --listen-port 8081 --base-url=https://server.company.com/upsource --hub-url=https://server.company.com/hub/hub
hub.bat start
upsource.bat start --J-Dbundle.websocket.compression.enabled=false
NOTE: I don't know why, but Hub appends an extra /hub after its base address, that's why the hub-url setting for UpSource ends with /hub/hub.
After that, all I needed to do was add the redirection URL to the list of allowed redirection URLs for UpSource in Hub > Settings > Services > UpSource, and now it works perfectly.
Well, almost perfectly. Whenever the server gets restarted I need to manually restart UpSource since I've not yet figured out a way to register upsource as a service with the --J-Dbundle.websocket.compression.enabled=false parameter, but apart from that, everything works perfectly.

IIS URL Rewrite ~ Rewriting Paths to Ports

My situation is as follows:
We have an array of CI services, such as TeamCity, YouTrack and OctopusDeploy on a single server. Currently, we are accessing all of these via DNS Name plus port, e.g.:
TeamCity: http://server.company.com
YouTrack: http://server.company.com:1234
OctopusDeploy: http://server.company.com:5678/octopus
I'm currently adjusting that so we can access these services via https. For that purpose, I've set up an IIS Server as a Reverse Proxy by using IIS URL Rewrite to handle SSL authentication. This already works, and I can now access TeamCity via https://server.company.com
However, while I'm at it, I would also like to use IIS URL Rewrite to beautify the addresses, and get rid of the need to memorize port numbers. What I want to achieve are the following redirects:
https://server.company.com -> http://server.company.com (already working)
https://server.company.com/youtrack -> http://server.company.com:1234
https://server.company.com/octopus -> http://server.company.com:5678/octopus
I have already tried adding the following rules:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<rewrite>
<rules>
<clear />
<rule name="ReverseProxyInboundRule2" stopProcessing="true">
<match url="^(https://server.company.com/youtrack)(.*)$" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="false" />
<action type="Rewrite" url="http://server.company.com:1234/{R:2}" />
</rule>
<rule name="ReverseProxyInboundRule1" stopProcessing="true">
<match url="(.*)" />
<conditions logicalGrouping="MatchAny" trackAllCaptures="false">
<add input="{PATH}" pattern="youtrack" negate="true" />
</conditions>
<action type="Rewrite" url="http://server.company.com/{R:1}" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>
...and tested it in the IIS Manager, which correctly recognizes the rules and conditions, and also returns the correct backwards rules.
However, when I try to access https://server.company.com/youtrack, I am redirected to http://server.company.com/youtrack instead of http://server.company.com:1234 as I intended. It's like the ReverseProxyInboundRule2 is not evaluated at all.
I'm sure there's a logical explanation for this and a way to make it work. I just can't see it on my own. What can I do to make this work?
UPDATE 30-Mar-2018:
Okay, so I figured out a configuration that partially works. It goes as follows:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<rewrite>
<rules>
<clear />
<rule name="Reverse Proxy to TeamCity" stopProcessing="true">
<match url="^teamcity/(.*)" />
<action type="Rewrite" url="http://server.company.com/{R:1}" />
</rule>
<rule name="Reverse Proxy to YouTrack" stopProcessing="true">
<match url="^youtrack/(.*)" />
<action type="Rewrite" url="http://server.company.com:1234/issues/{R:1}" />
</rule>
<rule name="Reverse Proxy to Hub" stopProcessing="true">
<match url="^hub/(.*)" />
<action type="Rewrite" url="http://server.company.com.de:5678/hub/{R:1}" />
</rule>
<rule name="Reverse Proxy to UpSource" stopProcessing="true">
<match url="^upsource/(.*)" />
<action type="Rewrite" url="http://server.company.com.de:9876/{R:1}" />
</rule>
<rule name="Reverse Proxy to Octopus" stopProcessing="true">
<match url="^octopus/(.*)" />
<action type="Rewrite" url="http://server.company.com:5432/octopus/{R:1}" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>
This configuration redirects the requests as follows:
https://server.company.com -> http://server.company.com
https://server.company.com/youtrack -> http://server.company.com:1234/issues/
https://server.company.com/hub -> http://server.company.com:5678/hub/
https://server.company.com/upsource -> http://server.company.com:9876
https://server.company.com/octopus -> http://server.company.com:5432/octopus/
This works perfectly for Octopus.
On Hub, there is a "Connection is not secure" notification next to the address, saying that "Parts of the web site (such as images) are not secure". Going into the page info, it says in the Security tab under Technical Details that the Connection is only Partially Encrypted. However, the rest of the page appears to be working fine.
TeamCity and YouTrack are more whimsical. On Chrome, TeamCity works fine (for a while until it gets a random disconnect) but YouTrack delivers a 404 error, while on Firefox both TeamCity and YouTrack are displayed in an unusable "text-only" form.
UpSource, meanwhile, delivers a blank page on either browser.
I've cross-tested this on various browsers and machines and came to the conclusion that TeamCity, YouTrack and Hub only "sort of" work if I am already logged into the http versions of those services on those servers. If I am not logged in, then I get 404 errors for TeamCity and YouTrack. As for Hub, I get a 405 error on trying to log in with a "POST-Method not supported" message.
So the basic result of my above written configuration on a clean slate is as follows:
TeamCity: 404 Error
YouTrack: 404 Error
Hub: 405 Error on login
UpSource: Blank Page
Octopus: Working
Okay, after a lot of back and forth, I figured out a working configuration for all of these services:
Set up a http IIS Reverse proxy
The following modules need to be installed in IIS:
URL Rewrite (via https://www.iis.net/downloads/microsoft/url-rewrite)
Application Request Routing (via https://www.iis.net/downloads/microsoft/application-request-routing)
WebSocket Protocol (va Server Manager: Web Server (IIS) > Web Server > Application Development > WebSocket Protocol)
Dynamic Content Compression (via Server Manager: Web Server (IIS) > Web Server > Performance > Dynamic Content Compression)
Create a website for the URL redirect
Create a https binding for that website using a valid certificate
Disable Dynamic Content Compression under [Server] > Compression
Go to [Server] > Application Request Routing > Server Proxy Settings and select ☑ “Enable Proxy”
Make sure the following values are set:
HTTP Version: Pass Through
Time-out: 120
Preserve client Ip for the following header: X-Forwarded-For
Memory Cache Duration: 60
Query String Support: Ignore Query String
Response Buffer: 4096
Response Buffer Threshold: 0
Set up the URL rewrite and related settings
Add the following server variables in [Server] > URL Rewrite > View Server Variables
HTTP_X_FORWARDED_HOST
HTTP_X_FORWARDED_SCHEMA
HTTP_X_FORWARDED_PROTO
Configure the web.config as follows:
web.config:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<rewrite>
<rules>
<clear />
<rule name="Reverse Proxy to TeamCity" stopProcessing="true">
<match url="^teamcity(.*)" />
<action type="Rewrite" url="http://server.company.com{R:1}" />
</rule>
<rule name="Reverse Proxy to Hub" stopProcessing="true">
<match url="^hub(.*)" />
<action type="Rewrite" url="http://server.company.com:8082/hub{R:1}" />
<serverVariables>
<set name="HTTP_X_FORWARDED_HOST" value="{HTTP_HOST}" />
<set name="HTTP_X_FORWARDED_SCHEMA" value="https" />
<set name="HTTP_X_FORWARDED_PROTO" value="https" />
</serverVariables>
</rule>
<rule name="Reverse Proxy to YouTrack" stopProcessing="true">
<match url="^youtrack(.*)" />
<action type="Rewrite" url="http://server.company.com:8080/youtrack{R:1}" />
<serverVariables>
<set name="HTTP_X_FORWARDED_HOST" value="{HTTP_HOST}" />
<set name="HTTP_X_FORWARDED_SCHEMA" value="https" />
<set name="HTTP_X_FORWARDED_PROTO" value="https" />
</serverVariables>
</rule>
<rule name="Reverse Proxy to UpSource" stopProcessing="true">
<match url="^upsource(.*)" />
<action type="Rewrite" url="http://server.company.com:8081/upsource{R:1}" />
<serverVariables>
<set name="HTTP_X_FORWARDED_HOST" value="{HTTP_HOST}" />
<set name="HTTP_X_FORWARDED_SCHEMA" value="https" />
<set name="HTTP_X_FORWARDED_PROTO" value="https" />
</serverVariables>
</rule>
<rule name="Reverse Proxy to Octopus" stopProcessing="true">
<match url="^octopus(.*)" />
<action type="Rewrite" url="http://server.company.com:8888/octopus{R:1}" />
</rule>
<rule name="Reverse Proxy to Collaboration General" stopProcessing="true">
<match url="(.*)" />
<action type="Rewrite" url="http://server.company.com/{R:1}" />
</rule>
</rules>
</rewrite>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="300000000" maxUrl="6144" maxQueryString="4096" />
</requestFiltering>
</security>
</system.webServer>
</configuration>
Configure Hub, YouTrack and UpSource as follows
NOTE: The commands beginning with hub.bat / youtrack.bat / upsource.bat need to be performed on the respective file in [Installation Directory]\bin of the respective service.
cmd:
upsource.bat stop
youtrack.bat stop
hub.bat stop
hub.bat configure --listen-port 8082 --base-url https://server.company.com/hub
youtrack.bat configure --listen-port 8080 --base-url=https://server.company.com/youtrack--hub-url=https://server.company.com/hub/hub
upsource.bat configure --listen-port 8081 --base-url=https://server.company.com/upsource --hub-url=https://server.company.com/hub/hub
hub.bat start
youtrack.bat start
upsource.bat start --J-Dbundle.websocket.compression.enabled=false
NOTE: I don't know why, but Hub appends an extra /hub after its base address, that's why the hub-url setting for UpSource ends with /hub/hub.
After that, all I needed to do was add the redirection URLs to the list of allowed redirection URLs for the services and change the basis urls for TeamCity and OctopusDeploy in Hub > Settings > Services, and now it all works perfectly.
Well, almost perfectly. Whenever the server gets restarted I need to manually restart UpSource since I've not yet figured out a way to register upsource as a service with the --J-Dbundle.websocket.compression.enabled=false parameter, but apart from that, everything works perfectly.

Rewriting a URL in an Azure web app

I have a simple wildcard routing rule I want to apply for my Azure web app.
<rule name="MyRule">
<match url="*" />
<action type="Rewrite" url="/index.html" />
</rule>
Do I have any option here given I can't RDP into the machine and fiddle with IIS? This is not an ASP.Net website, it's a simple SPA application.
You need to create a web.config file in your wwwroot folder and put the relevant config entries there.
Here's an example of an web.config rule, to give you an idea of what it should look like.
The below example redirect the default *.azurewebsites.net domain to a custom domain (via http://zainrizvi.io/blog/block-default-azure-websites-domain/)
<configuration>
<system.webServer>
<rewrite>
<rules>
<rule name="Redirect rquests to default azure websites domain" stopProcessing="true">
<match url="(.*)" />
<conditions logicalGrouping="MatchAny">
<add input="{HTTP_HOST}" pattern="^yoursite\.azurewebsites\.net$" />
</conditions>
<action type="Redirect" url="http://www.yoursite.com/{R:0}" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>
If simply want all URL's that resolve to this server & site to redirect to index.html you could use this rewrite section:
<configuration>
<system.webServer>
<rewrite>
<rules>
<rule name="SPA">
<match url=".*" />
<action type="Rewrite" url="index.html" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>
This is very similar to what you have except some minor syntax fixes e.g. the pattern should be ".*" and the rewrite URL target simply "index.html".
Note this means that ALL URL's to your site will be rewritten, even for other resources like CSS and JS files, images etc. So you'd better be fetching your resources from other domains.
If you want to do actual rewrites (not redirects), dont forget enabling ARR with applicationHost.xdt file put to the site folder with the following content:
<?xml version="1.0"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<system.webServer>
<proxy xdt:Transform="InsertIfMissing" enabled="true" preserveHostHeader="false" reverseRewriteHostInResponseHeaders="false" />
<rewrite>
<allowedServerVariables>
<add name="HTTP_ACCEPT_ENCODING" xdt:Transform="Insert" />
<add name="HTTP_X_ORIGINAL_HOST" xdt:Transform="Insert" />
</allowedServerVariables>
</rewrite>
</system.webServer>
</configuration>

Resources