How do I use a different robots.txt for https request than the one that is used for http connections in IIS 7?
Thanks.
There are a few options here, depending on how custom you need this to be. Most flexible approach would be to write a handler and map to it for robots request, and handle internally.
However, for most needs, try to URL rewrite module
http://www.iis.net/download/urlrewrite
off top of my head (aka prob doesnt work 100%), its something like :
<rule name="https_robots">
<match url="(.*)" />
<conditions>
<add input="{HTTPS}" pattern="off" ignoreCase="true" />
</conditions>
<action type="Redirect" redirectType="Found" url="https://{HTTP_HOST}/robots-https.txt" />
</rule>
Got exactly the same problem:
I am setting up https version now, want to look at it and debug it without a rush, meanwhile fending off Google from crawling https, but keeping http "business as usual". Here's the piece of code I used in a config file:
<rule name="https_robots" stopProcessing="true">
<match url="^robots\.txt$" />
<conditions>
<add input="{HTTPS}" pattern="^OFF$" />
</conditions>
<action type="Rewrite" url="robots_https.txt" />
</rule>
A few notes:
Basically, I've created a file "robots_https.txt" containing "disallow for all" instruction.
I used Rewrite, not Redirect. I am not sure if the spider would get the redirect. I really doubt that. With Rewrite you can't go wrong.
Hope that helps!
Related
We have a bunch of websites hosted in IIS using Flex. There are quite a lot of Rewrite/Redirect rules for each of them. One of them now needs a Rewrite rule to direct a non-www url to a www url. Have already tried quite a few popular solutions from various sites including other SO question. The last I tried is as following, without much to avail:
<rule name="Redirect to WWW" stopProcessing="true">
<match url=".*" />
<conditions>
<add input="{HTTP_HOST}" pattern="^your.domain.name$" />
</conditions>
<action type="Redirect" url="http://www.your.domain.name/{R:0}" redirectType="Permanent" />
</rule>
Ensure you have the latest URL Rewrite extension.
https://www.iis.net/downloads/microsoft/url-rewrite
Before testing the result, please clean the local cache. We had better verify it in an inprivate window.
Besides, Failed Request Tracing is a powerful tool for troubleshooting request-processing failures. FRT can be used with the URL rewrite module to trace how rewrite rules were applied to the request URL.
https://learn.microsoft.com/en-us/iis/extensions/url-rewrite-module/using-failed-request-tracing-to-trace-rewrite-rules
At last, please try the below configuration.
<rule name="Force www" enabled="true" stopProcessing="false">
<match url="(.*)" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="false">
<add input="{HTTP_HOST}" pattern="^example\.com$" />
</conditions>
<action type="Redirect" url="https://www.example.com{REQUEST_URI}" />
</rule>
Feel free to let me know if the problem still exists.
I've got the following redirect rule below to force all http traffic to https. However, we need to serve some pages that have a problem displaying in https and we need to force them to be http until we can get them updated to work under https.
<rule name="Redirect to https" stopProcessing="true">
<match url=".*" />
<conditions>
<add input="{HTTPS}" pattern="off" ignoreCase="true" />
</conditions>
<action type="Redirect" url="https://{HTTP_HOST}{REQUEST_URI}" redirectType="Permanent" appendQueryString="false" />
</rule>
One thing all these pages have in commons in that "/SSLA/" is part of the url. I'm not the rewrite guru I would like to be, but there are a couple of potential ways I can think of to fix this:
a) Somehow update this rule to exclude urls with "/SSLA/".
B) Add a rule before this rule that catches anything with "/SSLA/" and redirects to http if it's going to https and (either in the same rule or a separate rule?) also does nothing for http requests but stops processing so it doesn't hit this global http to https redirect rule.
Problem is, I'm not great with either regex or rewrite rules so I'm not quite sure how to accomplish either of those solutions (actually not even sure even either are the "proper" way to handle this).
So... please help! :)
EDIT: I should note that I have updated the c# code to redirect to http for these pages, but of course that causes a loop with the above rule. But I just point this out to say that all I need is for the redirect rules not to force http requests for pages with "/SSLA" to https since the code will redirect any such https requests to http. But if it's a trivial matter to also have the redirect rule force https requests with "/SSLA/" in them to http then I could remove the c# code that does the same.
try use
<match url="((?!SSLA).)*" />
insteed
<match url=".*" />
You should use two rewrite rules:
For redirecting HTTPS -> HTTP for SSLA/* pages
For redirecting HTTP -> HTTPS for non SSLA/* pages.
Check my example below how I am filtering SSLA and non SSLA pages
<rule name="Redirect to http SSLA" stopProcessing="true">
<match url="^SSLA" />
<conditions>
<add input="{HTTPS}" pattern="on" ignoreCase="true" />
</conditions>
<action type="Redirect" url="http://{HTTP_HOST}{REQUEST_URI}" redirectType="Permanent" appendQueryString="false" />
</rule>
<rule name="Redirect to https" stopProcessing="true">
<match url="^SSLA" nagate="true" />
<conditions>
<add input="{HTTPS}" pattern="off" ignoreCase="true" />
</conditions>
<action type="Redirect" url="https://{HTTP_HOST}{REQUEST_URI}" redirectType="Permanent" appendQueryString="false" />
</rule>
I need to redirect all https requests to http, for example, if someone visits https://www.example.com/another-page/ to http://www.example.com/another-page/
I have the following rewrite rule in my web.config right now, but it's not working correctly. It's redirecting https://www.example.com/another-page/ to https://www.example.com/, so to the root of the site, but instead I want the redirect to stay in the same URL and only rewrite https to http.
<rule name="Redirect to HTTP" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{R:1}" pattern="^onepage/(.*)$" negate="true" />
<add input="{HTTPS}" pattern="^ON$" />
</conditions>
<action type="Redirect" url="http://{HTTP_HOST}/{R:0}" redirectType="Permanent" />
</rule>
Any help on changing the above rule so that it only changes https to http, but keeps the full url visited would be greatly appreciated!
I set up your rule, cleaned up a little, and it worked; so this isn't really answering with much new.
Suggestion: Remove the onepage input condition just for testing, as cheesmacfly suggested in the question comment.
Also, try changing the action to {R:1} instead of {R:0}. It shouldn't matter in this case, but I just like using 1 and up, to match the specific capturing group. R:0 means the entire matched string, which always confuses me just a little.
<rule name="Redirect to HTTP" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{HTTPS}" pattern="^ON$" />
</conditions>
<action type="Redirect" url="http://{HTTP_HOST}/{R:1}" redirectType="Permanent" />
</rule>
One possibility is that your browser has cached a previous attempt of your rules. When the redirectType is Permanent, and you're still developing or testing, the browser often caches a previous rule. Clear your browser cache, and/or remove the Permanent, and/or browse in incognito mode. When done testing, change it to permanent. See number 2 and 3 in this answer: https://stackoverflow.com/a/9204355/292060
Please paste the below code in web.config file.
<rule name="Redirect to http" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{HTTP}" pattern="off" ignoreCase="true" />
</conditions>
<action type="Redirect" url="http://{HTTPS_HOST}{REQUEST_URI}" redirectType="Permanent" appendQueryString="false" />
</rule>
So I have been trying to rewrite using the following:
<rules>
<rule name="HttpToHttps" stopProcessing="true"/>
<match url="(.*)"/>
<conditions>
<add input="{HTTPS}" pattern="off" ignoreCase="true"/>
</conditions>
<action type="Redirect" url="https://{HTTP_HOST}{URL}"/>
</rule>
<rules>
In two environments, this redirects like it should. In a third environment, it doesn't work. It doesn't work in the sense of, looking at a fiddler log, I see the http call with the full url. When it redirects to https, it removes everything after the HTTP_HOST.
So using the following url:
nonsecure://www.mysite.com/page.aspx?var1=1&var2=2
In two environments, it becomes
secure://www.mysite.com/page.aspx?var1=1&var2=2
In the third it becomes:
secure://www.mysite.com
I tried rewriting it as https://{HTTP_HOST}{HTTP_URL} and that doubled the {URL} in the first two environments:
secure://www.mysite.com/page.aspx?var1=1&var2=2&var1=1&var2=2
I'm not very experienced with web.configs, kinda learning as I go, so if anyone has any input as to what's going on here, it would be greatly appreciated. If it has any bearing on anything, the third environment is on two servers that are load balanced.
Ruslany has several IIS URL Rewrite examples on his blog, HTTP to HTTPS is one of them:
<rule name="Redirect to HTTPS" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{HTTPS}" pattern="^OFF$" />
</conditions>
<action type="Redirect" url="https://{HTTP_HOST}/{R:1}" redirectType="Permanent" />
</rule>
See http://ruslany.net/2009/04/10-url-rewriting-tips-and-tricks/ for more.
I'm trying to rewrite some urls and i've done this in the past but for some reason it just won't stick this time. Here's the rule:
<rule name="Force HTTPS - Test.aspx" patternSyntax="ECMAScript" stopProcessing="true">
<match url="^Templates/Test\.aspx" />
<conditions>
<add input="{HTTP_HOST}" pattern="^my\.domain\.com$" />
<add input="{HTTPS}" pattern="^OFF$" />
</conditions>
<action type="Redirect" url="https://my.domain.com/Templates/test.aspx" />
</rule>
It should be taking http://my.domain.com/Templates/test.aspx and redirecting to https://my.domain.com/Templates/test.aspx.
Unfortunately it's not working at all, no matter what i try....
I figured out that IIS was configured to force SSL on the files I tried redirecting. That meant it would return an HTTP 403 error before even checking the url rewrite rules.
Hope that helps someone else.