I want to automatically add the X-Robots-Tag header for all php, htm, html pages EXCEPT /index.php and forgot.php.
This would cover all of them:
<FilesMatch "\.(htm|html|php)$">
<IfModule mod_headers.c>
Header set X-Robots-Tag "noindex, nofollow"
</IfModule>
</FilesMatch>
But how can I exclude /index.php and /forgot.php from that FilesMatch directive?
What I want it to do :
Valid for all .htm, .html, .php files
Exclude for /index.(htm|html|php), /forgot.(htm|html|php), but not */index.php should be valid.
Hopefully that makes sense... I just want to exclude it from those two specific files at the base of the site.
UPDATE:
Playing around with this on tester, but still have some issues :
(?!.*/(index.php|forgot))(^.*\.(php|htm|html)$)
this is excluding URLs like www.mysite.com/folder/index.php
I was looking at this wrong the whole time. Here is what I am using :
# BEGIN noindex,nofollow on all but login and forgot page
<IfModule mod_env.c>
RewriteRule ^(index\.php|forgot\.php)$ - [E=exclude:1]
<FilesMatch "\.(php|html?)$">
<IfModule mod_headers.c>
Header set X-Robots-Tag "noindex, nofollow" env=!exclude
</IfModule>
</FilesMatch>
</IfModule>
What have you tried? Have you tried something like this?
<FilesMatch "^(^index|^forgot)\.(htm|html|php)$">
Just want to point you in the right direction. I haven't tried this yet, but it's probably going to be a bit greedy and match files like index_example.php as well. Not just index.php.
These guys have an excellent reference on RegEx: http://www.regular-expressions.info/
Related
We are trying to solve language redirection "the right way" over at https://guestbell.com/. Some idioms first:
a) Each route has a starting URL parameter that identifies the language. e.g. https://guestbell.com/en for English and https://guestbell.com/es for Spanish. There are also https://guestbell.com/en/pricing etc.
b) You can also omit this parameter, e.g. https://guestbell.com/pricing . The language is then detected (cookie, browser-language, qs param or URL param) and added to the URL. Page is SPA in react, the detection is done by i18next library.
c) Every possible page is pre-rendered in HTML files that are served via static server.
Note that because the routes are pre-rendered, routes like https://guestbell.com/pricing doesn't in fact exist in the folder structure (because it's impossible to guess the language prior to front end detection)
What works so far:
You navigate to guestbell.com
You are redirected to https via htaccess
If the file is found, serve it.
If the file is not found, serve a PHP file that is written as follows:
<?php
$cookieName = "i18next";
$path = rtrim(strtok($_SERVER["REQUEST_URI"], '?'), '/');
$supportedLangs = [
'en',
'es',
];
$defaultLang = $supportedLangs[0];
$lang = substr($_SERVER['HTTP_ACCEPT_LANGUAGE'], 0, 2);
if(isset($_COOKIE[$cookieName])) {
$lang = $_COOKIE[$cookieName];
}
$finalLang = $lang;
if (!in_array($lang, $supportedLangs)) {
$finalLang = $defaultLang;
}
$newPath = $finalLang . $path . '.html';
if (file_exists($newPath) || empty($path)) {
$newPath = $finalLang . $path;
header("Location: $newPath", true, 302);
} else {
$newPath = $finalLang . '/404';
header("Location: $newPath", true, 302);
}
?>
As you can see, it attempts to detect via cookie or browser language (we know that by this point, the URL param is not present)
This approach works fine but there is one issue.
When navigating to guestbell.com (as most people would), this results into 2 redirects:
HTTP => HTTPS
/ => /en
Ideally, I would like to eliminate this added overhead and do it in one redirect. The only way (that I can imagine at the moment) is to do it via htaccess. The issue is I have no idea if this is possible.
This is the current htaccess for completion sake:
ErrorDocument 404 /404.html
#This is extremely important as it disables rewriting route from en => en/ and then 403-ing on directory
#https://stackoverflow.com/questions/28171874/mod-rewrite-how-to-prioritize-files-over-folders
DirectorySlash Off
# BEGIN WWW omit
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^ %{REQUEST_SCHEME}://%1%{REQUEST_URI} [R=301,L]
</IfModule>
# END WWW omit
# BEGIN HTTPS redirect
<IfModule BEGIN HTTPS redirectfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{ENV:HTTPS} !=on
RewriteRule ^.*$ https://%{SERVER_NAME}%{REQUEST_URI} [R,L]
</IfModule>
# END HTTPS redirect
# BEGIN Omit extension
<ifModule mod_rewrite.c>
#remove html file extension-e.g. https://example.com/file.html will become https://example.com/file
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^\.]+)$ $1.html [NC,L]
</ifModule>
# END Omit extension
# BEGIN File detection
<ifModule mod_rewrite.c>
RewriteEngine On
# If an existing asset or directory is requested go to it as it is
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f [OR]
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d
RewriteRule ^ - [L]
# If the requested resource doesn't exist, use index.php - that file then takes care of language redirection
RewriteRule ^ /index.php
</ifModule>
# END File detection
# BEGIN Compress text files
<ifModule mod_deflate.c>
<filesMatch "\.(css|js|x?html?|php)$">
SetOutputFilter DEFLATE
</filesMatch>
</ifModule>
# END Compress text files
# BEGIN Cache
<ifModule mod_headers.c>
<filesMatch "\\.(ico|pdf|flv|jpg|jpeg|png|gif|swf|svg|mp4)$">
Header set Cache-Control "max-age=31536000, public"
</filesMatch>
<filesMatch "\\.(css)$">
Header set Cache-Control "max-age=31536000, public"
</filesMatch>
<filesMatch "\\.(js)$">
Header set Cache-Control "max-age=31536000, private"
</filesMatch>
<filesMatch "\\.(xml|txt)$">
Header set Cache-Control "max-age=2592000, public, must-revalidate"
</filesMatch>
<filesMatch "\\.(html|htm|php)$">
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
</filesMatch>
<filesMatch "sw.js$">
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
</filesMatch>
</ifModule>
# END Cache
An alternative would be to leave the language detection to front-end in such cases, and thus losing the prerendering altogether. I don't like this too much as majority of people would navigate to root of the page instead of /en and therefore lose performance. But what worries me is that the performance will be lost anyways due to multiple redirect.
My question stands:
Is it possible to do cookie and browser-language redirection combined with HTTP => HTTPS inside htaccess? If so, could you provide any help in achieving such functionality? If not, could you share the best way of achieving this, or optionally verify that our approach using PHP is "good enough"?
Many thanks.
i've a htaccess where i want to
- hide contents of folders -> this one ok
- redirection where we're in the wrong link -> this one is ok
- solve the problem of validator with the chrome meta -> this one i'm not sure
Here is my htacces, is it correct ?
options -indexes
ErrorDocument 404 /404/index.php
<FilesMatch "\.(htm|html|php)$">
<IfModule mod_headers.c>
BrowserMatch MSIE ie
Header set X-UA-Compatible "IE=Edge,chrome=1" env=ie
</IfModule>
</FilesMatch>
On my development server running xampp on windows my .htacess rewrite rules are working fine. Once we went to our live server which is running Linux core 3.8.0-21-generic #32-Ubuntu SMP Server version: Apache/2.2.22 (Ubuntu), our rules which do not contain parameters no longer work, yet rules which do have parameters are working.
Options -Indexes
<filesMatch "\.(html|htm|txt|js|htaccess)$">
FileETag None
<ifModule mod_headers.c>
Header unset ETag
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires "Wed, 11 Jan 1984 05:00:00 GMT"
</ifModule>
</filesMatch>
ErrorDocument 404 /404.php
RewriteEngine On
#Main site rules
RewriteRule ^login/?$ login.php [NC,L]
RewriteRule ^contact/?$ contact.php [NC,L]
The above rules which go to contact.php and login.php do not work. But, this more complicated rule with parameters is working:
RewriteRule ^game/([a-zA-Z0-9]+)/?$ handles/handle-game-select.php?name=$1 [NC,L]
Is there differences between the two server environments which is causing this to occur?
Also, it appears that if we do something strange such as: RewriteRule ^contact.x contact.php [NC,L] we are able to reach contact.php...
Very confused on this one.
Thank you for any help.
I suspect that is due to enabling of MultiViews option. Add this line on top to disable it:
Options -MultiViews
Option MultiViews is used by Apache's content negotiation module that runs before mod_rewrite and and makes Apache server match extensions of files. So /file can be in URL but it will serve /file.php.
In my social engine web site , it always shows mydomain.com/index.php instead of mydomain.com
For example when we access the member page it shows mydomain.com/index.php/member , but actually I would like to get it as mydomain.com/members
Please see my htacess code under /public folder here
<FilesMatch "\.(avi|flv|mov|wmv|wma|mp3|mp4|m4a|mkv|swf|pdf|doc|ppt|ico|jpg|jpeg|png|gif|js|css)$">
<IfModule mod_expires.c>
ExpiresActive On
ExpiresDefault A29030400
<IfModule mod_headers.c>
Header append Cache-Control "public"
</IfModule>
</IfModule>
<IfModule !mod_expires.c>
<IfModule mod_headers.c>
Header set Expires "Sun, 1 Jan 2012 00:00:00 GMT"
Header append Cache-Control "max-age=172800, public"
</IfModule>
</IfModule>
</FilesMatch>
How can I solve this ?
Thanks in advance
Sunil
Why your .htaccess is inside /public folder? It should be inside a root socialengine folder, where .htaccess enables rewrite mode...
I try to make my site run with seo when i access
mysite.com/search/eminem/1/video.html
to give me the results for this url
mysite.com/index.php?search=eminem&page=1&type=video
And it show me the search page but with no results no images or javascripts etc. Is like css and js's are not implemented
this is my .htacces code
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteRule ^search/(.*)/(.*)/(.*).html?$ index.php?search=$3&page=$2&type=$1 [L]
<IfModule mod_deflate.c>
<FilesMatch "\.(php|js|css|mp3|wmv|flv|html|htm)$">
SetOutputFilter DEFLATE
</FilesMatch>
</IfModule>
</IfModule>
You either need to make all your relative links (for images, CSS, JavaScript, etc) to absolute links with a leading slash, or add this to the header of your index.php content:
<base href="/">