From the server logs i have recently noticed that googlebot getting http status 200 response for non existing URI
https://turkishclassified.com/?KKId=3914
https://turkishclassified.com/?KKId=4031
https://turkishclassified.com/?KKId=2890
However non-existent other URI without the /? returns the http status 404 and is correct response.
https://turkishclassified.com/KKId=3914
https://turkishclassified.com/KKId=4031
https://turkishclassified.com/KKId=2890
I am running Plesk that has htaccess file. I also have the following htaccess in public folder by theme developers
# Handle Front Controller...
RewriteEngine On
<IfModule mod_headers.c>
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ index.php [L]
</IfModule>
How can we give correct http status response for existing and non-existing URI. Thks for any replies.
I have added the following to https://example.com/robots.txt that google follows
Disallow: /?KKId=
Disallow: /?bid=
Disallow: /?Ino=
Disallow: /en/search?KKId=
Disallow: /search?KKId=
This has worked so far for the crawling part.
Related
I have written a .htaccess file to make sure that visitors are always redirected to https and to www. In addition to that I have also added a custom html page for 404 errors.
When visitors try to access a forbidden file I want them to see my custom 404 message, as to not reveal that the path contains a forbidden file.
Here is the problem. When writing for example "example-domain.com/.htaccess" (no www or https) in the browser, the URL in the address field in the browser changes to "https://www.example-domain.com/missing.html". But I want it to say "https://www.example-domain.com/.htaccess" while displaying my 404 page.
It works for 404 errors. But when typing in a path in the address field which both triggers the 403 error and fulfill at least one of the rewrite conditions in my .htaccess file (missing https and/or www) I experience the above described problem.
Here is the code in the .htaccess file:
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule .* https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
RewriteCond %{HTTP_HOST} ^example-domain.com [NC]
RewriteRule ^(.*)$ https://www.example-domain.com/$1 [L,R=301,NC]
ErrorDocument 403 /missing.html
ErrorDocument 404 /missing.html
Best regards
I have copied a site that that sits under www.domain.co.uk onto a new, testing server which has the url http://c4034105.myzen.co.uk (ultimately we are going to point www.domain.co.uk at this new version)
I am finding that redirects don't appear to work. Here is part of the .htaccess file
RewriteEngine On
ErrorDocument 404 /error.php
#CONTACT
RewriteRule ^contact$ contact.php [L]
RewriteRule ^contact/$ contact.php [L]
The first 404 redirect is working.
However, when the url http://c4034105.myzen.co.uk/memorials/contact is generated we want it to redirect to http://c4034105.myzen.co.uk/contact but it isn't. When it was under its domain.co.uk it worked fine.
What's wrong please
Thanks
the rules you posted only add the .php extension to http://www.example.com/contact, there's no rule which redirects http://example.com/memorials/contact to http://example.com/contact.
here's one:
RewriteRule ^memorials/contact/?$ /contact [L]
I want to redirect the links which are bad, getting 404 errors.
Let's say I have the following url:
http://www.example.com/foobar
If the above url is bad, I want it to redirect it with .html suffix
http://www.example.com/foobar.html
So basically I want to make sure URL is bad and it doesn't end with .html, then only apply the rule, which is same URL with .html suffix.
UPDATE: It's not just foobar.html, there are thousands of pages dynamic, so simple use of /foobar.html to redirect will not work.
Examples of what i am looking for
below url will go to standard 404 page
http://www.example.com/this-is-bad.html
below url will go to it's .html page
http://www.example.com/this-is-bad
below url will not redirect, because of slash at end
http://www.example.com/foobar/
below url is working and SHOULD NOT get redirected anywhere. This is not a file but a dynamic page.
http://www.example.com/this-is-good
Thanks
Just use ErrorDocument 404:
ErrorDocument 404 /foobar.html
If you want full redirect (change URL in broser) then useL
ErrorDocument 404 http://www.example.com/foobar.html
This sill redirect all 404 URLs to /foobar.html
UPDATE: Try this rule
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^./]+)/?$ /$1.html [L,R=302]
First of all, my ErrorDocument 404 /v1 is never redirecting, but if I look in the Chrome Inspector, I really have a 404 error. At first I thought it was because I really needed to specify a file ex: error.html, but that didn't work either. /v1 is just a wordpress subdomain, and I want to redirect all of my errors there. /v1 falls on my homepage of my subdomain.
Also I would like to make the error page dynamic, but I can't figure how to do that. Something like that:
RewriteCond %{REQUEST_FILENAME} public_html/(.*)
ErrorDocument 404 /v1/%1
Any ideas? Thank you!!
You're goinmg to need to return the 404 through a php script:
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ /404.php?path=$1 [L]
And in the 404.php file:
<?php
header("HTTP/1.0 404 Not Found");
print('The file that you requested: ' . $_GET['path'] . ' was not found\n');
// or include a special 404 page
include ('/path/to/' . $_GET['path']);
?>
Or you could lose the ?path=$1 part completely and just look in $_SERVER['REQUEST_URI'].
This syntax is incorrect:
RewriteCond %{REQUEST_FILENAME} public_html/(.*)
ErrorDocument 404 /v1/%1
Because ErrorDocument has no connection with RewriteCond (mod_rewrite) and it is unconditional.
Conditional 404 handling example:
ErrorDocument 404 /404.php
RewriteRule ^foo/ - [L,NC,R=404]
I use this syntax! Usually works!
ErrorDocument 404 "<script>window.location.href='404.php?page='+window.location.href.split('/')[window.location.href.split('/').length-1];</script>"
I'm having a small problem with my htaccess files. Currently, it redirects everything back to index.php for processing, except when someone tries to access an actual directory. When the directory exists, it displays the 403 error page instead of rewriting the path to index.php like it's supposed too. What can I modify to make it always go to index.php when the files are accessed via the internet, but still load the correct files in PHP on the server?
Options -Indexes +FollowSymLinks
RewriteEngine On
RewriteBase /
ErrorDocument 403 /index.php?403
ErrorDocument 404 /index.php?404
ErrorDocument 414 /index.php?414
RewriteCond %{HTTP_HOST} !^$ [NC]
RewriteRule !^(.*) index.php [L]
Sample file structure:
Web Directory
- com
- source
- index.php
- TEST.HTML
The folders such as 'com' and source' will display the 403 because the user doesn't have access to them. The files such as 'index.php' and 'TEST.HTML' execute normally. I want my htaccess to redirect everything in the Web Directory folder back to the index.php file, no matter what.
I think you want this instead:
Options -Indexes +FollowSymLinks
RewriteEngine On
RewriteBase /
ErrorDocument 403 /index.php?403
ErrorDocument 404 /index.php?404
ErrorDocument 414 /index.php?414
RewriteCond %{REQUEST_URI} !^/index.php
RewriteRule .* index.php [L]
This was on the assumption that you didn't want to be able to access TEST.HTML directly and didn't want to change the URL in the user's browser. If either of those assumptions were wrong, let me know and I'll update the answer with the appropriate rewrite information.