I want to block in the htaccess URL file (https://www.example.com/files/example/index.html) for Google robots. Searching through the search engine I found something like:
RewriteEngine on
RewriteCond% {HTTP_USER_AGENT} Googlebot [OR]
RewriteRule ^. * $ "Http \: \ / \ / yordomain \ .com" [R = 301, L]
This file is for the domain, and I want to block a specific URL, how can I save it?
Why would you want to use it in htaccess??
Try using robots.txt Place the file with the code in the public_html folder.
User-agent: *
Disallow: /path/to/the/page/rate
Else you can make an, .htaccess file and save it in the public_html folder.
Check this post How to block Googlebot from accessing one specific page
Related
I have a site where subdirectories are generated automatically, and need to be indexable under .htaccess
However, I don't want someone to be able to go to the root of these subdirectories and to view them all. I also don't want anything other than these subdirectories to be indexable.
E.g.
/ ~ Has "Options -Indexes" (Non-Indexable)
/foo/ ~ Has "Options -Indexes" (Non-Indexable)
/foo/bar/ ~ Has "Options +Indexes" (Indexable)
/foo/baz/ ~ Has "Options +Indexes" (Indexable)
It's not possible for me to generate an individual .htaccess file for every subdirectory individually, the system I'm using doesn't support it.
I'm assuming there's no other way to solve this problem (without possibly using the Apache config), so I'm just allowing the index recursively (i.e. from /foo), then inside /foo 's .htaccess file:
Options +Indexes
RewriteEngine on
RewriteCond %{REQUEST_URI} ^/foo[/]?$
RewriteRule (.*) / [R=301,L]
To just redirect people to the homepage. Also put a meta refresh redirect in there just incase.
I have a WordPress website with the basic structure: the URL keyword separator symbols are /. The problem is that the pages I create can be accessed using the / or + symbols in the URL.
I mean, I can access the same page in mydomain.com/example-page/ and mydomain.com/example+page/. I know that this is harmful for SEO so I make a question: is it possible to set, via htaccess, a noindex nofollow order to all the pages that uses the + symbol separator in the URL?
If you have a better solution, I will be grateful!
You can use (before your actual htaccess code):
RewriteEngine on
# executes repeatedly as long as there are more than 1 spaces in URI
RewriteRule "^(\S*)\s+(\S*\s.*)$" /$1-$2 [L,NE]
# executes when there is exactly 1 space in URI
RewriteRule "^(\S*)\s(\S*)$" /$1-$2 [L,R=302,NE]
Who redirects version example+page to example-page
TL;DR: My URL rewriting breaks if a subdirectory without a file is requested. Instead of loading a default home page as expected, I'm getting a 403 Forbidden error.
I'm using Apache URL rewriting to build a site using the Front Controller pattern. So far, my .htaccess looks like this:
Options -Indexes
RewriteCond %{REQUEST_URI} ^/(subdir|subdir/.*|subdir2|subdir2/.*).*$
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+) index.php?request=$1 [QSA]
So this is working for all requests except requests for subdirectories:
mydomain.com/ results in ->
mydomain.com/index.php with home.inc content
mydomain.com/page1 results in ->
mydomain.com/index.php with /pages/page1.inc content
mydomain.com/subdir/ results in ->
403 Forbidden
mydomain.com/subdir/page1 results in ->
mydomain.com/index.php with /subdir/pages/page1.inc content
More details below that probably don't matter, since the issue is likely in the .htaccess.
In index.php, I'm catching the request and using that to grab the corresponding include page from a directory /pages which has files with the content of each page. Here's the (somewhat simplified) code from index.php:
//grab the actual HTTP request
$request = $_GET['request'];
//if the request has slashes, isolate the directory part into $dir
$slashPos = strrpos($request, "/");
if($slashPos !== false){
$dir = substr($request, 0, $slashPos) . "/";
$page = basename($request, ".inc");
} else {
$dir = "";
$page = request;
}
//use "home" if no filename is specified.
if($page==""){$page="home";}
//build path to content include
$content = $dir . "pages/" . $page . ".inc";
//output page
require("header.php");
require($content );
require("footer.php");
This works perfect for the root directory. A request for mydomain.com/page1 results in mydomain.com/index.php being served with the contents of mydomain.com/pages/page1 included.
It also works for pages within a subdirectory: a request for mydomain.com/subdir/page1 results in mydomain.com/index.php being served with the contents of mydomain.com/subdir/pages/page1 included.
It all breaks when the request is for an actual directory. So mydomain.com/subdir/ returns 403 Forbidden. Why is it doing that? I expect it to load $dir with subdir and $page with home (I set as a default for when $page=="").
Yes, Options -Indexes is probably causing the 403, but why only on subdirectories? It doesn't 403 on root. And the line RewriteCond %{REQUEST_URI} ^/(subdir|subdir/.*|subdir2|subdir2/.*).*$ in .htaccess should catch it, right?
Ok, I figured it out. The problem is that subdir wasn't actually at web root. My entire site is actually working in its own subdirectory. In other words, I wasn't working in
mydomain.com/
as the root, I was working in
mydomain.com/mydivision/
as a root. So obviously in this line:
RewriteCond %{REQUEST_URI} ^/(subdir|subdir/.*|subdir2|subdir2/.*).*$
the ^ made it look for subdir at the actual webroot. To fix I removed ^ or add my actual subdomain to that line after ^.
Example: wp_file*.log -- what that should do is to password protect every file whose filename start with wp_file and ends with .log -- for example wp_file-22.log and wp_file_randomfile.log.
Possible?
The way I would make this, is by adding a rewrite rule for those files, redirect to some PHP file with the origional request in the GET. The PHP file can than show a password box and eventually the content of the log file once logged in.
RewriteEngine On
RewriteRule wp_file(.*)\.log /somePHPfile.php?logFile=wp_file$1 [L]
(not tested)
If you dont need access to the log files via your website (but use e.g. ftp), than you can rewrite the requests to those files to another page
RewriteEngine On
RewriteRule wp_file(.*)\.log /index.php [L]
In my website there is a link like this
http://backlinks.cheapratedomain.com/backlinks.php?url=emobileload.com
for better SEO I want to make it like this
http://backlinks.cheapratedomain.com/backlinks/emobileload.com
just want to replace '.php?url=' by '/'
My question is how can i do this by editing my .htaccess file ?
If it's only for that one URL, then you could try this:
RewriteRule ^backlinks/([a-zA-Z0-9\.\-]*)$ backlinks.php?url=$1 [L]
So requested URLs like http://backlinks.cheapratedomain.com/backlinks/domain.com will be redirected to http://backlinks.cheapratedomain.com/backlinks.php?url=domain.com