I am getting thousands of bot visits every hour to one specific URL of my website http://www.domain.com/.gtput/
I would like to block ALL traffic (human+bot) trying to access this URL. (This URL is not accessed by human, though!)
After lot of googling, I found an answer that worked from here --> Anyway to block visits to specific URLs, for eg via htaccess?. I am using following code in htaccess file to block this URL.
<IfModule mod_alias.c>
Redirect 403 /.gtput/
</IfModule>
Is there a BETTER way to block ALL traffic from accessing that one specific URL? So that I can save server resources (bandwidth etc.).
You can use the following Rule to forbid access to "/.gtput" :
RewriteEngine On
RewriteCond %{THE_REQUEST} /.gtput/? [NC]
RewriteRule ^ - [F,L]
Related
Malicious website owners are using the contents of our website to say example.com on their websites say spam.com like:
<?php
$url='https://example.com/';
// using file() function to get content
$lines_array=file($url);
// turn array into one variable
$lines_string=implode('',$lines_array);
//output, you can also save it locally on the server
echo $lines_string;
?>
We want to prevent the contents of our website from displaying on their websites and redirect those requests to a warning page on our website (to a webpage and not an image).
After doing some R&D, we tried doing this:
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^https://example\.com/.*$ [NC]
RewriteRule ^(.*) https://example.com/404 [R=301,L]
</ifModule>
But it doesn't work. What are we doing wrong?
Reference: htaccess prevent hotlink also prevents external links
"Hotlinking" and "webpage scraping" are two very different things. What you describe with the snippet of simplified PHP code is a form of "webpage scraping" or even "cloning". This does not (or is very unlikely to) generate a Referer header in the request, so cannot be blocked by simply checking the Referer (ie. HTTP_REFERER server variable) as you would do with "hotlinking".
(Your example mod_rewrite code blocks "hotlinking", not "scraping/cloning".)
The only way to block these types of requests is to block the IP address of the server making the request. For example, if the "malicious" requests are coming from 203.0.113.111 then you would do something like the following in the Apache 2.4 config (or .htaccess file) to block such requests:
<RequireAll>
Require all granted
Require not IP 203.0.113.111
</RequireAll>
However, the requests may not be coming from the same IP address that is hosting the "cloned" content. You'll need to determine this from your server's access logs. But to further complicate this the "attacker" may be using a series of IP addresses or have access to a botnet of ever-changing IPs. This can quickly become almost impossible to block without access to a more comprehensive firewall.
You can try other techniques such as issuing redirects to the canonical hostname from client-side code. However, more advanced "cloning" software (and/or reverse proxy servers) will "simply" modify the code/URLs to thwart your redirection attempts.
So, I'm try to google it, and finded this:
RewriteEngine On
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?example.com/.*$ [NC]
RewriteRule ^(.*)$ http://www.example.com/404 [R=404,L] # R=404 returns 404 page
I have a subfolder content in:
www.mydomain.com/content
Users can log in at this location. I have also created a page that will directly load a content using a PHP page:
www.mydomain.com/content/direct.php?direct=<contentid>
Users can use the link and share them. However, if the user is not yet authenticated, it should redirect them to the homepage with a message letting them know that they need to log in:
www.mydomain.com/content/index.php?error=4
I wanted to support simple URLS like:
www.mydomain.com/content/direct/<contentid>
However, I am getting too many redirects error. May I know how I should write my HTACCESS file?
Here is the HTACCESS I am using currently placed inside the subfolder /content/:
Options +FollowSymLinks
RewriteEngine On
#direct
RewriteRule ^/?direct/([^/d]+)/?$ direct.php?direct=$1 [QSA]
#SSL
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
The solution I settled with was to put the .HTACCESS file in the root instead of the subfolder with the following code
RewriteRule ^/?content/([^/d]+)/?$ subfolder/content.php?content=$1 [R=301,L]
I used [R=301,L] instead since I prefer that the real URL is visible to the user. I only needed the simple URL for easier means to integrate them into existing platforms that do not like query strings.
In content.php I place a redirect to the login page if the user is not authenticated. This prevented the multiple redirect error.
Haven't managed to find any answers to this at StackOverflow, so here goes:
I have a long list of banned IPs that I would like to block from my website entirely, but I would like to redirect them all to a special customized (not general-purpose) 403 forbidden message on the site. Is there a way to accomplish this via htaccess?
Thanks!
Try it like this,
RewriteEngine On
#ip addresses to block
RewriteCond %{REMOTE_ADDR} ^111\.11\.111\.111$
RewriteRule ^ file.html [R=301,L]
Google Analytics is showing my own root domain is the referral for a large portion of my website's traffic.
So, mywebsite.com is showing as the referrer for www.mywebsite.com.
I have tried adding a referral exclusion within Analytics with no success. I have cleaned my .htaccess which also hasn't been successful
.htaccess code
RewriteEngine On
Options +FollowSymLinks
# Redirects mywebsite/uk/anypage to mywebsite.eu/anypage
RewriteRule ^uk/(.*)$ /$1 [NC,R=302,NE,L]
# Redirect 404 to index
ErrorDocument 404 /
# Use www version
RewriteCond %{HTTP_HOST} ^mywebsite.eu [NC]
RewriteRule ^(.*)$ http://www.mywebsite.eu/$1 [L,R=301,NC]
This is what it looks like in Analytics:
I've put a lot of time into trying to find the cause so any suggestions, .htaccess related or otherwise, are very welcome
Did you migrate recently to Universal Analytics? It migrates the previous utmz cookie values that stored traffic source, so if your users had self-referrals prior to upgrading to UA, this will persist int your data. Could you check if there are new sessions in this traffic?
Make sure to add to the referral exclusion list your domain with and without the www.
For some reason certain sites it likes to have it while on others it does not, I would definitely try this and then see if the future sessions are any more clean before changing too much with the .htaccess.
I have a Drupal site where new content is added very rarely. Recently, there have been an increasing number of visits from bots to various URLs (node/add, user/register), which return Drupal's "Access denied" page.
I want to block access to these URLs in .htaccess. I have tried the following and it works:
<IfModule mod_alias.c>
Redirect 403 /node/add
Redirect 403 /user/register
</IfModule>
However, bots can still access ?q=node/add and ?q=user/register. I have tried including ?q= in the code above but no success.
How do I block access to these URLs in .htaccess ?
You can use mod_rewrite to do url-manipulation based on the query string. You'll need something like the (untested) code below.
RewriteEngine On
RewriteCond %{QUERY_STRING} ^q=(node/add|user/register)$
RewriteRule ^ - [F,L]
What does this do? It matches any url (^), then checks if the query string is equal to q=node/add or q=user/register. If either one matches, then the url is not rewritten (-), but access is denied [F] and the rewriting stops for this iteration [L].
While doing this via .htaccess is completely viable, I would reconsider this approach and consider putting these URLs into robots.txt for crawler bots. This way, they will ignore them completely, which is definitely more healthy for SEO.
Also, you can use Global Redirect module (https://drupal.org/project/globalredirect) to ensure that only clean URLs are used.