How to block unusual bot, like "bot[\s_ :,\.\;\/\\-]" name? - .htaccess

until today I was blocking unwanted bots in .htaccess
SetEnvIfNoCase User-Agent .*mj12bot.* bad_bot
SetEnvIfNoCase User-Agent .*baiduspider.* bad_bot
But I noticed lastly that I have unusual bot making mess on my serwer but don't know how to block it because his name is:
bot[\s_ :,\.\;\/\\-]
I will be grateful for any help


You can use the following to deny requests for bot[\s_ :,.\;/\-]
SetEnvIfNoCase user-agent bot\[.+\]|mj12bot|baiduspider bad_bot=1
Order Allow,Deny
Allow from all
Deny from env=bad_bot
To block multiple user-agents, you may use :
SetEnvIfNoCase user-agent bot\[.+\]|.*mj12bot.*|.*baiduspider.* bad_bot=1
Order Allow,Deny
Allow from all
Deny from env=bad_bot

Related

htaccess code for blocking bots is not working

I am trying to block some of these below listed bots using htaccess, and its not working.
In my PHP code, I track hits from unique bots, and log useragent of bots which passed through the htaccess block.
That is how I got this list below.
I assume that anything blocked by htaccess will not trigger the PHP script, is that right?
Bots:
Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)
Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)
HTAccess code:
SetEnvIfNoCase User-Agent "SemrushBot" bad_user
Deny from env=bad_user
SetEnvIfNoCase User-Agent "semrush" bad_user
Deny from env=bad_user
SetEnvIfNoCase User-Agent "BLEXBot" bad_user
Deny from env=bad_user
What am I doing wrong here? Why is htaccess not blocking these?

Deny access to ssi files

I'm using SSI on my website.
I have got html files which represent my site and shtml files which I include into some of these html files using the include command of SSI.
Is there any way to deny users from seeing the shtml files which I include?
I tried the following:
<FilesMatch "\.(shtml?)$">
Order allow,deny
Deny from all
allow from 127.0.0.1
</FilesMatch>
And here is the whole part of my current htaccess file which may be important:
allow from all
<Files ~ "^\.htaccess|^README\.txt">
Order allow,deny
Deny from all
</Files>
<FilesMatch "\.(shtml?)$">
Order allow,deny
Deny from all
allow from 127.0.0.1
</FilesMatch>
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
AddOutputFilter INCLUDES .html

How To Allow Access to Certain File Types but Deny All Others

While I'm confident this has been asked and answered somewhere, my Google and SO searches have not helped me solve what seems like a fairly easy problem.
The goal:
Deny access to ALL file types except images.
Current .htaccess file:
<Files *.*>
Order Deny,Allow
Deny from all
</Files>
<FilesMatch "\.(jpg|png|gif|jpeg|JPG|PNG|GIF|JPEG)$">
Order Deny,Allow
Allow from all
</FilesMatch>
I still cannot (via the browser) access any image files, with a "403 Forbidden" error.
Questions:
1. How do I make this work properly without rewrite rules?
2. Can I combine Files and FilesMatch rules like this?
3. Are the FilesMatch rules case sensitive?

You can easily achieve this via mod_rewrite rules:
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule !\.(jpe?g|png|gif)$ - [NC,F]
Using FilesMatch you could do this:
Order deny,allow
# first deny all files
<Files *>
deny from all
</Files>
# then allow all image files
<FilesMatch "(?i)\.(jpe?g|png|gif)$">
allow from all
</FilesMatch>

FilesMatch or Files in htaccess to deny two specific files

I mostly understand using Files for one file and FileMatch for broad regex matching, but if I wanted to deny access to two or three specific unique files how would I best do that?
Ideas:
<Files .htpasswd>
Order allow,deny
Deny from all
Satisfy All
</Files>
<Files htpasswd-example>
Order allow,deny
Deny from all
Satisfy All
</Files>
<Files htaccess-example>
Order allow,deny
Deny from all
Satisfy All
</Files>
or something like:
<FilesMatch "^(.htpasswd|htpasswd-example|htaccess-example)$">
Order allow,deny
Deny from all
Satisfy All
</FilesMatch>
note: unrelated but I can't make these example files . files for other reasons, hence my desire to hide them.

Try:
RewriteEngine On
RewriteCond %{HTTP_HOST} !^localhost$ [NC]
RewriteRule ^ - [L,F]
So, if the host is other than "localhost" deny it.
Check out this answer too:
.htaccess deny access from external request

.htaccess: Deny from all

This is working:
<Files *.fileext>
Order Allow,Deny
Deny from all
</Files>
This is not:
<Files *.fileext|somedirectory>
Order Allow,Deny
Deny from all
</Files>
Please help.

Files does not allow the use of regular expressions, but FilesMatch does, therefore it searches for a file with (something).fileext|somedirectory in the path, and this is not what you want to do. Your code will have to look like this:
<FilesMatch (\.fileext$|^somedirectory$)>
Order Allow,Deny
Deny from all
</FilesMatch>
see http://httpd.apache.org/docs/1.3/mod/core.html#files and http://httpd.apache.org/docs/1.3/mod/core.html#filesmatch

This can be slightly improved.
There is no need for an order directive and the end of string syntax can be used just once.
<FilesMatch (\.fileext|^somedirectory)$>
Deny from all
</FilesMatch>

Resources