Using .htaccess to block referrer spam - security

Our forum gets targeted a lot by automated bots that try to register automatically.
We can see an example here from the error log
[Sun Apr 03 14:04:46 2011] [error]
[client 70.183.110.133] File does not
exist:
/home/spoilert/public_html/forum/++++++++++++++++++++++++++++++++++++Result:+captcha+decoded+(23+attempts);+registered+(registering+only+mode+is+ON);,
referer:
http://forum.spoilertv.co.uk/++++++++++++++++++++++++++++++++++++Result:+captcha+decoded+%2823+attempts%29;+registered+%28registering+only+mode+is+ON%29;
[Sun Apr 03 13:45:54 2011] [error]
[client 70.183.110.133] File does not
exist:
/home/spoilert/public_html/2008,
referer:
I've updated my htaccess with this code
SetEnvIfNoCase Referer
"^http://(W)decoded.*$" banned
Deny
from env=banned
It "should" deny any referrer link with the word decoded in it but it seems that it's not working. I still seem to be getting a few of these robots getting through with the same URL so it seems that it's still happening.

What happens if you change it to
SetEnvIfNoCase Referer ".*+decoded+.*" banned
Deny from env=banned

Related

RewriteEngine not allowed here magento and apache2 on VPS

I've just installed Magento on my VPS (Debian 11), I must say Magento it's a pain compared to PretaShop and Wordpress, but I really need it.
After a painful installation, I run the page and I get the following:
Internal Server Error The server encountered an internal error or
misconfiguration and was unable to complete your request.
Please contact the server administrator at info#magento.mywebsite.ie to
inform them of the time this error occurred, and the actions you
performed just before this error.
More information about this error may be available in the server error
log.
Additionally, a 500 Internal Server Error error was encountered while
trying to use an ErrorDocument to handle the request.
Then I thought to check logs. So I opened the file magento.mywebsite.ie.error.log and there is a list of all the attempts, like the following:
[Sun Oct 23 17:04:41.916039 2022] [core:alert] [pid 886340:tid
139973219006208] [client 51.37.89.15:0]
/home/elliot/web/magento.mywebsite.ie/public_html/magento2/.htaccess:
RewriteEngine not allowed here [Sun Oct 23 17:04:42.225156 2022]
[core:alert] [pid 886340:tid 139973202220800] [client 51.37.89.15:0]
/home/elliot/web/magento.mywebsite.ie/public_html/magento2/.htaccess:
RewriteEngine not allowed here [Sun Oct 23 17:04:42.495916 2022]
[core:alert] [pid 886340:tid 139973185435392] [client 51.37.89.15:0]
/home/elliot/web/magento.mywebsite.ie/public_html/magento2/.htaccess:
RewriteEngine not allowed here [Sun Oct 23 17:04:42.705350 2022]
[core:alert] [pid 886340:tid 139973168649984] [client 51.37.89.15:0]
/home/elliot/web/magento.mywebsite.ie/public_html/magento2/.htaccess:
RewriteEngine not allowed here [Sun Oct 23 17:04:42.909888 2022]
[core:alert] [pid 886340:tid 139973311325952] [client 51.37.89.15:0]
/home/elliot/web/magento.mywebsite.ie/public_html/magento2/.htaccess:
RewriteEngine not allowed here [Sun Oct 23 17:05:22.958733 2022]
[core:alert] [pid 886660:tid 140355403986688] [client 51.37.89.15:0]
/home/elliot/web/magento.mywebsite.ie/public_html/magento2/.htaccess:
RewriteEngine not allowed here
The files .htaccess in my Magento new installation are exactly identical to the .htaccess that you would find here on GitHub:
Link: https://github.com/magento/magento2
You will find .htaccess in the parent folder magento2, another one in the Document Root "pub" inside magento2. My opinion is that the system is using the one in the parent directory, magento2, since if I delete that .htaccess, the webpage would show "This page isn’t working" instead of "Internal Server Error", so the error message would be different in this way.
And this is apache2.ssl.conf used as configuration file:
I removed the comments here
<VirtualHost 141.146.42.11:8443>
ServerName magento.mywebsite.ie
ServerAdmin info#magento.mywebsite.ie
DocumentRoot /home/elliot/web/magento.mywebsite.ie/public_html/magento2/pub
ScriptAlias /cgi-bin/ /home/elliot/web/magento.mywebsite.ie/cgi-bin/
Alias /vstats/ /home/elliot/web/magento.mywebsite.ie/stats/
Alias /error/ /home/elliot/web/magento.mywebsite.ie/document_errors/
#SuexecUserGroup elliot elliot
CustomLog /var/log/apache2/domains/magento.mywebsite.ie.bytes bytes
CustomLog /var/log/apache2/domains/magento.mywebsite.ie.log combined
ErrorLog /var/log/apache2/domains/magento.mywebsite.ie.error.log
<Directory /home/elliot/web/magento.mywebsite.ie/stats>
AllowOverride All
</Directory>
<Directory /home/elliot/web/magento.mywebsite.ie/public_html/magento2/pub>
AllowOverride All
SSLRequireSSL
Options +Includes -Indexes +ExecCGI </Directory>
SSLEngine on
SSLVerifyClient none
SSLCertificateFile /home/elliot/conf/web/magento.mywebsite.ie/ssl/magento.mywebsite.ie.crt
SSLCertificateKeyFile /home/elliot/conf/web/magento.mywebsite.ie/ssl/magento.mywebsite.ie.key
SSLCertificateChainFile /home/elliot/conf/web/magento.mywebsite.ie/ssl/magento.mywebsite.ie.ca
<FilesMatch \.php$>
SetHandler "proxy:unix:/run/php/php8.1-fpm-magento.mywebsite.ie.sock|fcgi://localhost"
</FilesMatch>
SetEnvIf Authorization .+ HTTP_AUTHORIZATION=$0
IncludeOptional /home/elliot/conf/web/magento.mywebsite.ie/apache2.ssl.conf_*
IncludeOptional /etc/apache2/conf.d/*.inc
</VirtualHost>
Please could you be so kind to help me with this? I have no experience at all about Apache issues, then I didn't know what else info I could provide to you. Hope you can help.
I solved the problem by adding the following line with the Parent Root Directory to my apache2.ssl.conf:
<Directory /home/elliot/web/magento.mywebsite.ie/public_html/magento2>
AllowOverride All
</Directory>
In this way also the parent directory has AllowOverride All.

how can i separate the mainDomain and the subDomains use mod-rewrite with htaccess file

I want to redirect subdomains on my site if they are queried, but if I do not query, I want to redirect it to a different domain.
I have reached some codes by searching the internet, but I have not improved.
If I can not tell what I want to do, I want the following url and file structure to be the same as I showed.
public_html/
|____.htaccess (main htaccess)
|____index.php (php empty)
|____mainDomain/
| |_____.htaccess (without subdomain htaccess)
| |_____index.php
| |_____public/
| | |____.htaccess (application htaccess)
| | |____index.php
|____subsDomain/
| |_____ucgen/
| |_____testFolder/
main htaccess:
RewriteCond %{ENV:REDIRECT_SUBDOMAIN} =""
RewriteCond %{HTTP_HOST} ^([a-z0-9][-a-z0-9]+)\.metehanboy\.com\.?(:80)?$ [NC]
RewriteCond %{DOCUMENT_ROOT}/subsDomain/%1 -d
RewriteRule ^(.*) subsDomain/%1/$1 [E=SUBDOMAIN:%1,L]
RewriteRule ^ - [E=SUBDOMAIN:%{ENV:REDIRECT_SUBDOMAIN},L]
RewriteRule ^$ mainDomain/ [L]
RewriteRule (.*) mainDomain/$1 [QSA,L]
without subdomain htaccess
RewriteEngine on
RewriteRule ^$ public/ [QSA,L]
RewriteRule (.*) public/$1 [QSA,L]
application htaccess
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?hedef=$1 [PT,L]
I added cpanelden subdomains but infiniteloop happens when I run the subdomain, how can I fix this?
example subdomain link: http://ucgen.metehanboy.com/example/example
other link: http://metehanboy.com/authentication/index
mainDomain folder include mvc php app
subsDomain folder includes test application scripts
Apache Error Log
[Wed May 02 14:16:26.044185 2018] [core:alert] [pid 3117] [client 95.70.153.106:59513] /home/metehanboy/public_html/.htaccess: Invalid command 'RewriteLog', perhaps misspelled or defined by a module not included in the server configuration
[Wed May 02 14:16:26.043954 2018] [core:alert] [pid 3117] [client 95.70.153.106:59513] /home/metehanboy/public_html/.htaccess: Invalid command 'RewriteLog', perhaps misspelled or defined by a module not included in the server configuration
[Wed May 02 14:15:42.256845 2018] [core:alert] [pid 2690] [client 95.70.153.106:59497] /home/metehanboy/public_html/.htaccess: Invalid command 'RewriteLogLevel', perhaps misspelled or defined by a module not included in the server configuration
[Wed May 02 14:15:42.256734 2018] [core:alert] [pid 2690] [client 95.70.153.106:59497] /home/metehanboy/public_html/.htaccess: Invalid command 'RewriteLogLevel', perhaps misspelled or defined by a module not included in the server configuration
[Wed May 02 14:15:34.964648 2018] [core:alert] [pid 2641] [client 95.70.153.106:59496] /home/metehanboy/public_html/.htaccess: Invalid command 'RewriteLogLevel', perhaps misspelled or defined by a module not included in the server configuration
[Wed May 02 14:15:34.964434 2018] [core:alert] [pid 2641] [client 95.70.153.106:59496] /home/metehanboy/public_html/.htaccess: Invalid command 'RewriteLogLevel', perhaps misspelled or defined by a module not included in the server configuration
[Wed May 02 14:15:21.351847 2018] [core:alert] [pid 2696] [client 95.70.153.106:59491] /home/metehanboy/public_html/.htaccess: Invalid command 'RewriteLogLevel', perhaps misspelled or defined by a module not included in the server configuration
[Wed May 02 14:15:21.351768 2018] [core:alert] [pid 2696] [client 95.70.153.106:59491] /home/metehanboy/public_html/.htaccess: Invalid command 'RewriteLogLevel', perhaps misspelled or defined by a module not included in the server configuration
[Wed May 02 14:13:47.582923 2018] [core:error] [pid 3117] [client 95.70.153.106:59459] AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer: http://ucgen.metehanboy.com/
[Wed May 02 14:13:47.582877 2018] [core:error] [pid 3117] [client 95.70.153.106:59459] AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer: http://ucgen.metehanboy.com/
[Wed May 02 14:13:46.839964 2018] [core:error] [pid 3118] [client 95.70.153.106:59457] AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer: http://ucgen.metehanboy.com/
[Wed May 02 14:13:46.839899 2018] [core:error] [pid 3118] [client 95.70.153.106:59457] AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer: http://ucgen.metehanboy.com/
[Wed May 02 14:13:39.737762 2018] [core:error] [pid 2696] [client 95.70.153.106:59446] AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer: http://ucgen.metehanboy.com/
[Wed May 02 14:13:39.737686 2018] [core:error] [pid 2696] [client 95.70.153.106:59446] AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer: http://ucgen.metehanboy.com/
In cPanel, you can add subdomains adjacent to your main domain, instead of a subdirectory. Why do you need that over-complicated structure? What's the benefit? Use something like this and save yourself the headache:
public_html/
|____.htaccess (application htaccess)
|____index.php
public_html_ucgen/
|____.htaccess (subdomain htaccess)
|____index.php

how many htaccess files there shall be?

How i shall organise htaccess files?
Shall there be a single file, or file in each folder?
I have somedomain/public/.htaccess, which puts url string to $_REQUEST['url'] variable. This is the second part of the code below, and it works.
Later, i created somedomain/.htaccess with code for redirecting to https://www.
Two separate files did not work, thus i pasted the somedomain/public/.htaccess to somedomain/.htaccess. But nothing works. How to achieve these two functionalities. I means redirection to https://www and url fetching to variable. Where is the mistake below?
Shall i create somedomain/.htaccess, somedomain/public/.htaccess, somedomain/somefolder/.htaccess, ....
Or i shall have only one somedomain/.htaccess which deals with all redirections?
If i have only one somedomain/.htaccess as it is below, how i shall redirect
somedomain to somedomain/public/index.php
somedomain/{url not matching somedomain/public/index.php/parameters} to error
folder structure
somedomain.com/
.htaccess
/public
#.htaccess #the second part of code, which i copied to somedomain.com/.htaccess
index.php
.htaccess in somedomain.com/
#first part of htaccess
# Rewrite to HTTPS:
# If www. is already there it will be included, if not
# it will be catched by the subsequent rule.
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
#[L,R=301]
# Now, rewrite any request to the wrong domain to use www.
# [NC] is a case-insensitive match
RewriteCond %{HTTP_HOST} !^www\. [NC]
RewriteRule (.*) https://www.%{HTTP_HOST}%{REQUEST_URI}
#[L,R=301]
RewriteEngine On
# the default webpage loaded by the server:
DirectoryIndex /public/index.php/security/login
IndexIgnore *
#Second part of htaccess
#rules to fetch controller name and method name starting from /public/index.php/
RewriteBase /public/
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteCond %{ENV:REQUEST_FILENAME} !-d
RewriteCond %{ENV:REQUEST_FILENAME} !-f
RewriteCond %{ENV:REQUEST_FILENAME} !-l
RewriteRule ^index.php?(.+)$ index.php?url=$1 [QSA,L]
I am gettting the error : The server encountered an internal error or misconfiguration and was unable to complete your request. ...
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
Previously i had only the second part of .htaccess in somedomain.com/public .
Server error logs:
[Thu Apr 20 08:37:36 2017] [error] [client 213.133.81.6] File does not exist: /home/gintare/public_html/typejoy.biz/500.shtml
[Thu Apr 20 08:45:43 2017] [alert] [client 213.133.81.6] /home/gintare/public_html/typejoy.biz/.htaccess: Invalid command '\xef\xbb\xbf', perhaps misspelled or defined by a module not included in the server configuration
[Thu Apr 20 08:45:43 2017] [alert] [client 213.133.81.6] /home/gintare/public_html/typejoy.biz/.htaccess: Invalid command '\xef\xbb\xbf', perhaps misspelled or defined by a module not included in the server configuration
[Thu Apr 20 08:46:04 2017] [alert] [client 213.133.81.6] /home/gintare/public_html/typejoy.biz/.htaccess: Invalid command '\xef\xbb\xbf', perhaps misspelled or defined by a module not included in the server configuration
[Thu Apr 20 08:46:04 2017] [alert] [client 213.133.81.6] /home/gintare/public_html/typejoy.biz/.htaccess: Invalid command '\xef\xbb\xbf', perhaps misspelled or defined by a module not included in the server configuration
[Thu Apr 20 08:46:08 2017] [alert] [client 213.133.81.6] /home/gintare/public_html/typejoy.biz/.htaccess: Invalid command '\xef\xbb\xbf', perhaps misspelled or defined by a module not included in the server configuration
[Thu Apr 20 08:46:08 2017] [alert] [client 213.133.81.6] /home/gintare/public_html/typejoy.biz/.htaccess: Invalid command '\xef\xbb\xbf', perhaps misspelled or defined by a module not included in the server configuration
[Thu Apr 20 10:05:04 2017] [warn] RSA server certificate CommonName (CN) `typejoy.biz' does NOT match server name!?
[Thu Apr 20 10:50:27 2017] [error] [client 115.118.128.202] File does not exist: /home/gintare/public_html/typejoy.biz/favicon.ico
[Thu Apr 20 10:50:27 2017] [error] [client 115.118.128.202] File does not exist: /home/gintare/public_html/typejoy.biz/404.shtml
[Thu Apr 20 10:51:41 2017] [error] [client 115.118.128.202] File does not exist: /home/gintare/public_html/typejoy.biz/favicon.ico
[Thu Apr 20 10:51:41 2017] [error] [client 115.118.128.202] File does not exist: /home/gintare/public_html/typejoy.biz/404.shtml

Failing miserably with .htaccess rewrite, unsure how to troubleshoot

I have been trying to figure out URL rewriting on my local development site all day with no luck. Initially the .htaccess files were ignored. Now they are being read but not working. I don't know how to troubleshoot an .htaccess file though. From what I have read, it seems Apache 2.4 got rid of specifying your own RewriteLog. The only help I am getting is from /var/log/apache2/error.log which is all Greek to me.
The rewrite I am attempting is simply:
local.domain.com/users/index.php?id=1 -> local.domain.com/users/1/
My .htaccess looks like:
RewriteEngine On
RewriteRule ^users/([0-9]+)/?$ users/index.php?id=$1 [NC,L]
When requesting local.domain.com/users/index.php?id=1 , the URL remains unchanged. The error.log for apache gives the following 3 lines:
[Tue Jun 17 15:20:04.705939 2014] [rewrite:trace3] [pid 6569] mod_rewrite.c(468): [client 127.0.0.1:46208] 127.0.0.1 - - [local.domain.com/sid#b63f02c0][rid#b6b12058/initial] [perdir /var/www/vhosts/domain.com/] strip per-dir prefix: /var/www/vhosts/domain.com/users/index.php -> users/index.php
[Tue Jun 17 15:20:04.705979 2014] [rewrite:trace3] [pid 6569] mod_rewrite.c(468): [client 127.0.0.1:46208] 127.0.0.1 - - [local.domain.com/sid#b63f02c0][rid#b6b12058/initial] [perdir /var/www/vhosts/domain.com/] applying pattern '^users/([0-9]+)$' to uri 'users/index.php'
[Tue Jun 17 15:20:04.705990 2014] [rewrite:trace1] [pid 6569] mod_rewrite.c(468): [client 127.0.0.1:46208] 127.0.0.1 - - [local.domain.com/sid#b63f02c0][rid#b6b12058/initial] [perdir /var/www/vhosts/domain.com/] pass through /var/www/vhosts/domain.com/users/index.php
The location of the .htaccess is /var/www/vhosts/domain.com/ . Is there a way to get better/more log info? Is the /var/log/apache2/error.log really the log I should be using for this? Is the problem really with my .htaccess code or is there some sort of configuration I am missing or something? I know there are similar questions but so far I haven't found one that was both understandable and a solution to my problem.
Thanks in advance!
You are rewriting an incoming URI /users/1/ to /users/index.php?id=1 (SEO form to dynamic form). Your .htaccess looks correct for that (I assume it's in the root).
Are you sure that your Apache server is built with the RewriteEngine enabled? Are you overlooking an error message? Your typed-in URI is /users/1/?
You regex is looking for [0+9]+ which means it is looking for numbers after /users. In your input url (assuming that's your input url), /var/www/vhosts/domain.com/users/index.php there is no numbers after `'/users'
Put a number after users like /users/55555/ and see what happens.

htaccess Deny from all and 500 Internal Server Error

I want to restrict direct access to a specific directory (and all the files inside) on my local server.
The directory is: C:/Server/www/project/html/
I've tried the following code (.htaccess is placed in www directory - /project/html/ doesn't work too):
<Directory "C:/Server/www/project/html/">
AllowOverride all
Order Deny,Allow
Deny from all
</Directory>
However, it causes 500 Internal Server Error and I can't understand why.
Apache error log:
[Fri Aug 05 16:06:01 2011] [alert] [client 127.0.0.1] C:/Server/www/.htaccess: <Directory not allowed here, referer: http://localhost/project/index.php?id=8
[Fri Aug 05 16:06:01 2011] [alert] [client 127.0.0.1] C:/Server/www/.htaccess: <Directory not allowed here, referer: http://localhost/project/index.php?id=8
[Fri Aug 05 16:06:01 2011] [alert] [client 127.0.0.1] C:/Server/www/.htaccess: <Directory not allowed here, referer: http://localhost/project/index.php?id=8
[Fri Aug 05 16:06:01 2011] [alert] [client 127.0.0.1] C:/Server/www/.htaccess: <Directory not allowed here, referer: http://localhost/project/index.php?id=8
[Fri Aug 05 16:06:01 2011] [alert] [client 127.0.0.1] C:/Server/www/.htaccess: <Directory not allowed here, referer: http://localhost/project/index.php?id=8
[Fri Aug 05 16:06:02 2011] [alert] [client 127.0.0.1] C:/Server/www/.htaccess: <Directory not allowed here, referer: http://localhost/project/index.php?id=8
[Fri Aug 05 16:54:12 2011] [alert] [client 127.0.0.1] C:/Server/www/.htaccess: <Directory not allowed here, referer: http://localhost/project/index.php?id=8
[Fri Aug 05 16:54:12 2011] [alert] [client 127.0.0.1] C:/Server/www/.htaccess: <Directory not allowed here
[Fri Aug 05 17:05:06 2011] [alert] [client 127.0.0.1] C:/Server/www/.htaccess: <Directory not allowed here, referer: http://localhost/project/index.php?id=8
Check Apache error log for exact error description.
In any case -- the reason for this error is simple: <Directory> directive CANNOT be placed in .htaccess file -- only server config or virtual host.
http://httpd.apache.org/docs/current/mod/core.html#directory
for me it required enable headers module to apache

Resources