modsecurity does not work if no required SSL certificate was sent - security

Question

I have a lot of rules in modsecurity but none works if the host is numeric in SSL https://SERVER_IP, i get this response:
400 Bad Request No required SSL certificate was sent
My SSL is only valid to my domain name, but should not modsecurity work anyways? Because any request pass thru modsecurity before go to the application or something like that.
Question:
1 - How can i fix it?
2 - Why modsecurity does not work, and am i vunerable if i don't fix it?
This is my nginx.conf:
load_module modules/ngx_http_modsecurity_module.so;
user nobody;
worker_processes 1;
error_log /var/log/nginx/error.log error;
pid /var/run/nginx.pid;
events {
worker_connections 5000;
use epoll;
multi_accept on;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
modsecurity on;
modsecurity_rules_file /etc/nginx/modsec/main.conf;
client_header_timeout 20s;
client_body_timeout 20s;
client_max_body_size 20m;
client_header_buffer_size 6k;
client_body_buffer_size 128k;
large_client_header_buffers 2 2k;
send_timeout 10s;
keepalive_timeout 30 30;
reset_timedout_connection on;
server_names_hash_max_size 1024;
server_names_hash_bucket_size 1024;
ignore_invalid_headers on;
connection_pool_size 256;
request_pool_size 4k;
output_buffers 4 32k;
postpone_output 1460;
include mime.types;
default_type application/octet-stream;
# SSL Settings
ssl_certificate /etc/nginx/ssl/cf_cert.pem;
ssl_certificate_key /etc/nginx/ssl/cf_key.pem;
ssl_client_certificate /etc/nginx/ssl/origin-pull-ca.pem;
ssl_verify_client on;
ssl_verify_depth 5;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 1h;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA!RC4:EECDH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS";
ssl_session_tickets on;
ssl_session_ticket_key /etc/nginx/ssl/ticket.key;
ssl_dhparam /etc/nginx/ssl/dhparam.pem;
ssl_ecdh_curve secp384r1;
ssl_buffer_size 4k;
# Logs
log_format main '$remote_addr - $remote_user [$time_local] $request '
'"$status" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
log_format bytes '$body_bytes_sent';
access_log off;
# Cache bypass
map $http_cookie $no_cache {
default 0;
~SESS 1;
~wordpress_logged_in 1;
}
etag off;
server_tokens off;
# Headers
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Frame-Options deny always;
server {
listen 443 ssl http2;
server_name domain.com;
root /home/user/public_html;
index index.php index.html;
access_log /var/log/domain/domain.com.bytes bytes;
access_log /var/log/domain/domain.com.log combined;
error_log /var/log/domain/domain.com.error.log warn;
location / {
location ~.*\.(jpeg|jpg|png|gif|bmp|ico|svg|css|js)$ {
expires max;
}
location ~ [^/]\.php(/|$) {
try_files $uri =404;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_pass unix:/opt/alt/php-fpm73/usr/var/sockets/user.sock;
fastcgi_index index.php;
include /etc/nginx/fastcgi_params;
}
}
}
}

Answer 1

In short: This is unrelated to modsecurity.
Your server configuration requires the client to send client certificate. The TLS handshake will fail, if the client does not send such certificate - and this is the error you see.
modsecurity only analyzes the application data at the HTTP level. With HTTPS the TLS handshake first needs to be successfully done before the any application data gets exchanged. Since in this case the TLS handshake fails due to no certificate send by the client, the connection gets closed before any HTTP data gets exchanged and thus before modsecurity is used.