Symfony 5 and varnish - varnish

I installed varnish on my server, it's active (checked via the sudo systemctl varnish restart command). I configured the /etc/varnish/default.vcl file following the tutorial https://symfony.com/doc/current/http_cache/varnish.html but I don't have the impression that varnish is taken into account on Symfony (I don't see X-Varnish in the response header (however it appears well when I access directly to my server via http://localhost/.
Here is my default.vcl file:
#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and https://www.varnish-cache.org/trac/wiki/VCLExamples for more examples.
# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;
# Default backend definition. Set this to point to your content server.
backend default {
.host = "127.0.0.1";
.port = "8080";
}
sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
if (req.http.X-Forwarded-Proto == "https" ) {
set req.http.X-Forwarded-Port = "443";
} else {
set req.http.X-Forwarded-Port = "80";
}
# Remove all cookies except the session ID.
if (req.http.Cookie) {
set req.http.Cookie = ";" + req.http.Cookie;
set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
set req.http.Cookie = regsuball(req.http.Cookie, ";(PHPSESSID)=", "; \1=");
set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
if (req.http.Cookie == "") {
// If there are no more cookies, remove the header to get page cached.
unset req.http.Cookie;
}
}
# Add a Surrogate-Capability header to announce ESI support.
set req.http.Surrogate-Capability = "abc=ESI/1.0";
}
sub vcl_backend_response {
# Happens after we have read the response headers from the backend.
#
# Here you clean the response headers, removing silly Set-Cookie headers
# and other mistakes your backend does.
# Check for ESI acknowledgement and remove Surrogate-Control header
if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
unset beresp.http.Surrogate-Control;
set beresp.do_esi = true;
}
}
sub vcl_deliver {
# Happens when we have all the pieces we need, and are about to send the
# response to the client.
#
# You can do accounting or modifying the final object here.
}
Maybe then there is something to configure on symfony but I don't see what. Thank you.

If you're unsure that Varnish is correctly handling requests, you should check if the varnishd process runs on the right TCP port.
You can do this by running the following command:
sudo systemctl status varnish
In my case, this is the output:
● varnish.service - Varnish Cache Plus, a high-performance HTTP accelerator
Loaded: loaded (/lib/systemd/system/varnish.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/varnish.service.d
└─override.conf
Active: active (running) since Fri 2021-04-09 17:50:00 UTC; 3min 5s ago
Main PID: 1242 (varnishd)
Tasks: 226
CGroup: /system.slice/varnish.service
├─1242 /usr/sbin/varnishd -a :80 -a 127.0.0.1:8443,proxy -S /etc/varnish/secret -T localhost:6082 -f /etc/varnish/default.vcl -s mse,/etc/varnish/mse.conf
└─1256 /usr/sbin/varnishd -a :80 -a 127.0.0.1:8443,proxy -S /etc/varnish/secret -T localhost:6082 -f /etc/varnish/default.vcl -s mse,/etc/varnish/mse.conf
The relevant information is the command that is executed, along with the runtime options and paramters.
In this case varnishd is listening for incoming connections on 2 ports:
80 for regular HTTP traffic
8443 for PROXY traffic
In your case, the listening ports may be configured differently.
Out the box, these are the default values:
/usr/sbin/varnishd \
-a :6081 \
-a localhost:8443,PROXY \
-p feature=+http2 \
-f /etc/varnish/default.vcl \
-s malloc,256m
As you can see port 6081 is used for incoming HTTP traffic. You may need to change this to port 80.
Just run the following command to edit the unit file:
sudo systemctl edit --full varnish
Once you've adjusted the runtime parameters, just run the following command to restart the service:
sudo systemctl restart varnish
WARNING: if you run Varnish on the same server as the webserver, you will need to change the listening port of the webserver as well. A good option is port 8080.

Related

Varnish: Is it possible to log all GET requests for further processing?

Is it possible to use Varnish for the following task?
Imagine an URL(e.g. /vote?poll-id=1&answer-id=2) that is requested via direct links where we display poll results for the chosen poll-id.
I would like to save/pull/process all those requested URL(in near real time) to generate those poll results.
Is it possible to get those URLs as some sort of stream for further processing?
The reason why Varnish is used is because I would like to reduce the load on a slower upstream backend service. And because some delay in showing the actual results is OK.
Varnish has built-in shared memory logs. These can be consulted using various tools.
The main ones that could be useful for you are:
varnishlog: in-depth logging about every aspect of the request, response, and internal processing
varnishncsa: an Apache/NCSA style logging tool
You can also leverage the VCL programming language and log requests from within VCL to the operating system's syslog mechanism.
varnishlog
The following command will display all logging information for URLs that start with /vote:
varnishlog -g request -q "ReqUrl ~ '^/vote'"
You can filter out the fields you need:
varnishlog -i requrl -i reqheader -g request -q "ReqUrl ~ '^/vote'"
This one will only display the request URL and all request headers.
You can also write the output to a file:
varnishlog -A -a -w /var/log/varnish/vsl_vote.log -i requrl -i reqheader -g request -q "ReqUrl ~ '^/vote'"
See http://varnish-cache.org/docs/6.5/reference/varnishlog.html to learn more about varnishlog and http://varnish-cache.org/docs/6.5/reference/vsl-query.html to learn more about the vsl-query language.
varnishncsa
If you want Apache-style logging, you can use the following command:
varnishncsa -g request -q "ReqUrl ~ '^/vote'"
You can also write these logs to a logfile:
varnishncsa -a -w /var/log/varnish/vote_access.log -g request -q "ReqUrl ~ '^/vote'"
Both varnishncsa and varnishlog binaries can be daemonized using the -D parameter
See http://varnish-cache.org/docs/6.5/reference/varnishncsa.html to learn more about varnishncsa. There is also a section in the docs about including custom fields into your varnishncsa output.
syslog from VCL
If you use the following snippet, you can log vote requests to syslog:
vcl 4.1;
import std;
sub vcl_recv {
if(req.url ~ "^/vote") {
std.syslog(6, "Vote request captured: " + req.url);
}
}
This is boilerplate VCL that cannot just be copy/pasted like that. Please make sure to add import std; to your VCL file, and use std.syslog() to log to your local syslog facility.
See http://varnish-cache.org/docs/6.5/reference/vmod_std.html#void-syslog-int-priority-string-s to learn more about std.syslog().

Why is my screen session terminating in the middle of a server startup?

Some background first, I am running Ubuntu 64-bit server on a machine running ESXi. I have just installed this VM today specifically for this task. This task is to run a tModLoader server with as little outside interference as possible.
I set up firewalld with a service for terraria that looks like this:
<?xml version="1.0" encoding="utf-8"?>
<service>
<short>Terraria</short>
<description>Open TCP port 7777 for incoming Terraria client connections.</description>
<port protocol="tcp" port="7777"/>
</service>
I set up UFW to allow only ssh and port 7777/tcp.
I added a rule for iptables with this command:
sudo iptables -A INPUT -p tcp --dport 7777 -j ACCEPT
I created a user called "terraria" with options -r -m -d
I set up the server files in /opt/terraria with the exception of the ModLoader files which are installed in two locations because they are dependent on the user that starts the process. The location of those are ~/.local/share/Terraria/ModLoader
I created a service for terraria in /etc/systemd/system/terraria.service that looks like this:
[Unit]
Description=server daemon for terraria
[Service]
Type=forking
User=terraria
KillMode=none
ExecStart=/usr/bin/screen -dmS terraria /bin/bash -c "/opt/terraria/tModLoaderServer -config /opt/terraria/serverconfig.txt"
ExecStop=/usr/local/bin/terrariad exit
[Install]
WantedBy=multi-user.target
I made a script that allows me to easily access the screen session the service starts in:
#!/usr/bin/env bash
send="`printf \"$*\r\"`"
attach='script /dev/null -qc "screen -r terraria"'
inject="screen -S terraria -X stuff $send"
if [ "$1" = "attach" ] ; then cmd="$attach" ; else cmd="$inject" ; fi
if [ "`stat -c '%u' /var/run/screen/S-terraria/`" = "$UID" ]
then
$cmd
else
su - terraria -c "$cmd"
fi
With all of that out of the way, the issue I am running into is that the service starts, and I can attach to the screen session while the service is running, however the screen session terminates after a few seconds while the server is starting up. I have no idea why that happens. Starting the server as my own user seems to work properly, but I need it to be able to run as a service with a system user so that the server will automatically run upon boot.
As a side note the config file doesn't work oddly.
Any ideas and help on this issue would be greatly appreciated.

psql command not responding

When I run the psql command just by itself (no arguments, postgres-10 installed, running from Debian Sid), it just stays blank and runs forever.
I tried checking the status with systemctl status postgresql and it says the server is active. It does the same when I run it by specifying the host as localhost.
This is the connection settings section of my postgresql.conf file:
listen_addresses = '*' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost'; use '*' for all
# (change requires restart)
port = 5432 # (change requires restart)
max_connections = 100 # (change requires restart)
# Note: Increasing max_connections costs ~400 bytes of shared memory per
# connection slot, plus lock space (see max_locks_per_transaction).
#superuser_reserved_connections = 3 # (change requires restart)
unix_socket_directories = '/var/run/postgresql' # comma-separated list of directories
# (change requires restart)
#unix_socket_group = '' # (change requires restart)
#unix_socket_permissions = 0777 # begin with 0 to use octal notation
# (change requires restart)
#bonjour = off # advertise server via Bonjour
# (change requires restart)
#bonjour = on # MJB
#bonjour_name = '' # defaults to the computer name
# (change requires restart)
# - Security and Authentication -
#authentication_timeout = 1min # 1s-600s
ssl = true # (change requires restart)
#ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL' # allowed SSL ciphers
# (change requires restart)
#ssl_prefer_server_ciphers = on # (change requires restart)
#ssl_ecdh_curve = 'prime256v1' # (change requires restart)
#ssl_renegotiation_limit = 512MB # amount of data between renegotiations
ssl_cert_file = '/etc/ssl/certs/ssl-cert-snakeoil.pem' # (change requires restart)
ssl_key_file = '/etc/ssl/private/ssl-cert-snakeoil.key' # (change requires restart)
#ssl_ca_file = '' # (change requires restart)
#ssl_crl_file = '' # (change requires restart)
#password_encryption = on
#db_user_namespace = off
# GSSAPI using Kerberos
#krb_server_keyfile = ''
#krb_caseins_users = off
# - TCP Keepalives -
# see "man 7 tcp" for details
#tcp_keepalives_idle = 0 # TCP_KEEPIDLE, in seconds;
# 0 selects the system default
#tcp_keepalives_interval = 0 # TCP_KEEPINTVL, in seconds;
# 0 selects the system default
#tcp_keepalives_count = 0 # TCP_KEEPCNT;
# 0 selects the system default
Where are the places I should look in order to address this problem?
So we probably need a bit more information. Are you dropping to a terminal and running psql? And if so, are you running JUST psql with no arguments (ie $ psql <enter>?
When you run the psql command by itself, it will try to connect to a Postgres server on a socket or at 127.0.0.1. If it's hanging, it's probably trying to connect to the 127.0.0.1 and either the firewall is dropping packets or there's nothing responding, causing it to hang. If you wait long enough, it should time out.
Maybe a better question is "what are you expecting to happen?" Is there a Postgres server running on the same machine as the terminal? Are you sure it's running? Is it installed a service? If so, does sudo systemctl status postgresql-10 indicate that it's running? What happens if you run psql -h 127.0.0.1? Is Postgres configured to listen on a network interface? Is there a firewall blocking port 5432? Is Postgres running on port 5432?
...And the list of potential questions goes on. Try to nail down some answers to the question above. Your question is the SO equivalent of "my car doesn't work- what's wrong?"; we need some more information in order to help you out. :)

How to Start Snort on OpenWRT?

Recently I replace my router OS with OpenWRT, and I've install snort(2.9) on it with :
opkg install snort
My One and Only rule in /etc/snort/rules/local.rules :
alert icmp any any -> [My Router Private IP like : 192.168.0.1] any (msg: "NMAP ping sweep Scan"; dsize:0;sid:10000004; rev: 1;)
The problem is when i run :
snort -A console -q -c /etc/snort/snort.conf -i br-lan --daq-dir /usr/lib/daq
On the command line, it is Okay and it detects some Nmap scan attacks and wrote the alerts in console :
04/12-08:19:50.152690 [**] [1:10000005:2] NMAP TCP Scan [**] [Priority: 0] {TCP} 192.168.0.10:46287 -> 192.168.0.1:22
and log file, But when I start the service by :
/etc/init.d/snort start
Nothing happens and no log file created when I use same Nmap command (nmap -sX -p22 192.168.0.1).
My Questions are :
Why the server is not running ?
There is no way detecting if every thing okay with service without Systemctl.
Why the log created when i run snort command is nonsense?
When I type for example cat /var/log/snort/snort.log.1523473976 I get :
�����Z�^8Mvv�n6(爈���Ehu�##A3��<���
in the console.
PS :
1 -cat /etc/init.d/snort :
#!/bin/sh /etc/rc.common
# Copyright (C) 2015 OpenWrt.org
START=90
STOP=10
USE_PROCD=1
PROG=/usr/bin/snort
validate_snort_section() {
uci_validate_section snort snort "${1}" \
'config_file:string' \
'interface:string'
}
start_service() {
local config_file interface
validate_snort_section snort || {
echo "validation failed"
return 1
}
procd_open_instance
procd_set_param command $PROG "-c" "$config_file" "-q" "--daq-dir" "/usr/lib/daq/" "-i" "$interface" "-s" "-N"
procd_set_param file $CONFIGFILE
procd_set_param respawn
procd_close_instance
}
stop_service()
{
service_stop ${PROG}
}
service_triggers()
{
procd_add_reload_trigger "snort"
procd_add_validation validate_snort_section
}
2- I actually followed This link to configure. but I uncomment and set config logdir: to /var/log/snort/ .
(Any help would be greatly appreciated)
It's set to localhost by default - lo
Check the output of uci show snort.snort.interface
You can change it with uci set snort.snort.interface=br-lan
Restart snort to confirm it has actually taken the new parameters - /etc/init.d/snort restart
You can view the whole command line in top/htop
If all is good, save the changes to UCI: uci commit
And your logs are binary with that config, not text, I have mine log to syslog and send them to a remote rsyslog server, in snort.conf - output alert_syslog: LOG_AUTH LOG_ALERT

How to configure https_check URL in nagios

I have installed Nagios (Nagios® Core™ Version 4.2.2) in Linux Server.I have written a JIRA URL check using check_http for HTTPS url.
It should get a response 200, but It gives response HTTP CODE 302.
[demuc1dv48:/pkg/vdcrz/Nagios/libexec][orarz]# ./check_http -I xx.xx.xx -u https://xxx.xxx.xxx.com/secure/Dashboard.jspa -S CONNECT
SSL Version: TLSv1
HTTP OK: HTTP/1.1 302 Found - 296 bytes in 0.134 second response time |time=0.134254s;;;0.000000 size=296B;;;
So I configured the same in the nagios configuration file.
define command{
command_name check_https_jira_prod
command_line $USER1$/check_http -I xxx.xxx.xxx.com -u https://xxx.xxx.xxx.com/secure/Dashboard.jspa -S CONNECT -e 'HTTP/1.1 302'
}
Now my JIRA server is down, But it is not reflected in the nagios check.The nagios response still shows HTTP code 302 only.
How to fix this issue?
You did not specify, but I assume you defined your command in the Nagios central server commands.cfgconfiguration file, but you also need to define a service in services.cfg as services use commands to run scripts.
If you are running your check_httpcheck from a different server you also need to define it in the nrpe.cfg configuration file on that remote machine and then restart nrpe.
As a side note, from the output you've shared, I believe you're not using the flags that the check_http Nagios plugin supports correctly.
From your post:
check_http -I xxx.xxx.xxx.com -u https://xxx.xxx.xxx.com/secure/Dashboard.jspa -S CONNECT -e 'HTTP/1.1 302'
From ./check_http -h:
-I, --IP-address=ADDRESS
IP address or name (use numeric address if possible to bypass DNS lookup).
You are using a host name instead (xxx.xxx.xxx.com )
-S, --ssl=VERSION
Connect via SSL. Port defaults to 443. VERSION is optional, and prevents auto-negotiation (1 = TLSv1, 2 = SSLv2, 3 = SSLv3).
You specified CONNECT
You can't get code 200 unless you set follow parameter in chech_http script.
I suggest you to use something like this:
./check_http -I jira-ex.telefonica.de -u https://xxx.xxx.xxx.com/secure/Dashboard.jspa -S -f follow
The -f follow is mandatory for your use case.

Resources