Varnish 4 Basic authentication - varnish

I have to cache multiple backend servers, I switch from Nginx to Varnish and finally discover 2 server need to run HTTP Basic Authentication.
I try this link http://blog.tenya.me/blog/2011/12/14/varnish-http-authentication and it not work for me (they ran Varnish 3)
Is there an easy way to configure Basic Authentication in Varnish 4?

You can use the VMOD basicauth
Install the Varnish VMOD
First you need to install it. Download the source from the Git repo for basicauth. Extract into your homedir e.g. ~/vmod-basicauth/
You'll also need the Varnish source to build the VMOD.
In Debian/Ubuntu type
apt-get source varnish
This will copy the source to your pwd.
Then do this to install it. Note that you need to change the paths according to your setup and version of varnish
cd ~/vmod-basicauth
./configure VARNISHSRC=$HOME/varnish-4.0.2
make
sudo make install
sudo make check
Update
It seems like the source have been removed from the Ubuntu and Debian package repos (most likely by accident).
Download the source directly from Git (v4.0.2)
Make Varnish
You'll have to "make" the downloaded source
cd ~
wget https://github.com/varnish/Varnish-Cache/archive/varnish-4.0.2.zip
unzip varnish-4.0.2.zip
cd Varnish-Cache-varnish-4.0.2
sudo ./autogen.sh
sudo ./configure --prefix=/usr
sudo make
Note that you don't have to install the source, so don't "make-install" because that might mess up your current installation.
Build & install VMOD
cd ~
./configure VARNISHSRC=$HOME/Varnish-Cache-varnish-4.0.2
make
sudo make install
sudo make check
It might be that you also have to specify your VMOD install directory if it can't be autodetected. If ./configure fails try this
./configure VARNISHSRC=$HOME/Varnish-Cache-varnish-4.0.2 VMODDIR=/usr/lib/varnish/vmods/
Some build dependencies
I often require alot of different build dependencies so I often install these when I setup a new Varnish server.
sudo apt-get install git-core zlib1g-dev automake build-essential libtool libssl-dev libreadline-dev libyaml-dev libsqlite3-dev ncurses-dev sqlite3 libxml2-dev libxslt1-dev libpcre3-dev libcurl4-openssl-dev python-docutils python-software-properties libvarnishapi-dev
Configure Varnish to use the VMOD
It uses a .htpasswd file for authentication instead of storing the password directly in the VCL.
Make sure to change "/var/www/.htpasswd" to the path of your htpasswd file.
#default.vcl
import basicauth;
sub vcl_recv {
if (!basicauth.match("/var/www/.htpasswd", req.http.Authorization)) {
return(synth(401, "Authentication required"));
}
}
#Prompt the user for a password
sub vcl_synth {
if (resp.status == 401) {
set resp.http.WWW-Authenticate = "Basic";
}
}

this also works:
sub vcl_recv {
if (! req.http.Authorization ~ "Basic Zm9vOmJhcg==") {
return(synth(401, "Authentication required"));
}
unset req.http.Authorization
}
sub vcl_synth {
if (resp.status == 401) {
set resp.status = 401;
set resp.http.WWW-Authenticate = "Basic";
return(deliver);
}
}
src: http://blog.tenya.me/blog/2011/12/14/varnish-http-authentication/#comment-2882579903

For anyone who follows these steps on Debian Jessie - you may come across a couple of issues when building Varnish from source.
That automake requires subdir-options specified in the configure.ac line 18
AM_INIT_AUTOMAKE([1.11 foreign color-tests parallel-tests subdir-options])
The Makefiles in the bin/varnishadm and bin/varnishhist require the variable $(top_srcdir) replaced with ../../ due to a bug in variable expansion in automake (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=402727)
varnishadm_SOURCES = \
varnishadm.c \
../../lib/libvarnish/vas.c \
../../lib/libvarnish/vsa.c \
../../lib/libvarnish/vtcp.c \
../../lib/libvarnish/vss.c
Fix those and then you can follow the instructions in the answer by jacob-rastad above.
I have made some further notes here : http://www.blue-bag.com/blog/compiling-varnish-modules

This is how I made basic authentication VMOD working with Varnish 4.1 in my Docker container https://github.com/blmr/varnish-basic-auth-docker
1) Install dependencies
apt-get install -y apt-transport-https \
&& apt-get install -y git-core zlib1g-dev automake build-essential libtool libssl-dev \
libreadline-dev libyaml-dev libsqlite3-dev ncurses-dev sqlite3 libxml2-dev libxslt1-dev \
libpcre3-dev libcurl4-openssl-dev python-docutils python-software-properties libvarnishapi-dev
2) Add Varnish repo
curl https://repo.varnish-cache.org/GPG-key.txt | apt-key add -
printf "deb https://repo.varnish-cache.org/ubuntu/ trusty varnish-4.1 \ndeb-src https://repo.varnish-cache.org/ubuntu/ trusty varnish-4.1" >> /etc/apt/sources.list.d/varnish-cache.list && apt-get update
3) Install Varnish 4.1
apt-get install -qy varnish
4) Get Varnish source and compile it
apt-get source varnish && rm *.diff.gz *.dsc *.tar.gz \
&& mv varnish* varnish-source && cd varnish-source && ./autogen.sh && ./configure --prefix=/usr/sbin && make
5) Get Varnish basic auth VMOD and compile it
git clone http://git.gnu.org.ua/cgit/vmod-basicauth.git && cd vmod-basicauth \
&& git clone http://git.gnu.org.ua/repo/acvmod.git && ./bootstrap \
&& ./configure VARNISHSRC=/varnish-source VMODDIR=/usr/lib/varnish/vmods/ && make && make install && make check
6) Update default.vcl
sub vcl_recv {
if (!basicauth.match("/etc/varnish/htpasswd", req.http.Authorization)) {
return(synth(401, "Authentication required"));
}
}
sub vcl_synth {
if (resp.status == 401) {
set resp.http.WWW-Authenticate = "Basic";
}
}

Related

Use latest curl version on docker

I've the following docker image
FROM debian:10.7
RUN apt-get update && \
apt-get install --yes --no-install-recommends curl
when I run it and use curl --version I got version 7.64 but the latest is 7.74
https://curl.haxx.se/download.html
How should I upgrade the curl to the latest version 7.74 ?
is there a way to do it?
You can use the downloaded packages directly to solve this problem by installing with the make command.
FROM debian:10.7
RUN apt-get update && \
apt-get install --yes --no-install-recommends wget build-essential libcurl4 && \
wget https://curl.se/download/curl-7.74.0.tar.gz && \
tar -xvf curl-7.74.0.tar.gz && cd curl-7.74.0 && \
./configure && make && make install
Note that it requires running ./configure.
After installation curl will work perfectly in the version you need, in this case, version 7.74.0.
If you want to optimize your container, remove the build-essential package, it alone will consume more than 200MB of storage. To do this, add at the end of the compilation:
apt-get autoremove build-essential
You could clone curl source code from Git and build and install it
manually in your Dockerfile like that:
FROM debian:10.7
RUN apt-get update && \
apt-get install --yes --no-install-recommends autoconf libtool automake make git
RUN GIT_SSL_NO_VERIFY=1 git clone https://github.com/curl/curl --depth 1
RUN cd curl && ./buildconf && ./configure && make -j$(nproc) install && \
echo /usr/local/lib >> /etc/ld.so.conf.d/local.conf && ldconfig
After docker run:
root#d7ea28ad22e2:/# curl --version
curl 7.75.0-DEV (x86_64-pc-linux-gnu) libcurl/7.75.0-DEV
Release-Date: [unreleased]
Protocols: dict file ftp gopher http imap mqtt pop3 rtsp smtp telnet tftp
Features: alt-svc AsynchDNS IPv6 Largefile UnixSockets
The latest version of Curl on Alpine Linux is available from the following link:
https://github.com/curl/curl-docker/blob/master/alpine/latest/Dockerfile

Azure CLI in a Docker Container

I have an Ubuntu 18.04 Docker image that I need Azure CLI installed in. For a Docker image, it seems the preferred way is to use pip, however, I have other pip Azure libraries needed in the container that overlap with the CLI install and get blown away because Azure CLI requires older versions; then making it so I can't run my python scripts.
I have tried to use the script installer but that hasn't worked and errored out because I believe the install is interactive.
The last option I can find is the manual apt install, though I am not sure this is a correct way nor do I have a good idea of how to replicate that in a Dockerfile.
Is there a preferred/good way of getting Azure CLI in a container not using pip?
FROM ubuntu:18.04
RUN apt-get update && apt-get -y upgrade && \
apt-get -f -y install curl python3-pip python-pip && \
pip3 install --upgrade pip && \
pip2 install --upgrade pip && \
pip3 install azure-storage-blob==12.3.0 & \\
pip3 install azure-cli
I have a preference to use the package manager to install dependencies, it's why I will do something like that:
Add base dependencies for https repostory and curl
Add the gpg key and repository for the CLI
Add the CLI
This is the Dockerfile with thoses steps:
FROM ubuntu:18.04
RUN apt-get update && apt-get -y upgrade && \
apt-get -f -y install curl apt-transport-https lsb-release gnupg python3-pip python-pip && \
curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > /etc/apt/trusted.gpg.d/microsoft.asc.gpg && \
CLI_REPO=$(lsb_release -cs) && \
echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ ${CLI_REPO} main" \
> /etc/apt/sources.list.d/azure-cli.list && \
apt-get update && \
apt-get install -y azure-cli && \
rm -rf /var/lib/apt/lists/*
In addition, I clean up the apt cache by removing /var/lib/apt/lists. Tt reduces the image size, since the apt cache is not stored in a layer.
See: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt#option-1-install-with-one-command
RUN curl -LsS https://aka.ms/InstallAzureCLIDeb | bash && rm -rf /var/lib/apt/lists/*

Unable to install php7.1-soap in Ubuntu 19.04

I am running PHP 7.2 on an Apache server on Ubuntu, and I need to add the PHP SoapClient. However, when I try to install it using apt-get, I get the following error:
The following packages have unmet dependencies:
php7.2-soap : Depends: php7.2-common (= 7.2.24-0ubuntu0.19.04.2) but 7.2.26-1+ubuntu19.04.1+deb.sury.org+1 is to be installed
E: Unable to correct problems, you have held broken packages.
I've tried the following before reattempting:
sudo apt-get update
sudo apt-get clean && sudo apt-get update
sudo apt-get upgrade
sudo apt-get upgrade-dist
sudo apt-get -f install
sudo dpkg --configure -a
If it helps, when I run uname -r the output is
5.0.0-37-generic
Thanks in advance!
After a looooong time tackling this issue I finally got my head around it and found a solution :)
It seems the issue is I was using the ppa:ondrej/apache2 PPA which was installing the latest version of a selection of PHP packages, which at the time of writing is using PHP7.2.27. As far as I can tell, the most up-to-date version of the php7.2-soap package is for php7.2.24, meaning there is an unmet dependency for the php7.2-common package provided by the PPA.
So, the solution was to remove the PPA and downgrade the PHP packages. Here's a step by step for any poor souls who might encounter a similar issue:
First, if not already installed, install aptitude sudo apt-get install aptitude
Next, install ppa-purge:
mkdir ppa-purge && cd ppa-purge && wget http://mirror.pnl.gov/ubuntu/pool/universe/p/ppa-purge/ppa-purge_0.2.8+bzr56_all.deb && wget http://mirror.pnl.gov/ubuntu//pool/main/a/aptitude/aptitude_0.6.6-1ubuntu1_i386.deb && sudo dpkg -i ./*.deb
(I ran this in my user folder, i.e. ~/
Then I remove the PPA with the following commands:
sudo ppa-purge ppa:ondrej/apache2
sudo add-apt-repository --remove ppa:ondrej/apache2
sudo apt-get autoclean
Now we need to downgrade the dependencies. First I determined which version I needed. The package that was causing the unmet dependency was php7.2-common, so I ran sudo apt list -a php7.2-common. I chose the version which matched the original error message, in this case, 7.2.24-0ubuntu0.19.04.2.
So you are able to install a specific version by following the package with a = then a version number. If you, like me, are working on a live server, I'm going to shout this next bit about the next command you will write:
DO NOT USE THE -y TAG!!
CHOOSE "NO" WHEN IT ASKS YOU IF YOU WANT TO CONTINUE!!
If you do any of these, it will automatically remove any dependencies for php7.2-common and the PHP stop working on your site.
To determine which dependencies we need to update along with php7.2-common, I ran sudo apt-get install php7.2-common=7.2.24-0ubuntu0.19.04.2. It then showed me a list of other packages it would remove as well. I copied these dependencies then chose 'n' to cancel the install.
Next, I put the copied list into a text editor and used it to create the following script:
sudo apt-get install php7.2-common=7.2.24-0ubuntu0.19.04.2 \
libapache2-mod-php7.2=7.2.24-0ubuntu0.19.04.2 \
php7.2=7.2.24-0ubuntu0.19.04.2 \
php7.2-bcmath=7.2.24-0ubuntu0.19.04.2 \
php7.2-bz2=7.2.24-0ubuntu0.19.04.2 \
php7.2-cgi=7.2.24-0ubuntu0.19.04.2 \
php7.2-cli=7.2.24-0ubuntu0.19.04.2 \
php7.2-curl=7.2.24-0ubuntu0.19.04.2 \
php7.2-dev=7.2.24-0ubuntu0.19.04.2 \
php7.2-enchant=7.2.24-0ubuntu0.19.04.2 \
php7.2-fpm=7.2.24-0ubuntu0.19.04.2 \
php7.2-gd=7.2.24-0ubuntu0.19.04.2 \
php7.2-imap=7.2.24-0ubuntu0.19.04.2 \
php7.2-intl=7.2.24-0ubuntu0.19.04.2 \
php7.2-json=7.2.24-0ubuntu0.19.04.2 \
php7.2-ldap=7.2.24-0ubuntu0.19.04.2 \
php7.2-mbstring=7.2.24-0ubuntu0.19.04.2 \
php7.2-mysql=7.2.24-0ubuntu0.19.04.2 \
php7.2-odbc=7.2.24-0ubuntu0.19.04.2 \
php7.2-opcache=7.2.24-0ubuntu0.19.04.2 \
php7.2-pspell=7.2.24-0ubuntu0.19.04.2 \
php7.2-readline=7.2.24-0ubuntu0.19.04.2 \
php7.2-tidy=7.2.24-0ubuntu0.19.04.2 \
php7.2-xml=7.2.24-0ubuntu0.19.04.2 \
php7.2-xmlrpc=7.2.24-0ubuntu0.19.04.2 \
php7.2-zip=7.2.24-0ubuntu0.19.04.2
Finally, I ran this command. Instead of warning me about removing the packages, it warned me that these packages would be "downgraded", which is fine. I pressed Y and it reinstalled all the packages and viola! I was then able to install php7.2-soap :)
See these links which I credit to finding a solution for this:
https://askubuntu.com/a/92021
https://appuals.com/fix-unmet-dependencies-error-ubuntu/
I have a similar issue one month ago with a Debian 9 and PHP 7 Did you consider to upgrade your PHP to the latest stable version before installing the PHP Client ?

Setting Up a Debian Docker image with older boost library

I have an already compiled Linux app which has become dated. To use it, I want to create a Docker image and an appropriate environment to work with. My problem is that is app requires an older version of the boost libraries. 1.57.0 to be specific.
I have been able to get boost installed (I believe correctly) but the app errors out.
The error that I am getting is:
undefined symbol: _ZN5boost15program_options3argE
I am hoping someone has experience with this. Briefly, my pipeline is:
get the rocker/verse Docker image that has Debian and R and some more goodies I need.
Bash in to it, apt-get install ... etc.
sudo apt-get update && sudo apt-get upgrade
sudo apt-get install build-essential g++ python-dev autotools-dev libicu-dev build-essential libbz2-dev libboost-all-dev
cd home
wget -O boost_1_57_0.tar.gz https://sourceforge.net/projects/boost/files/boost/1.57.0/boost_1_57_0.tar.gz/download
tar xzvf boost_1_57_0.tar.gz
cd boost_1_57_0
./bootstrap.sh --with-libraries=atomic,chrono,context,coroutine,container,date_time,exception,filesystem,graph,graph_parallel,iostreams,locale,log,math,mpi,program_options,python,random,regex,serialization,signals,system,test,thread,timer,wave
./b2 toolset=gcc cxxflags=-std=gnu++0x
sudo ./b2 install
sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade
-----Edit: added additional bash code that was missing here
sudo sh -c 'echo "/usr/local/lib" >> /etc/ld.so.conf.d/local.conf’

implement CVE-2017-7494 (samba cry) on ubuntu server 16.04

Recently my experiment needs to implement CVE-2017-7494 (the so called sambacry), and I am trying to rebuild the vulnerable environment
I am new to this, not very sure how to set all the path and conf right
here is what I did:
# wget https://download.samba.org/pub/samba/stable/samba-4.5.9.tar.gz
# apt-get install acl attr autoconf bison build-essential \
debhelper dnsutils docbook-xml docbook-xsl flex gdb krb5-user \
libacl1-dev libaio-dev libattr1-dev libblkid-dev libbsd-dev \
libcap-dev libcups2-dev libgnutls-dev libjson-perl \
libldap2-dev libncurses5-dev libpam0g-dev libparse-yapp-perl \
libpopt-dev libreadline-dev perl perl-modules pkg-config \
python-all-dev python-dev python-dnspython python-crypto \
xsltproc zlib1g-dev
Reference about the above package.
# tar -xvf samba-4.5.9.tar.gz
# cd samba-4.5.9
# ./configure
# make
# make install
after that I found it installed under /local, and cannot start samba normally because, say, smbd not found, etc
I think it's a problem of path and config file then I tried this to fix it.
But didn't get well realizing.
Would anyone please help?
Since you did not specify a path in your configure parameters, it should be by default at /usr/local/samba/sbin/smbd.
You can try running this in your shell (and add it to your profile) to add it to your path:
export PATH=/usr/local/samba/bin/:/usr/local/samba/sbin/:$PATH

Resources