Places like quay.io provide an analysis of known vulnerabilities for the container images they host. How can I connect that to my deployed software in Kubernetes? In other words, I want a process that will periodically:
query the apiserver to list all pods
get the image associated with each container in the pod
check each image against a known vulnerability list.
By analogy, we can do this at the OS level by using built-in tools or external things like Nessus. I've found plenty of tools that can do a static analysis of container images; that's like the CVE database of .apt packages. How do I apply that list of image vulnerabilities to a running system?
I've found plenty of tools that can do a static analysis of container images;
That is the preferred approach indeed.
As an alternative to connect to running container, and get their image (that a docker inspect might give you: docker inspect --format='{{.Config.Image}}' $INSTANCE_ID), you might consider:
doing this analysis in advance (at the image level)
signing the image
only allowing to run container from signed images
That is what describes Antonio Murdaca (Senior Engineer at Red Hat Inc. and one of the CRI-O guys. Docker (Moby) Core Maintainer) in "Secure your Kubernetes production cluster".
digitally sign a container image with a GPG key generating its detached signature, put the signature where it can be retrieved and verified and finally validate it when someone requests the image back on a host.
The story behind all this is pretty simple: if the signature for a given image is valid, the node is allowed to pull the image and run your containerwith it. Otherwise, your node rejects the image and fail to run your container.
That way, you only allow for running container whose image have been pre-validated.
Related
I have to deploy my application software which is a linux based package (.bin) file on a VM instance. As per system requirements, it needs minimum 8vCPUs and 32GB RAM.
Now, i was wondering if it is possible to deploy this software over multiple containers that load share the CPU and RAM power in the kubernetes cluster, rather than installing the software on a single VM instance.
is it possible?
Yes, it's possible to achieve that.
You can start using docker compose to build your customs docker images and then build your applications quickly.
First, I'll show you my GitHub docker-compose repo, you can inspect the folders, they are separated by applications or servers, so, one docker-compose.yml build the app, only you must run a command docker-compose up -d
if you need to create a custom image with docker you should use this docker command docker build -t <user_docker>/<image_name> <path_of_files>
<user_docker> = your docker user
<image_name> = the image name that you choose
<path_of_files> = somelocal path, if you need to build in the same folder you should use . (dot)
So, after that, you can upload this image to Dockerhub using the following commands.
You must login with your credentials
docker login
You can check your images using the following command
docker images
Upload the image to DockerHub registry
docker push <user_docker>/<image_name>
Once the image was uploaded, you can use it in different projects, make sure to make the image lightweight and usefully
Second, I'll show a similar repo but this one has a k8s configuration into the folder called k8s. This configuration was made for Google cloud but I think you can analyze it and learn how you can start in your new project.
The Nginx service was replaced by ingress service ingress-service.yml and https certificate was added certificate.yml and issuer.yml files
If you need dockerize dbs, make sure the db is lightweight, you need to make a persistent volume using PersistentVolumeClaim (database-persistent-volume-claim.yml file) or if you use larger data onit you must use a dedicated db server or some db service in the cloud.
I hope this information will be useful to you.
There are two ways to achieve what you want to do. The first one is to write a dockerfile and create the image. More information about how to write a dockerfile can be found from here. Apart for that you can create a container from a base image and install all the software and packages and export it as a image. Then you can upload to a docker image repo like Docker Registry and Amazon ECR
We have On-premises software docker image.Also, We have licensing for application security and code duplication.
But to add extra security is it possible to do any of the below ?
Can we lock docker image such that no one can copy or save running container and start new docker container in another environment.
or is it possible to change something in docker image while build that may prevent user to login inside container.
Goal is to secure docker images as much as possible in terms of duplication of the docker images and stop login inside running container to see the configuration.
No. Docker images are a well known format with an open specification that is essentially a set of tar files and some json metadata. Once someone has this image, they can do with it what they want. This includes running it with any options they'd like, coping it, and extending it with their own changes.
The company where I work (strictly regulated/audited environment) is yet to embrace containers but would like to adopt them for some applications. There is the view that as the image build process issues commands as root (or could be overridden by the user by use of the USER command), that building (not running) a container is effectively giving a user unfettered access as root during the build process. This is anathema to them and goes against all manner of company policies. Access to certain commands for computers is restricted via PowerBroker, i.e. access to certain commands requires explicit permissioning and is logged/subject to audit.
We need to allow container images to be built by a CI/CD system as well as ideally to allow developers to be able to build containers locally. Containers will generally be run in Kubernetes, but may be run directly on a VM. I'd like to be able to have CI build agents spin up on demand, as there are a lot of developers, so I want to run the build process within Kubernetes.
What is the best practice for building docker containers in this sort of environment please? Should we look to restrict access to commands within the Dockerfile?
My current thinking for this approach:
CI/CD:
Define "company-approved" image to act as build agent within
Kubernetes.
Build image defines a user that the build process runs as (not
root).
Build agent image contains PowerBroker, enabling locking down access
to sensitive commands.
Scan docker file for use of user command and forbid this.
Build agent runs docker-in-docker, as per here
(https://applatix.com/case-docker-docker-kubernetes-part-2/). This
achieves isolation between multiple build instances whilst ensuring
all containers are controlled via Kubernetes.
Images are scanned for security compliance via OpenSCAP or similar.
Passing the scan is part of the build process. Passing the scan
allows the image to be tagged as compliant and pushed to a registry.
I'm uncomfortable with the thinking around (4), as this seems a bit rule bound (i.e. it's a sort of blacklist approach) and I'm sure there must be a better way.
Developer's localhost:
Define "company-approved" base images (tagged as such inside a
trusted registry).
Image defines a user that the build process runs
as (not root).
Base image contains PowerBroker, enabling locking
down access to sensitive commands.
Create wrapper script on localhost that wraps docker build. No direct access to docker build: user must use script instead. Access to script is secured via PowerBroker. Script can also scan docker file for use of user command and forbid this.
Pushing of images to registry requires tagging which requires scanning for security compliance via OpenSCAP or similar as above.
I'd like to use the OpenSCAP results plus the CI system to create an audit trail of the images that exist; similarly for the deploy process. The security team that monitor for CVEs etc should be able to understand what containers exist and have been deployed and be able to trigger rebuilds of images to make use of updated libraries, or to flag up to developers when containers need to be rebuilt/redeployed. I want to be able to demonstrate that all containers meet a security configuration policy that is itself defined as code.
Is this a sensible way to go? Is there even a risk for allowing a user to build (but not run) a container image without restriction? If there is not, what's the best way to ensure that a foolish/malicious developer has not undone the best practices inside the "approved base image", other than a manual code review (which is going to be done anyway, but might miss something)?
By the way, you must assume that all code/images are hosted in-house/on-premises, i.e. nothing is allowed to use a cloud-based product/service.
When docker build runs each layer executes in the context of a container. So the risks presented by that command executing are constrained by what access is available to the container.
Locking down the build environment could be achieved by restricting what the Docker engine instance which will complete the build can do.
Things like ensuring that user namespaces are used can reduce the risk of a command run inside a container having a wider effect on the environment.
Of course that doesn't mitigate the risks of a developer curl|bashing from an untrusted location, but then what's to stop that being done outside of Docker? (i.e. what additional risk is being introduced by the use of Docker in this scenario)
If you have a policy of restricting externally hosted code, for example, then one option could be to just restrict access from the Docker build host to the Internet.
If you're making use of Kubernetes for the build process and are concerned about malicious software being executed in containers, it could be worth reviewing the CIS Kubernetes standard and making sure you've locked down your clusters appropriately.
There is the view that as the image build process issues commands as
root (or could be overridden by the user by use of the USER command),
that building (not running) a container is effectively giving a user
unfettered access as root during the build process
This view is not correct. When you build an image, all what you are doing is creating new docker layers (files) which are stored under /var/lib/docker/aufs/layers. There are simply no security concerns when building docker images.
There are tools to analyze the security of images you already built. One is the image analyzer built into Dockerhub.
How to ensure, that docker container will be secure, especially when using third party containers or base images?
Is it correct, when using base image, it may initiate any services or mount arbitrary partitions of host filesystem under the hood, and potentially send sensitive data to attacker?
So if I use third party container, which Dockerfile proves the container to be safe, should I traverse the whole linked list of base images (potentially very long) to ensure the container is actually safe and does what it intends of doing?
How to ensure the trustworthy of docker container in a systematic and definite way?
Consider Docker images similar to android/iOS mobile apps. You are never quite sure if they are safe to run, but the probability of it being safe is higher when it's from an official source such as Google play or App Store.
More concretely Docker images coming from Docker hub go through security scans details of which are undisclosed as yet. So chances of a malicious image pulled from Docker hub are rare.
However, one can never be paranoid enough when it comes to security. There are two ways to make sure all images coming from any source are secure:
Proactive security: Do security source code review of each Dockerfile corresponding to Docker image, including base images which you have already expressed in question
Reactive security: Run Docker bench, open sourced by Docker Inc., which runs as a privileged container looking for runtime known malicious activities by containers.
In summary, whenever possible use Docker images from Docker hub. Perform security code reviews of DockerFiles. Run Docker bench or any other equivalent tool that can catch malicious activities performed by containers.
References:
Docker security scanning formerly known as Project Nautilus: https://blog.docker.com/2016/05/docker-security-scanning/
Docker bench: https://github.com/docker/docker-bench-security
Best practices for Dockerfile: https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/
Docker images are self-contained, meaning that unless you run them inside a container with volumes and network mode they have no way of accessing any network or memory stack of your host.
For example if I run an image inside a container by using the command:
docker run -it --network=none ubuntu:16.04
This will start the docker container ubuntu:16.04 with no mounting to host's storage and will not share any network stack with host. You can test this by running ifconfig inside the container and in your host and comparing them.
Regarding checking what the image/base-image does, a conclusion from above said is nothing harmful to your host (unless you mount your /improtant/directory_on_host to container and after starting container it removes them).
You can check what an image/base-image conatins after running by checking their dockerfile(s) or docker-compose .yml files.
I'd like to build my own layer on top of a public Docker image. Fine, I know how to do that. However, my layer will contain proprietary code that I can't share in a public docker image. I do, however want to be able to share it among servers inside my organization.
Is my only option to run my own instance of docker registry? Or are there workflows that allow moving of layers/images around without a central repository?
You can:
run your own docker registry,
use one of the private registry services available out there,
move images around with docker save and docker load,
build images locally each time (not recommended, but eh!)
To expand on Jerome's answer, we are one of the private Docker registry services: Quay.io
We offer a robust permissions model that should be sufficient for your needs, including teams, shared organizations, and individual repository level controls.
We have many companies using us to store their proprietary code and to distribute it within their datacenters. If you do consider using us, I am sure your first questions will deal with the security of our service, for which we have dedicated a special page on our site: Security
We run Gandalf.io, an affordable private docker registry service. You'll like Gandalf.io if you need to get started with private dockers quickly and inexpensively. Its still pretty much in the early stages so we're offering just one service - private docker sharing among teams using the docker cli - and its works well for that use case.