Running a container in privileged mode is discouraged for security reasons.
For example: https://www.cncf.io/blog/2020/10/16/hack-my-mis-configured-kubernetes-privileged-pods/
It seems obvious to me that is is preferable to avoid privileged containers when a non-privileged container instead would be sufficient.
However, let's say I need to run a service that requires root access on the host to perform some tasks. Is there an added security risk in running this service in a privileged container (or with some linux capabilities) rather than, for example, a daemon that runs as root (or with those same linux capabilities)? What is the added attack surface?
If a hacker manages to run a command in the context of the container, all right, it is game over. But what kind of vulnerability would allow him to do so that couldn't also be exploited in the case of the aforementioned daemon (apart from sharing the kubeconfig file thoughtlessly)?
Firstly and as you said, it is important to underline that running a container in privileged mode is highly discouraged for some obvious security reasons and here is why:
The risk of running a privileged container lies in the fact that it has access to the host's resources, including the ability to modify the host's system files, access sensitive information, and gain elevated privileges. Basically as it provides more permissions to the container than it would have in a non-privileged mode it significantly increase the risk of a attack surface.
If a hacker gains access to the privileged container, he can potentially access and manipulate the host system and potentially move laterally to other systems and compromising the security of the entire of your infrastructure. A similar vulnerability in a daemon running as root or with additional Linux capabilities would carry the same risk, as the hacker would have access to the same resources and elevated privileges.
In both cases, it is very important to follow best practices for securing the system, such as reducing the attack surface, implementing least privilege, and maintaining proper network segmentation to reduce the risk of compromise.
In this security article written by the astra security team they have mentioned PHP Remote Code Execution Vulnerability (2020) using which the attacker can get hold of your server. If this process is being run by a non root user the attack surface will be reduced but if the same service is having root user access the attacker can get access to remaining containers. This is the reason why it’s always preferred to have least privileged access configured for all the services, also go through this document for getting an overview on attacks that can be performed using privileged containers.
Related
The bounty expires in 5 days. Answers to this question are eligible for a +50 reputation bounty.
PJEM is looking for an answer from a reputable source:
I need to know which combination of security context config can have a security escalation, and also can control other pods in the cluster
I want to check if pod in the cluster running as privileged pods, which can indicate that we may have security issue, so I check if
privileged: true
However under the
securityContext: spec there is additional fields like
allowPrivilegeEscalation
RunAsUser
ProcMount
Capabilities
etc
Which may be risky (not sure about it) ,
My question is in case the pod is marked as privileged:false and the other fields are true like the following example,if this indicate some security issue ? Does this pods can do some operation on other pods etc , access external data?
For example the following configuration which indicate the the pod is not privileged but allowPrivilegeEscalation: true
securityContext:
allowPrivilegeEscalation: true
privileged: false
I want to know which securityContext combination of pod config can control other pods/process in the cluster ?
The securityContext are more related to the container itself and some access to the host machine.
The allowPrivilegeEscalation allow a process to gain more permissions than its parent process. This is more related to setuid/setgid flags in binaries, but inside a container there is no much to get worried about.
You can only control other containers in the host machine from inside a container if you have a hostPath volume, or something like that, allowing you to reach the .sock file as /run/crio/crio.sock or the docker.sock. Is pretty obvious that, if you are concerned about this, allowing requests to Docker API through the network should be disabled.
Of course, all of these access are ruled by DAC and MAC restrictions. This is why podman uidmap is better, because the root inside the container do not have the same root id outside the container.
From Kubernetes point of view, you don't need this kind of privileges, all you need is a ServiceAccount and the correct RBAC permissions to control other things inside Kubernetes. A ServiceAccount binded to a cluster-admin ClusterRole can do anything in the API and much more, like adding ssh keys to the hosts.
If you are concerned about pods executing things in Kubernetes or in the host, just force the use of nonRoot containers, avoid indiscriminate use of hostPath volumes, and control your RBAC.
Openshift uses a very nice restriction by default:
Ensures that pods cannot run as privileged
Ensures that pods cannot mount host directory volumes
Requires that a pod is run as a user in a pre-allocated range of UIDs (openshift feature, random uid)
Requires that a pod is run with a pre-allocated MCS label (selinux related)
I don't answer exactly what you want, because I shifted the attention to RBAC, but I hope this can give you a nice idea.
Strictly in the scope of securityContext (as of Kubernetes 1.26 API), here's few things that may be risky:
Certainly risky
capabilities.add will add Linux capabilities (like CAP_SYS_TIME to set system time) to a container. The default depends on container runtime (see for example Docker default set of capabilities) and should be reasonably secure, but adding capabilities like CAP_SYS_ADMIN may represent a risk. Excessive capabilities outlines a few possible escalations.
privileged: true grants all capabilities, so you'll definitely want to check for that (as you already do).
allowPrivilegeEscalation: true is risky as it allow a process to gain more privileges than its parent.
procMount will allow a container mounting node's /proc and expose sensible host information.
windowsOptions may be risky. According to Kubernetes doc it enables privileged access to the Windows node. I don't know much about Windows security, but I'd say risky :-)
Maybe risky (though usually intended to restrict permissions)
runAsGroup and runAsUser may be risky when set to root / 0. Given that by default container runtime will probably run container as root already it's mostly used to restrict container's permissions to a non-root user. But if your container runtime is configured to run container as non-root by default, it might be used to bypass that and run a container with root.
seLinuxOptions may be used to provide an insecure SELinux context, but is usually intended to define a more secure context.
seccompProfile defines system calls a container is allowed to make. It may be used to get access to sensitive system calls, though it's usually intended to restrict them.
(probably) Not risky
readOnlyRootFilesystem (default false) will make container root system read-only.
runAsNonRoot (default false) will prevent a container from running as root
capabilities.drop will drop Linux capabilities, restricting further what a container can do.
You can read more on Configure a Security Context official doc
What about non-Security Context related risks?
Security Context is not the only thing you should be wary about: you should also consider volume mounts to unsecure locations, RBAC, network, Secrets, etc. A good overview is provided by Security Checklist.
There are many sources telling it is bad to run apps under root inside Docker container but they always refer to this link: https://blog.docker.com/2014/06/docker-container-breakout-proof-of-concept-exploit/ with issue fixed long ago because newer Docker versions whitelist kernel capabilities.
Therefore:
Were there any other Docker exploits that worked under container root user but didn't work under container non-root user?
Were there any linux kernel exploits that worked under container root user but didn't work under container non-root user?
So, this is skirting the question a little bit but I'm going to try my best to give you an informative and in-depth answer to help you understand the issues involved with running an application as root.
First off, this isn't a 100% definite no-go. You can run applications as root, and in some cases you may need to. But in software, we have something known as the Principle of Least Privilege, also known as the Principle of Least Authority in some areas. This is an important concept in computer security, promoting minimal privileges on computers, based on users' job necessities. Each system component or process should have the least authority necessary to perform its duties. This helps reduce the "attack surface" of the computer by eliminating unnecessary privileges that can result in network exploits and computer compromises. You can apply this principle to the computers you work on by ordinarily operating without administrative rights.
By unnecessarily running an application as root is giving the program permissions to do things that it does not need to do - such as perform system functions and to manage a variety of the operating system's configuration settings. If your application is a basic website filled with cooking recipes, it does not need access to the system configuration files.
Applications are meant to be run with non-administrative security (or as mere mortals) so you have to elevate their privileges to modify the underlying system. This is how the general security model has worked for years.
It also makes applications easier to deploy and adds a layer of scalability. In general, the fewer privileges an application requires the easier it is to deploy within a larger environment. Applications that install device drivers or require elevated security privileges typically have additional steps involved in their deployment. For example, on Windows a solution with no device drivers can be run directly with no installation, while device drivers must be installed separately using the Windows installer service in order to grant the driver elevated privileges.
I apologise if this does not answer your question, but I've done my best to explain why you should not run applications as root. I hope this helps!
To combat image misuse. Running applications as non-privileged user helps keeping system secure when image users misuse Docker - e.g. run with --privileged or mount system directories into container.
So in order to get MIT-SHM working between application running inside docker container and x11 running on the host, I have to pass --ipc host during starting the container. I've read the documentation about what it's supposed to do.
Assuming the application is NOT running as root (inside the container), what possible attack vectors does this open? In other words, how much does the --ipc host compromise the security?
In terms of attack surface --ipc=host removes a layer of security and creates new attack vectors as any application running on the host that misbehaves when presented with malicious data in shared memory segments can become a potential attack vector.
Performance-sensitive programs use shared memory to store and exchange volatile data (x11 frame buffers are one example). In your case the non-root user in the container has access to the x11 server shared memory.
Running as non-root inside the container should somewhat limit unauthorized access, assuming correct permissions are set on all shared objects. Nonetheless if an attacker gained root privileges inside your container they would have access to all shared objects owned by root (some objects might still be restricted by the IPC_OWNER capability which is not enabled by default).
You may ask yourself for each application on the host :
What are the odds of a compromise from maliciously crafted shared memory segments?
What are the consequences of a compromise? Is the application confined in any way?
As long as your image of the container is from a reliable source it shouldn't affect your host.
You can read about the ipc settings here..
https://docs.docker.com/engine/reference/run/#ipc-settings---ipc
I've seen plenty of ink spilled by now about how Docker is not sufficiently isolated to allow arbitrary containers to be run in a multi-tenant environment, and that makes sense. "If it's root in Docker, consider it root in the host machine." What about non-root though?
If I want to take some untrusted code and run it in a container, can it be done safely so long as the container is running as a non-root non-sudo user? What are the potential security pitfalls of doing something like that?
I'm fairly sure there are production applications doing this today (CI systems, runnable pastebins), but are they just lucky not to have had a determined attacker or is this a reasonable thing to do in a production system?
As of Docker v1.12, if one runs a container as a non-root user with user namespaces enabled, there are two levels of privilege escalation a malicious actor needs to perform in order to become root on host:
Escalate from non-root to root user inside container
Escalate to root user in container to root user on the host
So in case untrusted code is run inside a Docker container as non-root user, it will be slightly more difficult for an attacker to become root on host, since we add an extra step of becoming root inside container. That's the only advantage in terms of security compared to running containers with root privileges.
In case of privilege escalation through both layers of security, following should help restrict the attack surface:
Workloads(more specifically docker containers, in this context) with different trust levels should be isolated from each other by use of overlay networks following least privilege principle.
Enabling available Linux security module in enforcement mode(e.g. SELinux, AppArmor)
References:
Running with non-root privileges inside containers: https://groups.google.com/forum/#!msg/docker-user/e9RkC4y-21E/JOZF8H-PfYsJ
Overlay networks: https://docs.docker.com/engine/userguide/networking/get-started-overlay/
User namespaces: https://docs.docker.com/engine/security/security/#/other-kernel-security-features
All containers share the same kernel.
In case your un-trusted code manages to perform a kernel exploit, it can do whatever it wants on the host and/or any other running container.
A Docker blog post indicates:
Docker containers are, by default, quite secure; especially if you
take care of running your processes inside the containers as
non-privileged users (i.e. non root)."
So, what is the security issue if I'm running as a root under the docker? I mean, it is quite secure if I take care of my processes as non-privileged users, so, how can I be harmful to host in a container as a root user? I'm just asking it to understand it, how can it be isolated if it is not secure when running as root? Which system calls can expose the host system then?
When you run as root, you can access a broader range of kernel services. For instance, you can:
manipulate network interfaces, routing tables, netfilter rules;
create raw sockets (and generally speaking, "exotic" sockets, exercising code that has received less scrutiny than good old TCP and UDP);
mount/unmount/remount filesystems;
change file ownership, permissions, extended attributes, overriding regular permissions (i.e. using slightly different code paths);
etc.
(It's interesting to note that all those examples are protected by capabilities.)
The key point is that as root, you can exercise more kernel code; if there is a vulnerability in that code, you can trigger it as root, but not as a regular user.
Additionally, if someone finds a way to break out of a container, if you break out as root, you can do much more damage than as a regular user, obviously.
You can reboot host machine by echoing to /proc/sysrq-trigger on docker. Processes running as root in docker can do this.
This seems quite good reason not to run processes as root in docker ;)