How to set information in Kubernetes Network Policy dynamically? - security

Question

I have a few questions regarding Kubernetes: How to secure Kubernetes cluster?
My plan is to develop an application that secures Kubernetes cluster by default. I have written and tested a few Network Policy successfully.
As a second step I want to set these information dynamically in my application based on cloud provider and so one.
1.) I want to block access the host network as well as the meta data services (my cluster runs on AWS):
egress:
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 10.250.0.0/16 # host network
- 169.254.169.254/32 # metadata service
Does anyone know how I can access the host network dynamically?
I found an issue that says that you must use the Meta Data Service: https://github.com/kubernetes/kubernetes/issues/24657
Does anyone know how I can find out on which cloud provider I am currently running?
Based on that information, I want to set the meta data service IP.
2.) I want to block access to the "kube-system" namespace:
egress:
- to:
- podSelector:
matchExpressions:
- key: namespace
operator: NotIn
values:
- kube-system
Does anyone know how I can enforce the actual denied access?
As far as I understood, the key labled "namespace" is just a name that I choosed. How does Kubernetes know that I actually mean the namespace and nothing else?
3.) I want to block Internet access:
spec:
podSelector: {}
policyTypes:
- Egress
Does anyone know, if something like the DNS server in the DMZ zone is still reachable?
4.) I want to block communication to pods with a different namespace:
egress:
- to:
- namespaceSelector:
matchLabels:
project: default
Here, I developed a controller that set the namespace dynamically.

Answer 1

Your ideas are good in terms of a least-privilege policy but the implementation is problematic due to the following reasons.
The logic you are trying to achieve it beyond the capabilities of Kubernetes network policies. It is very difficult to combine multiple block and allow policies in k8s without them conflicting with each other. For example, your first snippet allows access to any IP outside of the cluster and then your 3rd question is about blocking access to the internet - these two policies can't work simultaneously.
You shouldn't block access to the kube-system namespace because that's where the k8s DNS service is deployed and blocking access to it will prevent all communications in the cluster.
To answer your 1st question specifically:
How I can access the host network dynamically?
The cluster subnet is defined when you deploy it on AWS - you should store it during creation and inject it to your policies. Alternatively, you may be able to get it by calling an AWS API.
You can also get the cluster node IPs from Kubernetes: kubectl get nodes -o wide
How I can find out on which cloud provider I am currently running?
Kubernetes doesn't know which platform it is running on, but you can guess it based on the node name prefix, for example: aks-nodepool1-18370774-0 or gke-...
Your 4th point about blocking access between namespaces is good but it would be better to do it with an ingress policy like this:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-all
namespace: default
spec:
podSelector: {}
policyTypes:
- Ingress
For more details, I recommend this blog post that explains the complexities of k8s network policies: https://medium.com/#reuvenharrison/an-introduction-to-kubernetes-network-policies-for-security-people-ba92dd4c809d

Answer 2

As Mark pointed out, NP may not be able to address all your use cases. You might want to check out the Open Policy Agent project and there specifically the Gatekeeper tool, which could be utilized, to at least in part cover some of your needs.