Explaining my confusion / lack of understanding
When reading about the external LoadBalancer in K8s, which is a cloud provider only feature, I don't quite understand when it should be used, as when one creates a Deployment K8s will do Round Robin load balancing on the pods in that Deployment.
So from my current understanding all one would need to do is make a NodeIP, and you have the equivalent of an external load balancer?
Or should I think of the LoadBalancer type as haproxy/nginx/Envoy, where one can do SSL, reverse proxy, and many other useful things?
My current guess is that the proper use of LoadBalancer is to add many NodeIP's, but I can't find anything to back that up.
Question
Can anyone explain when and why to use LoadBalancer and not just using the NodeIP?
For example, You want to deploy multiple applications in your cluster, say 10 apps.
You would like to access these 10 apps over internet. One way is to set those 10 application services as nodeport so you can access them from outside. For this to happen kubernetes opens 10 nodeports on each cluster node. This is a security risk.
In most of the enterprises where they work behind firewall in a closed network dont allow external traffic to/from any ports other than http/https ( 80/443 ).
One way is to set service type as Loadbalancer for each application service. So, to access 10 app, you will be provisioning 10 load balancers to access the app servers over http/https ports. Since loadbalancers are charged resources, economically it is not viable to have one load balancer for each service that you want to access over itnernet.
Is there a way to access all those 10 app services running inside kubernetes over single port. This is where ingress controller comes into picture.
Ingress controller allows single ip-port to access all services running in k8s through ingress rules. The ingress controller service is set to load balancer so it is accessible from public internet
Related
I need to host the frontend and backend parts of my application on ingress kubernetes. I would like only the frontend part to be sent to the backend part, even though both are available in ingress under one host (but a different path). Is it possible to set something like this in a kubernetes cluster? So that no other applications can send requests to the backend part. Can you do something like this with kubernetes security headers?
Within the cluster, you can restrict traffic between services by using Network Policies. E.g. you can declare that service A can send traffic to service B, but that service C can not send traffic to service B. However, you need to make sure that your cluster has a CNI with support for Network Policies. Calico is an example for such add-on.
Ingress is useful for declaring what services can receive traffic from outside of the cluster.
Also, Service Meshes, like Istio is useful for further enhance this security. E.g. by using an Egress proxy, mTLS and require JWT based authentication between services.
My understanding is that setting the Service type to LoadBalancer creates a new Azure Load Balancer and assigns an IP address to the Service. Does this mean that I can have multiple Services using port 80? If the app behind my Service (an ASP.NET Core app) can handle TLS and HTTPS why shouldn't I just use LoadBalancer's for any Service I want to expose to the internet?
What is the advantage of using an Ingress if I don't care about TLS termination (You can let Cloudflare handle TLS termination)? If anything, it slows things down by adding an extra hop for every request.
Update
Some answers below mention that creating load balancers is costly. It should be noted that load balancers on Azure are free but they do charge for IP addresses of which they give you five for free. So for small projects where you want to expose up to five IP addresses, it's essentially free. Any more than that, then you may want to look ad usign Ingress.
Some answers also mention extra complexity if you don't use Ingress. I have already mentioned that Cloudflare can handle TLS termination for me. I've also discovered the external-dns Kubernetes project to create DNS entries in Cloudflare pointing at the load balancers IP address? It seems to me that cutting out Ingress reduces complexity as it's one less thing that I have to configure and manage. The choice of Ingress is also massive, it's likely that I'll pick the wrong one which will end up unmaintained after some time.
There is a nice article here which describe the differences on Service(Load Balancer) and Ingress.
In summary, you can have multiple Service(Load Balancer) in the cluster, where each application is exposed independently from each other. The main issue is that each Load Balancer added will increase the cost of your solution, and does not have to be this way, unless you strictly need this.
If multiple applications listen on port 80, and they are inside the container, there is no reason you also need to map it to the port 80 in the host node. You can assign it to any port, because the Service will handle the dynamic port mappings for you.
The ingress is best in this scenario, because you can have one ingress listing on port 80, and route the traffic to the right service based on many variables, like:
Domain
Url Path
Query String
And many other
Ingress in not just for TLS termination, it is in simple terms a proxy\gateway that will control the routing to the right service, TLS termination is just one of the features.
No, you cant have multiple services listening on port 80, as load balancer wont know where to route them (ingress will, however). If you can affort to host each service on different port you could use load balancer. alternatively, if you have public ip for each service and different backend port on each service you can achieve this.
quote: The protocol and port combination you entered matches another rule used by this load balancer. The protocol and port combination of each load balancing rule and inbound NAT rule on a load balancer must be unique.
again, if you are a developer, you probably do not realize how much more convenient it is to manage certificate on ingress, and not on all individual containers that are supposed to be accessible
I want to deploy some kubernetes workloads, which are visible from some other VM's on Azure but not visible from the outside world.
For example: I might have a VM running a Zuul Gateway which for some routes I want to redirect to the K8s cluster, yet I don't want to allow people to directly access my K8s cluster.
Is it possible to place my AKS inside a VPN? If so, how should I achieve this?
In addition to options, pointed out by #4c74356b41, you can run ingress controller on the cluster, and limit it to your internal server IP only
So this isnt possible now (at least out of the box) due to the nature of AKS being a service with no VNet integration as of yet. You can try to hack around this, but it will probably not work really well as your agents need to talk to the master.
I see 2 options:
Use internal load balancers instead of public ones to expose your services
Use ACS which has vnet integration, but I'm not sure if you can apply 2 routes to the same vnet
I want to expose multiple services trough single load balancer. Each service points to exactly one pod.
So far I tried to:
kubectl expose <podName> --port=7000
And in Azure portal to manually set either load balancing rules or Inbound Nat rules, pointing to exposed pod.
So far I can connect to pod using external IP and specified port.
Depends on how you want to separate services on the same IP. The two ways that come to my mind are :
use NodePort services and then map some ports from your LB to that part on your cluster nodes. This gives separation by port.
way more interesting in my opinion is to use Ingress/IngressController. You would expose only IC on standard ports like 80 & 443 and then it will map to your services by hostname and uri
In Azure container service, Azure will use Load Balancer to expose k8s services, like this:
root#k8s-master-E27AE453-0:~# kubectl get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
jasonnginx 10.0.41.194 52.226.33.200 8080:32011/TCP 4m
kubernetes 10.0.0.1 <none> 443/TCP 11m
mynginx 10.0.144.49 40.71.230.60 80:32366/TCP 5m
yournginx 10.0.147.28 40.71.226.23 80:32289/TCP 4m
root#k8s-master-E27AE453-0:~#
Via Azure portal, check Azure load balancer frontend IP configuration(different IP address):
ACS will create Load Balancer rules and add rontend IP address automatically.
How to expose multiple kubernetes services trough single azure load
balancer?
ACS expose k8s services through that Azure Load Balancer, do you mean you want to expose k8s services with a single Public IP address?
If you want to expose k8s services with a single public IP address, as Radek said, maybe you should use Nginx Ingress Controller.
The Ingress Controller works like this:
Thanks guys. I think I have found viable solution to my problem. I should have been more specific about what I'm going to do.
I want to host game server over UDP. So any kubernetes ingress controller is not really an option, since they rarely support UDP routing.
I also don't need to host multitude of services on single machine 1-4 of pods per single node is probably the maximum.
I have found about using :
hostNetwork: true
in yaml config and it actually works pretty well for this scenario.
I get IP directly from host node. I can then select matching node within load balancer and create NAT or load balancing rule
Create multiple nodePort type service, and fix the nodePort.
And the cloud load balancer, set multiple listener groups. The listen port is same as the service's nodeport, and the target are all the worker nodes.
How to add multiple ingress or Load balancers in kubernetes for separate services,
here is the post who I ended up creating a ingress to my sub-domain. Is there any way we can specific the same IP address created by GCE to launch multiple Ingress resources.
I am using GCE for hosting my cluster. If there is a better way to handle this scenario to have multiple resources to expose a service with a sub-domain www.app1.domain.com, www.app2.domain.com which are entirely different apps and have two ingress resources that point to two these specific services using same external IP address.
From the post I could able to create but unable to specify the external IP address to it.
Any help is much appreciated, thank you.
You can just define multiple Ingress resources and put them to Kubernetes - they don't have to be in the same yaml file. All ingress resources share the same proxy and they are routed via the defined hostname and path to the wanted service.
I am not sure what you mean with the external IP address.