I want to set up a kubernetes cluster with a loadbalancer. Kubernetes will create a load balancer in azure and connect a public ip address with it.
But I don't want to make the api public, it should be exclusive for my api management service.
I tried to direct the load balancer into a vnet with the api service but I found nothing.
So I thought I could just limit the access to the public ip (a whitelist with only the incluced ip of my service) but I found nothing on the internet.
Is it possible to set such rule on a public ip or do I need some extra service for this problem?
With Kubernetes, assuming you have a service defined
Use the following commands:
kubectl get service
kubectl edit svc/<YOUR SERVICE>
change the type from LoadBalancer to ClusterIP
Now you can consume internally the service.
Related
I have a ingress controller loadbalancer for kubernetes cluster which has an external IP
I have another app service which has network restrictions. I should be able to acess this App Service with the external IP of k8s, i have from above load balancer by whitelisting it.
I have whitelisted but it is not working and it is giving 403 forbidden as response.
Is this is the IP we should use for whitelisting or do we have to do any other process. I have used this load balancer IP for subdomain mapping as well and it is working fine but unable to access App service when i whitelisted it but I am able to access App service when i have whitelisted a other vm IP address.
If you are using the Azure CNI, Pod IPs are first-class members in the virtual network. You should add an allow rule for the entire Kubernetes Pod IP range to make this work.
If you want more control over the outgoing IP address you can set that up with an egress gateway in your service mesh of choice. For example Istio.
I am using multiple pods and their services, some of the services are of the type load balancer, which will expose the public IP.
But many of the services are called internally and no need to use public IP instead I can use private IP, what change do I need to make to the load balancer to use private IP.
I assume, the load balancer cost more compared to other types of services in the aks cluster.
Please let me know how to reduce the cost
Just do not annotate the services with type: LoadBalancer but instead use type: ClusterIP
Declare type:ClusterIP instead of type:LoadBalancer under kind:Service
It will generate Private IP for the service, which can be accessed with either IP or name of service.
http://<servicename>.<namespace>.svc.cluster.local:<port number>
You can annotate the service so that the Loadbalancer gets an private IP from your subnet:
service.beta.kubernetes.io/azure-load-balancer-internal: true
You can also check the docs here.
One hint: you should only expose the service from your Ingress Controller and not Services directly, This is an Kubernetes anti-pattern and insecure.
I have a separate VM in the same network as my kubernetes in Azure.
I have a kafka pod and I am able to reach this pod using the IP. The problem is that the pod IP is changing all the time.
Is there any way to get the correct IP each time the pod IP is changing?
I would suggest using a kubernetes service to expose pod. This avoids the problem with change in POD IP because service IP does not change.
Kubernetes ServiceTypes allow you to specify what kind of Service you want. The default is ClusterIP.
Type values and their behaviors are:
ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType.
NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You'll be able to contact the NodePort Service, from outside the cluster, by requesting :.
LoadBalancer: Exposes the Service externally using a cloud provider's load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created.
ExternalName: Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record
Since you are accessing the POD from outside the kubernetes cluster itself use NodePort or LoadBalancer type service.
As mentioned by #arghya-sadhu already going for Kubernetes service, is the best option. The kubernetes service has an IP depending on the type of kubernetes service.
For services of type ClusterIP, you get a cluster IP address
For services of type Load Balancer, you get a Loadbalancer IP address (i.e) public IP address
For services of type NodePort, you can access using the node's address.
But, whatever the type of service is, you can access the service using the kube-DNS within the cluster. So, let's say your service name is other-service and it exposes port 8080, running on namespace abc, then you can access the service as follows:
http://other-service.abc:8080
Since, your VM runs outside the cluster, it is better to use Loadbalancer and access the pod using Loadbalancer url or IP address. You can set an ingress in case there are multiple pods in the cluster that you want to connect to.
I am little but puzzled by Azure Network Analytics! Can someone help resolving this mystery?
My Kubernetes cluster in Azure is private. It's joined to a vNET and there is no public ip exposed anywhere. Service is configured with internal load balancer. Application gateway calls the internal load balancer. NSG blocks all inbound traffics from internet to app gateway. Only trusted NAT ips are allowed at the NSG.
Question is- I am seeing lot of internet traffic coming to aks on the vNET. They are denied of course! I don't have this public ip 40.117.133.149 anywhere in the subscription. So, how are these requests coming to aks?
You can try calling app gateway from internet and you would not get any response! http://23.100.30.223/api/customer/FindAllCountryProvinceCities?country=United%20States&state=Washington
You would get successful response if you call the Azure Function- https://afa-aspnet4you.azurewebsites.net/api/aks/api/customer/FindAllCountryProvinceCities?country=United%20States&state=Washington
Its possible because of following nsg rules!
Thank you for taking time to answer my query.
In response to #CharlesXu, I am sharing little more on the aks networking. Aks network is made of few address spaces-
Also, there is no public ip assigned to any of the two nodes in the cluster. Only private ip is assigned to vm node. Here is an example of node-0-
I don't understand why I am seeing inbound requests to 40.117.133.149 within my cluster!
After searching all the settings and activity logs, I finally found answer to the mystery IP! A load balancer with external ip was auto created as part of nginx ingress service when I restarted the VMs. NSG was updated automatically to allow internet traffic to port 80/443. I manually deleted the public load balancer along with IP but the bad actors were still calling the IP with a different ports which are denied by default inbound nsg rule.
To reproduce, I removed the public load balancer again along with public ip. Azure aks recreated once I restarted the VMs in the cluster! It's like cat and mouse game!
I think we can update the ingress service annotation to specify service.beta.kubernetes.io/azure-load-balancer-internal: "true". Don't know why Microsoft decided to auto provision public load balancer in the cluster. It's a risk and Microsoft should correct the behavior by creating internal load balancer.
I am currently using Azure AKS.
I have a frontend application which is using LoadBalancer to have a public IP to access the service.
Should I just direct my domain name to the public IP address?
Because the IP is dynamic, if the port is destroy and recreate again, a new IP is generated.
Should I use Ingress/Nginx controller to manage the IP?
You can use A record points to the external IP address.
You can change the public IP address to static via Azure portal, in this way, restart the service will not change the IP.
But in Azure, if we delete the Azure AKS, the Public IP address will collected by Azure platform, and we will lose this IP address.
You can use kubernetes-incubator/external-dns to automatically update the A record in your Azure DNS zone with the (dynamic) IP of the Azure Loadbalancer or Ingress controller. Read here how to set up.
You're not limited to use Azure DNS, you could use other providers to, in v0.4: Google CloudDNS, AWS Route 53, AzureDNS, CloudFlare, DigitalOcean,
DNSimple, Infoblox
Should I just direct my domain name to the public IP address?
As Mohit said, we can set static public IP via Azure portal, and map your domian name to that Public IP address.
Because the IP is dynamic, if the port is destroy and recreate again,
a new IP is generated.
By default, AKS expose pods to internet will create a Kubernetes service, the Public IP address work for that service.
If one pod was not work(multiple pods), AKS will create another pod in your service and that will not get a new public IP. But if you only have one pod in that service and re-create that pod, we will get a new Public IP address.
For now, Azure does not support to keep the public IP address for AKS service.
Hope this helps.