I would like to know how Kubernates Ingress works.
Is there an apply order, service and deployment are applied, does ingress come before or after?
To test ingress with azure, I created a kubernates service in azure (service and deployment, etc. applied there). In Azure there is the API-Serveraddress under the Kubernate version or under Network (like xyzzy.123.k8s.io ). Can I enter this domain as a host in my Ingress YAML?
like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test
spec:
tls:
- hosts:
- xyzzy.123.k8s.io <--- Azure
rules:
- host: xyzzy.123.k8s.io
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: test
port:
number: 8080
where can I see in Azure all my applied ingresses?
there is no apply order, kubernetes will create the resources in the order you apply them
no, the address you mention is the API address (kubernetes API), you need to create your own DNS name and connect it to the LB IP of the ingress controller
the resources you create are inside the cluster and you can see them via the kubectl command
Related
I am trying to integrate AGIC with AKS in the following method, but using Terraform: https://learn.microsoft.com/en-us/azure/application-gateway/tutorial-ingress-controller-add-on-existing
I have Terraform scripts to create Azure Application Gateway (AGW) with a basic configuration, then create AKS with the AGIC addon. After that the AGW should configure the corresponding backends, listeners and rules automatically when we create the Ingress resource with correct annotation.
In order to configure SSL, there are 2 approaches it seems:
Method 1:
Add SSL certs as secret in kubernetes cluster, then configure 'tls' section and proper annotatons (to pint to AGIC) in the ingress resource. That will automatically add the certificate in the listener of AGW.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-app-ingress-tls
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
tls:
- secretName: test-tls-secret
hosts:
- test.mydomain.com
rules:
- host: test.mydomain.com
http:
paths:
- path: /
backend:
service:
name: hello-app-service-tls
port:
number: 80
pathType: Exact
Method 2:
Convert SSL certs into pfx format, add it to the AGW directly ( with name test-ssl-certs ). Then mention 2 annotations in the ingress resource:
one - tell to use agic as ingress controller,
two - another annotation mentioning the certificate name that we manually added in AGW.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "test-ssl-certs"
spec:
rules:
- host: "test.mydomain.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: hello-world
port:
number: 80
What is the difference between these 2 methods?
In both the cases, where is the SSL termination happens -inside k8s cluster, or in azure application gateway??
Which one is the better method?
What is the difference between these 2 methods?
This force-ssl-redirect method will route to Ingress-nginx to convert all incoming HTTP request to Https
`ingress.kubernetes.io/force-ssl-redirect: "true"`
This appgw-ssl-certificate Initiate an interaction between Ingress and Applicaiton Gateway with certificate named "test-ssl-certs", which we manually added in Applicaiton Gateway listener.
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "test-ssl-certs"
Details:
SSL certificate reference at Applicaiton gateway
Reference of the same from ingress file
NOTE: If any Certificate name mismatch it will turn to 504 error on Applicaiton Gateway.
In both the cases, where is the SSL termination happens -inside k8s cluster, or in azure application gateway?? Which one is the better method?
In My view appending certificate at ingress level is a best approach, SSL termination happens at load balancer [Application Gateway] level not at K8s. Gateway acts as a reverse proxy, means just routing requests from clients to services
We have defined our internal Load Balancer.
apiVersion: v1
kind: Service
metadata:
name: ads-aks-test
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
type: LoadBalancer
ports:
- protocol: TCP
port: 9000
selector:
app: ads-aks-test
It has its IP and External IP. We want to access this service from VM in another Virtual Network.
We need to know it's DNS name - fully qualified name in advance because we are deploying multiple applications from deployment platform and we want to know based on its Service Name how we can access it once it is being successfully deployed and not to wait for IP address to be determined (either manually or automatically). So for example that is our APP1, and after that automatically we install application APP2 which needs to reach this service.
So for that reason we would like to avoid using the IP information.
How we can determine what is the service "hostname" by which we will access it from the second application?
Only information in docs which I found is: "If your service is using a dynamic or static public IP address, you can use the service annotation service.beta.kubernetes.io/azure-dns-label-name to set a public-facing DNS label." - but this is for public load balancer which we do not want!
Set up ExternalDNS in your K8s cluster. Here is a guide for Azure Private DNS. This will allow you to update the DNS record for any hostname you pick for the service, dynamically via Kubernetes resources.
Sample config looks like this (excerpted from Azure Private DNS guide)
apiVersion: apps/v1
kind: Deployment
metadata:
name: externaldns
spec:
selector:
matchLabels:
app: externaldns
strategy:
type: Recreate
template:
metadata:
labels:
app: externaldns
spec:
containers:
- name: externaldns
image: k8s.gcr.io/external-dns/external-dns:v0.7.3
args:
- --source=service
- --source=ingress
- --domain-filter=example.com
- --provider=azure-private-dns
- --azure-resource-group=externaldns
- --azure-subscription-id=<use the id of your subscription>
volumeMounts:
- name: azure-config-file
mountPath: /etc/kubernetes
readOnly: true
volumes:
- name: azure-config-file
secret:
secretName: azure-config-file
An internal load balancer makes a Kubernetes service accessible only to applications running in the same virtual network as the Kubernetes cluster.
https://learn.microsoft.com/en-us/azure/aks/internal-lb
it seems you want this configuration? is there a peering? you also need to allow communication in NSG .
you can do kubectl get svc
and use the External IP of service ads-aks-test as in annotation you have mentioned "true" so it will be internal IP.
if you are looking forward to resolving the services name in the same cluster you can use the service name itself.
https://kubernetes.io/docs/concepts/services-networking/service/
you can do something like : your-svc.your-namespace.svc.cluster.local
note it will only work when services are in the same Kubernetes cluster.
I have created a AKS and deployed a simple web server on it with following yaml.
Azure LoadBalancer gives a public IP address to it automatically and works fine.
Now I would like to limit the source IP address so I can access it from a specify IP address only.
I've tried adding a Azure Firewall to the virtual network of AKS (aks-vnet-XXXXXXX) with some network rule but doesn't work.
Creating a NAT rule in Firewall and redirects packets to the LoadBalancer works but I can still access the pod with the Public IP address of the LoadBalancer.
Any suggestions?
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
type: LoadBalancer
selector:
app: nginx
ports:
- name: http
port: 80
targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
(skipped something not important)
spec:
containers:
- name: nginx
image: nginx:1.17.6
ports:
- containerPort: 80
What you're trying to achieve can be done with NSG (Network Security Group) applied to the subnet where your AKS cluster resides: https://learn.microsoft.com/en-us/azure/aks/concepts-security#network-security
More generic approach with a fine-grained control will require creation of Ingress Controller, creation of an Ingress object for your service and applying ingress.kubernetes.io/whitelist-source-range annotation to it.
We have created the kubernetes cluster on the azure VM, with Kube master and two nodes. We have deployed application and created the service with "NodePort" which works well. But when we try to use the type: LoadBalancer then it create service but the external IP goes pending state. Currently we unable create service type load balance and due this "ingress" nginx controller also going to same state. So we are not sure how do we setup load balancing in this case.
We have tried creating Load Balancer in Azure and trying to use that ip like shown below in service.
kind: Service
apiVersion: v1
metadata:
name: jira-service
labels:
app: jira-software
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
selector:
app: jira-software
type: LoadBalancer
loadBalancerIP: xxx.xxx.xxx.xxx
ports:
- name: jira-http
port: 8080
targetPort: jira-http
similarly we have one more application running on this kube cluster and we want to access application based on the context path.
if we invoke jira it should call backend server jira http://dns-name/jira
if we invoke some other app like bitbucket http://dns-name/bitbukcet
If I understand correctly you used type LoadBalancer in Virtual Machine, which will not work - type LoadBalancer works only in managed Kubernetes services like GKE, AKS etc.
You can find more information here.
I've followed this doc from microsoft Deploy an HTTPS ingress controller on Azure Kubernetes Service (AKS) and have successfully deployed a managed Kubernetes cluster (AKS) with nginx ingress controller. it works with https as expected.
However, the domain that responds of the format subdomain.eastus2.cloudapp.azure.com. However I would like to use my own custom domain www.somedomain.com. I then add a CNAME entry to my custom domain, pointing to the public ip address configured by the kubernetes cluster.
However, when I do this, I get a response on the browser of
default backend - 404
It looks like I need to change the public ip address in Azure (or somewhere) so that it understands that it will be used by a custom domain as well as by an azure subdomain.
I've had a look at the command:
az network
command. However, it's not very clear is this is the right command to use or not. Does anyone know how I can make the changes required so that my custom FQDN can be routed properly to my kubernetes cluster?
thanks
Here's the yaml that worked for me.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: webapp-ingress
annotations:
kubernetes.io/ingress.class: nginx
certmanager.k8s.io/cluster-issuer: letsencrypt-staging
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- subdomain.eastus2.cloudapp.azure.com
- subdomain.domain.com
secretName: tls-secret
rules:
- host: subdomain.eastus2.cloudapp.azure.com
http:
paths:
- path: /
backend:
serviceName: aks-helloworld
servicePort: 80
- host: subdomain.domain.com
http:
paths:
- path: /
backend:
serviceName: aks-helloworld
servicePort: 80
See here for worked through example: Deploy an HTTPS ingress controller on Azure Kubernetes Service (AKS)
The 'default backend 404' indicates that it is hitting the ingress controller. If it were being rejected or not reaching I'd expect a generic 404 without the 'default backend' bit. This response means it is hitting the ingress contoller but the inress controller doesn't know where to send it on to. This is because there's no ingress route/resource matching the host of that request. The steps to create that route/resource are specific to the domain so the ingress rules only match for the azure domain and not the custom one. I think you'll need to go back and repeat the ingress resource and certificate steps for your custom domain as those steps are domain-specific.
I've been facing the same problem the last couple of days and came across an awesome step-by-step guide which allowed me to use custom domains and provisioning certs with Letsencrypt.
If you want to use your own custom certificates you may want to follow this article instead