I've followed this doc from microsoft Deploy an HTTPS ingress controller on Azure Kubernetes Service (AKS) and have successfully deployed a managed Kubernetes cluster (AKS) with nginx ingress controller. it works with https as expected.
However, the domain that responds of the format subdomain.eastus2.cloudapp.azure.com. However I would like to use my own custom domain www.somedomain.com. I then add a CNAME entry to my custom domain, pointing to the public ip address configured by the kubernetes cluster.
However, when I do this, I get a response on the browser of
default backend - 404
It looks like I need to change the public ip address in Azure (or somewhere) so that it understands that it will be used by a custom domain as well as by an azure subdomain.
I've had a look at the command:
az network
command. However, it's not very clear is this is the right command to use or not. Does anyone know how I can make the changes required so that my custom FQDN can be routed properly to my kubernetes cluster?
thanks
Here's the yaml that worked for me.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: webapp-ingress
annotations:
kubernetes.io/ingress.class: nginx
certmanager.k8s.io/cluster-issuer: letsencrypt-staging
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- subdomain.eastus2.cloudapp.azure.com
- subdomain.domain.com
secretName: tls-secret
rules:
- host: subdomain.eastus2.cloudapp.azure.com
http:
paths:
- path: /
backend:
serviceName: aks-helloworld
servicePort: 80
- host: subdomain.domain.com
http:
paths:
- path: /
backend:
serviceName: aks-helloworld
servicePort: 80
See here for worked through example: Deploy an HTTPS ingress controller on Azure Kubernetes Service (AKS)
The 'default backend 404' indicates that it is hitting the ingress controller. If it were being rejected or not reaching I'd expect a generic 404 without the 'default backend' bit. This response means it is hitting the ingress contoller but the inress controller doesn't know where to send it on to. This is because there's no ingress route/resource matching the host of that request. The steps to create that route/resource are specific to the domain so the ingress rules only match for the azure domain and not the custom one. I think you'll need to go back and repeat the ingress resource and certificate steps for your custom domain as those steps are domain-specific.
I've been facing the same problem the last couple of days and came across an awesome step-by-step guide which allowed me to use custom domains and provisioning certs with Letsencrypt.
If you want to use your own custom certificates you may want to follow this article instead
Related
I am trying to integrate AGIC with AKS in the following method, but using Terraform: https://learn.microsoft.com/en-us/azure/application-gateway/tutorial-ingress-controller-add-on-existing
I have Terraform scripts to create Azure Application Gateway (AGW) with a basic configuration, then create AKS with the AGIC addon. After that the AGW should configure the corresponding backends, listeners and rules automatically when we create the Ingress resource with correct annotation.
In order to configure SSL, there are 2 approaches it seems:
Method 1:
Add SSL certs as secret in kubernetes cluster, then configure 'tls' section and proper annotatons (to pint to AGIC) in the ingress resource. That will automatically add the certificate in the listener of AGW.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-app-ingress-tls
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
tls:
- secretName: test-tls-secret
hosts:
- test.mydomain.com
rules:
- host: test.mydomain.com
http:
paths:
- path: /
backend:
service:
name: hello-app-service-tls
port:
number: 80
pathType: Exact
Method 2:
Convert SSL certs into pfx format, add it to the AGW directly ( with name test-ssl-certs ). Then mention 2 annotations in the ingress resource:
one - tell to use agic as ingress controller,
two - another annotation mentioning the certificate name that we manually added in AGW.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "test-ssl-certs"
spec:
rules:
- host: "test.mydomain.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: hello-world
port:
number: 80
What is the difference between these 2 methods?
In both the cases, where is the SSL termination happens -inside k8s cluster, or in azure application gateway??
Which one is the better method?
What is the difference between these 2 methods?
This force-ssl-redirect method will route to Ingress-nginx to convert all incoming HTTP request to Https
`ingress.kubernetes.io/force-ssl-redirect: "true"`
This appgw-ssl-certificate Initiate an interaction between Ingress and Applicaiton Gateway with certificate named "test-ssl-certs", which we manually added in Applicaiton Gateway listener.
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "test-ssl-certs"
Details:
SSL certificate reference at Applicaiton gateway
Reference of the same from ingress file
NOTE: If any Certificate name mismatch it will turn to 504 error on Applicaiton Gateway.
In both the cases, where is the SSL termination happens -inside k8s cluster, or in azure application gateway?? Which one is the better method?
In My view appending certificate at ingress level is a best approach, SSL termination happens at load balancer [Application Gateway] level not at K8s. Gateway acts as a reverse proxy, means just routing requests from clients to services
I would like to know how Kubernates Ingress works.
Is there an apply order, service and deployment are applied, does ingress come before or after?
To test ingress with azure, I created a kubernates service in azure (service and deployment, etc. applied there). In Azure there is the API-Serveraddress under the Kubernate version or under Network (like xyzzy.123.k8s.io ). Can I enter this domain as a host in my Ingress YAML?
like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test
spec:
tls:
- hosts:
- xyzzy.123.k8s.io <--- Azure
rules:
- host: xyzzy.123.k8s.io
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: test
port:
number: 8080
where can I see in Azure all my applied ingresses?
there is no apply order, kubernetes will create the resources in the order you apply them
no, the address you mention is the API address (kubernetes API), you need to create your own DNS name and connect it to the LB IP of the ingress controller
the resources you create are inside the cluster and you can see them via the kubectl command
I have deployed multiple microservices on an AKS cluster and exposed it on nginx ingress controller. The ingress pointing to a static ip with dns as blabla.eastus.azure.com
Application is exposed on blabla.eastus.azure.com/application/ and blabla.eastus.azure.com/application2/ .. etc.
I have created a Traffic manager profile in blabla.trafficmanager.net in Azure. How should i configure the AKS ingress in traffic manager such that traffic manager reroutes the request to an application deployed on AKS ingress.
---Ingress.yaml configuration used
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
namespace: ns
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: blabla.eastus.azure.com
http:
paths:
- backend:
serviceName: application1
servicePort: 80
path: /application1(/|$)(.*)
- backend:
serviceName: application2
servicePort: 80
path: /application2(/|$)(.*)
- backend:
serviceName: aks-helloworld
servicePort: 80
path: /(.*)
When i hit curl http://blabla.trafficmanager.net the response is default backend - 404
When i update the host to http://blabla.trafficmanager.net, i am able to access the application through http://blabla.trafficmanager.net\application1
The same is true for any custom cname created. I created a cname as custom.domain.com and redirected it to blabla.eastus.azure.com. So unless i update the host in ingress directly to custom.domain.com I am not able to access it through the custom domain
The actual request will never pass via Traffic Manager. Traffic Manager is a DNS based load balancing solution that is offered by Azure.
When you browse Azure TM endpoint, it resolves and gives you an IP. Then your browser request that IP address.
In your case, your AKS should have a Public Endpoint to which TM can resolve the DNS query. Also you need to create an CNAME record to map TM FQDN to your Custom Domain. If this is not done, you will get 404.
The above mentioned custom header settings are for the probes, but the actual request will be sent from the client browser to the endpoint/IP which the TM resolves to.
One approach to achieve your need is to rigidly control the traffic between public DNS and the Ingress Controller Public IP in each region; delegate
the flexibility of how you publish services to the HTTP SNI protocol:
To keep it simple, the Ingress Controller does not have any A DNS record assigned to its public IP.
So, we'll implement the architecture from right to left following the diagram.
The traffic manager will have two endpoints: one per region. The value of each endpoint will be the corresponded Ingress public IP.
The DNS service will have configured a CNAME (alias) for app.mydomain.com as mine-apps.trafficmanager.net.
In this way, the client connecting to app.mydomain.com will resolve the Traffic Manager (TM) service, which is a Geo DNS, and based on the client's IP, will return to the client the closer target region between A and B.
In the same way, you can use the URL or path-based routing for exposing services via the Ingres and control how clients connect to them. Just make sure that your DNS is aware of how to connect to the Traffic Manager. The rest will be handled
magically by TM and the Ingress object in Kubernetes.
Last but not least, once all the integrations are properly configured and they satisfy your primary need you can start to extend the existing architecture and adapt to your real requirements; for example: getting rid of static IPs in the Traffic Manager's endpoints.
I have a service that I want to expose publicly over http running on AKS. I created a Kubernetes cluster and enabled the HTTP addon following these instructions here. I then launched a service myservicea in namespace mynamespace. And then I created an ingress controller for using the following configuration.
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myservicea-ingress
namespace: mynamespace
annotations:
kubernetes.io/ingress.class: addon-http-application-routing
spec:
rules:
- host: myservicea.XXXXXXX.westus.aksapp.io
http:
paths:
- backend:
serviceName: myservicea
servicePort: 80
path: /
The domain myservicea.XXXXXXX.westus.aksapp.io is reachable after several minutes, however, I only get an HTTP 503 error. So the ingress controller does not seem to be able to find my service. What could be the issue?
Well, if you create the service in a specific namespace. Then maybe you can set the deployment and ingress also in that specific namespace. And you should check in the portal if the host route the request to right namespace. I did the test follow the link you posted with just changing the namespace and it works in my side. The screenshots here:
Hope this will be helpful. If you need more help please give me the message.
I have deployed a Kubernetes cluster to a custom virtual network on Azure using acs-engine. There is an ASP.NET Core 2.0 Kestrel app running on the agent VMs and the app is accessed over VPN through a Service of the Azure internal load balancer type. Now I would like to enable HTTPS on the service. I have already obtained a domain name and a certificate but have no idea how to proceed. Apparently configuring Kestrel to use HTTPS and copying the certificate to each container is not the way to go.
I have checked out tutorials such as ingress on k8s using acs and configure Nginx Ingress Controller for TLS termination on k8s on Azure but both of them end up exposing a public external IP and I want to keep the IP internal and not accessible from the internet. Is this possible? Can it be done without ingresses and their controllers?
While for some reason I still can't access the app through the ingress I was able to create an internal ingress service with the IP I want with the following configuration:
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
name: nginx-ingress-svc
spec:
type: LoadBalancer
ports:
- port: 443
targetPort: 443
loadBalancerIP: 130.10.1.9
selector:
k8s-app: nginx-ingress-controller
The tutorial you linked is a bit outdated, at least the instructions have you go to a 'examples' folder in the GitHub repo they link but that doesn't exist. Anyhow, a normal nginx ingress controller consists of several parts: the nginx deployment, the service that exposes it and the default backed parts. You need to look at the yamls they ask you to deploy, look for the second part of what I listed - the ingress service - and change type from LoadBalancer to ClusterIP (or delete type altogether since ClusterIP is the default)