I have updated the SSL certificates on my AKS service. But I am getting CORS error.
New certificates are getting reflected though.
Can someone provide a solution for it.
Here is the TLS part of my ingress file.
spec:
tls:
- hosts:
- "prodazure.thesolarlabs.com"
secretName: tls-secret
Related
I am trying to integrate AGIC with AKS in the following method, but using Terraform: https://learn.microsoft.com/en-us/azure/application-gateway/tutorial-ingress-controller-add-on-existing
I have Terraform scripts to create Azure Application Gateway (AGW) with a basic configuration, then create AKS with the AGIC addon. After that the AGW should configure the corresponding backends, listeners and rules automatically when we create the Ingress resource with correct annotation.
In order to configure SSL, there are 2 approaches it seems:
Method 1:
Add SSL certs as secret in kubernetes cluster, then configure 'tls' section and proper annotatons (to pint to AGIC) in the ingress resource. That will automatically add the certificate in the listener of AGW.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-app-ingress-tls
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
tls:
- secretName: test-tls-secret
hosts:
- test.mydomain.com
rules:
- host: test.mydomain.com
http:
paths:
- path: /
backend:
service:
name: hello-app-service-tls
port:
number: 80
pathType: Exact
Method 2:
Convert SSL certs into pfx format, add it to the AGW directly ( with name test-ssl-certs ). Then mention 2 annotations in the ingress resource:
one - tell to use agic as ingress controller,
two - another annotation mentioning the certificate name that we manually added in AGW.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "test-ssl-certs"
spec:
rules:
- host: "test.mydomain.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: hello-world
port:
number: 80
What is the difference between these 2 methods?
In both the cases, where is the SSL termination happens -inside k8s cluster, or in azure application gateway??
Which one is the better method?
What is the difference between these 2 methods?
This force-ssl-redirect method will route to Ingress-nginx to convert all incoming HTTP request to Https
`ingress.kubernetes.io/force-ssl-redirect: "true"`
This appgw-ssl-certificate Initiate an interaction between Ingress and Applicaiton Gateway with certificate named "test-ssl-certs", which we manually added in Applicaiton Gateway listener.
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "test-ssl-certs"
Details:
SSL certificate reference at Applicaiton gateway
Reference of the same from ingress file
NOTE: If any Certificate name mismatch it will turn to 504 error on Applicaiton Gateway.
In both the cases, where is the SSL termination happens -inside k8s cluster, or in azure application gateway?? Which one is the better method?
In My view appending certificate at ingress level is a best approach, SSL termination happens at load balancer [Application Gateway] level not at K8s. Gateway acts as a reverse proxy, means just routing requests from clients to services
We are running a AKS Kubernetes cluster on Azure. I'm using the "NGINX Ingress Controller" and "cert-manager" for routing and certificate generation (through Let's Encrypt). I followed the basic setup advice from the Microsoft documentation: https://learn.microsoft.com/en-us/azure/aks/ingress-tls
When visiting our page in a web-browser, we notice nothing out of the ordinary at first - HTTPS was working fine. The browser can validate the Let's Encrypt certificate. However, we noticed later on that the ingress controller actually serves two certificates (one of which has a common name: "Kubernetes Ingress Controller Fake Certificate" and an Alternative name: "ingress.local"): https://www.ssllabs.com/ssltest/analyze.html?d=test-aks-ingress.switzerlandnorth.cloudapp.azure.com&hideResults=on
Long story short - yesterday, I tried everything from re-installing the Nginx-ingress and cert-manager to starting a new Azure Kubernetes Service from scratch, but every time I end up in the same situation.
I have read many of the discussions from people experiencing similar problems. Typically, they are a bit different though, as they don't actually see a valid certificate at all. I confirmed that we are using the production Let's Encrypt ClusterIssuer:
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: ***#***
privateKeySecretRef:
name: letsencrypt
solvers:
- http01:
ingress:
class: nginx
podTemplate:
spec:
nodeSelector:
"kubernetes.io/os": linux
I also created a new test-app with test-ingress:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-world-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/use-regex: "true"
cert-manager.io/cluster-issuer: letsencrypt
spec:
tls:
- hosts:
- test-aks-ingress.switzerlandnorth.cloudapp.azure.com
secretName: tls-secret
rules:
- host: test-aks-ingress.switzerlandnorth.cloudapp.azure.com
http:
paths:
- backend:
serviceName: aks-helloworld-one
servicePort: 80
path: /hello-world-one(/|$)(.*)
From my understanding there is usually some issue with the secret for the people that have reported this previously. Here I assume that the ClusterIssuer will generate the relevant certificates and store them in tls-secret, which has been generated automatically:
Name: tls-secret
Namespace: test
Labels: <none>
Annotations: cert-manager.io/alt-names: test-aks-ingress.switzerlandnorth.cloudapp.azure.com
cert-manager.io/certificate-name: tls-secret
cert-manager.io/common-name: test-aks-ingress.switzerlandnorth.cloudapp.azure.com
cert-manager.io/ip-sans:
cert-manager.io/issuer-group: cert-manager.io
cert-manager.io/issuer-kind: ClusterIssuer
cert-manager.io/issuer-name: letsencrypt
cert-manager.io/uri-sans:
Type: kubernetes.io/tls
Data
====
tls.crt: 3530 bytes
tls.key: 1675 bytes
Maybe what I am still confused about is the different secrets / certificates at play here. The cert-manager operates in the cert-manager namespace and creates a letsencrypt secret there, while my test-setup is running everything else in a test namespace (including the ingress controller).
[UPDATE]
But what is the actual problem here? Everything "just works" in a normal browser, right? Unfortunately, the real problem is that connections do not work for a specific client application, which may not have SNI support.
Is there a way to not have a default certificate? How would I change the configuration here to provide the "Let's Encrypt" signed certificate by default - is that possible?
It is expected behavior. By default Ingress controller creates self-signed certificate with CN indicating it's fake one. This is used when a request doesn't match to rules defined in Ingress. So when we access this URL from browser, it returns correct certificate but with openssl s_client without servername field, it doesn't match the rule defined in Ingress and goes to default backend and returns self-signed certificate.
You can also specify default certificate for Ingress. Refer https://github.com/kubernetes/ingress-nginx/issues/4674 for more details.
I am using the Azure Application Gateway Ingress Controller for Kubernetes. I was able to successfully configure the controller to expose my Kubernetes Services over http. However, I would like the Application Gateway to do SSL offload. So that the Application Gateway will handle https requests, and then forward a plain http request to my Kubernetes service. Currently the Ingress Controller documentation for https requires you to specify the certificate for your Kubernetes Service.
Is it possible to configure the Ingress Controller to do SSL offload so that I don't have to configure https on my Kubernetes Services?
I would assume this is the document you are looking for.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: guestbook
annotations:
kubernetes.io/ingress.class: azure/application-gateway
spec:
tls:
- secretName: <guestbook-secret-name>
rules:
- http:
paths:
- backend:
serviceName: frontend
servicePort: 80
ps. no idea why you would use application gateway with k8s. its garbage.
Adding appgw.ingress.kubernetes.io/ssl-redirect: "true" annotation with properly configured TLS certificate on ingress enables us to configure Application Gateway to automatically redirect HTTP URLs to their HTTPS counterparts. i.e. it creates https & http listener on Application Gateway
I've followed this doc from microsoft Deploy an HTTPS ingress controller on Azure Kubernetes Service (AKS) and have successfully deployed a managed Kubernetes cluster (AKS) with nginx ingress controller. it works with https as expected.
However, the domain that responds of the format subdomain.eastus2.cloudapp.azure.com. However I would like to use my own custom domain www.somedomain.com. I then add a CNAME entry to my custom domain, pointing to the public ip address configured by the kubernetes cluster.
However, when I do this, I get a response on the browser of
default backend - 404
It looks like I need to change the public ip address in Azure (or somewhere) so that it understands that it will be used by a custom domain as well as by an azure subdomain.
I've had a look at the command:
az network
command. However, it's not very clear is this is the right command to use or not. Does anyone know how I can make the changes required so that my custom FQDN can be routed properly to my kubernetes cluster?
thanks
Here's the yaml that worked for me.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: webapp-ingress
annotations:
kubernetes.io/ingress.class: nginx
certmanager.k8s.io/cluster-issuer: letsencrypt-staging
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- subdomain.eastus2.cloudapp.azure.com
- subdomain.domain.com
secretName: tls-secret
rules:
- host: subdomain.eastus2.cloudapp.azure.com
http:
paths:
- path: /
backend:
serviceName: aks-helloworld
servicePort: 80
- host: subdomain.domain.com
http:
paths:
- path: /
backend:
serviceName: aks-helloworld
servicePort: 80
See here for worked through example: Deploy an HTTPS ingress controller on Azure Kubernetes Service (AKS)
The 'default backend 404' indicates that it is hitting the ingress controller. If it were being rejected or not reaching I'd expect a generic 404 without the 'default backend' bit. This response means it is hitting the ingress contoller but the inress controller doesn't know where to send it on to. This is because there's no ingress route/resource matching the host of that request. The steps to create that route/resource are specific to the domain so the ingress rules only match for the azure domain and not the custom one. I think you'll need to go back and repeat the ingress resource and certificate steps for your custom domain as those steps are domain-specific.
I've been facing the same problem the last couple of days and came across an awesome step-by-step guide which allowed me to use custom domains and provisioning certs with Letsencrypt.
If you want to use your own custom certificates you may want to follow this article instead
I have a simple meteor app deployed on kubernetes. I associated an external IP address with the server, so that it's accessible from within the cluster. Now, I am up to exposing it to the internet and securing it (using HTTPS protocol). Can anyone give simple instructions for this section?
In my opinion kube-lego is the best solution for GKE. See why:
Uses Let's Encrypt as a CA
Fully automated enrollment and renewals
Minimal configuration in a single ConfigMap object
Works with nginx-ingress-controller (see example)
Works with GKE's HTTP Load Balancer (see example)
Multiple domains fully supported, including virtual hosting multiple https sites on one IP (with nginx-ingress-controller's SNI support)
Example configuration (that's it!):
kind: ConfigMap
apiVersion: v1
metadata:
name: kube-lego
namespace: kube-lego
data:
lego.email: "your#email"
lego.url: "https://acme-v01.api.letsencrypt.org/directory"
Example Ingress (you can create more of these):
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: site1
annotations:
# remove next line if not using nginx-ingress-controller
kubernetes.io/ingress.class: "nginx"
# next line enable kube-lego for this Ingress
kubernetes.io/tls-acme: "true"
spec:
tls:
- hosts:
- site1.com
- www.site1.com
- site2.com
- www.site2.com
secretName: site12-tls
rules:
...
There are several ways to setup a ssl endpoint, but your solution needs to solve 2 issues: First, you need to get a valid cert and key. Second, you would need to setup a ssl endpoint in your infrastructure.
Have a look at k8s ingress controller. You can provide an ingress controller with a certificate/key secret from the k8s secret store to setup a ssl endpoint. Of course, this requires you to already have a valid certificate and key.
You could have a look at k8s specific solutions for issuing and using certificates like the Kubernetes Letsencrypt Controller, but I have never used them and cannot say how well they work.
Here are some general ideas to issue and use ssl certificates:
1. Getting a valid ssl certificate and key
AWS
If you are running on AWS, the easiest way I can think of is by setting up an ELB, which can issue the ssl cert automatically for you.
LetsEncrypt
You could also have a look at LetsEncrypt to issue free certificates for your domain. Nice thing about it is that you can automate your cert issuing process.
CA
Of course, you could always go the old-fashion way and issue a certificate from a provider that you trust.
2. Setting up the ssl endpoint
AWS
Again, if you have an ELB then it already acts as an endpoint and you are done. Of course your client <-> ELB connection is encrypted, but ELB <-> k8s-cluster is unencrypted.
k8s ingress controller
As mentioned above, depending on the k8s version you use you could also setup a TLS ingress controller.
k8s proxy service
Another option is to setup a service inside your k8s cluster, which terminates the ssl connection and proxies the traffic to your meteor application unencrypted.
You could use nginx as a proxy for this. In this case I suggest you store your certificate's key inside k8s secret store and mount it inside the nginx container. NEVER ship a container which has secrets such as certificate keys stored inside! Of course you still somehow need to send your encrypted traffic to a k8s node - again, there several ways to achieve this... Easiest would be to modify your DNS entry to point to the k8s nodes, but ideally you would use a TCP LB.