event grid validation with AKS - azure

Azure eventgrid subscription verification with AKS/ingress TLS certificates generated by certificate manager
Trying to add a subscription to azure eventgrid custom topic with webhook. Webhook is springboot application deployed on AKS. Exposed using ingress controller. Used certificate manager to generate certificates.
No problem with the application code as i could successfully create a event subscription with the same code running on my local(exposing through public IP).
I am able to access the application from postman. When I POST validation JSON from postman i am getting valid response.
{"validationResponse": "0000000000-0000-0000-0000-00000000000000"}
So the problem should be with certificates or security or ingress.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: sample-ingress
namespace: hello
annotations:
kubernetes.io/ingress.class: nginx
certmanager.k8s.io/cluster-issuer: letsencrypt-staging
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
tls:
- hosts:
- demo-test.eastus.cloudapp.azure.com
secretName: tls-secret
rules:
- host: demo-test.eastus.cloudapp.azure.com
http:
paths:
- backend:
serviceName: api
servicePort: 80
path: /(.*)
- backend:
serviceName: api
servicePort: 80
path: /api/(.*)
- backend:
serviceName: workflow
servicePort: 80
path: /workflow/(.*)
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
namespace: hello
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: user#contoso.com
privateKeySecretRef:
name: letsencrypt-staging
http01: {}
Azure eventgrid subscription validation should be success.
https://learn.microsoft.com/en-us/azure/aks/ingress-tls
followed the above link.

The problem was with self signed certificates. Things worked after giving valid certificates

Related

Kubernetes Azure OAuth2 with multiple hosts in one Nginx Ingress - CORS error

Im looking for the way to configure OAuth2 and Azure provider for Nginx Ingress with multiple hosts definitions.
I need add authentication over for my application dev.example.com on AKS which has internal auth based on Identity service.
I've read examples like this:
https://kristhecodingunicorn.com/post/k8s_nginx_oauth/#setting-up-authentication-with-oauth-20
I've created Application in Azure AD and configure it like described above:
Redirect URIs - https://dev.example.com/oauth2/callback
Front-channel logout - URL https://dev.example.com/oauth2/sign_out
As far I can see almost all works ok - when I open dev.example.com in browser it redirect me to MS Sign In form, then runs 2FA, and opens dev.example.com.
But there is one error during opening web site:
Access to XMLHttpRequest at
'https://api.dev.example.com/identity/.well-known/openid-configuration'
from origin 'https://dev.example.com' has been blocked by CORS policy:
Response to preflight request doesn't pass access control check:
Redirect is not allowed for a preflight request.
the main application tries to connect to internal Identity service and gets an error because of CORS.
Is there a way to fix this?
here is my Nginx ingress for application:
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
name: dev-ingress
annotations:
cert-manager.io/issuer: letsencrypt-cert
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start?rd=$escaped_request_uri
nginx.ingress.kubernetes.io/auth-url: https://$host/oauth2/auth
nginx.ingress.kubernetes.io/ssl-redirect: 'false'
nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
tls:
- hosts:
- dev.example.com
- api.dev.example.com
secretName: letsencrypt-cert
rules:
- host: dev.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: dev-service
port:
number: 80
- host: api.dev.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-service
port:
number: 80
and OAuth2 Configuration:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
k8s-app: oauth2-proxy
name: oauth2-proxy
namespace: dev
spec:
replicas: 1
selector:
matchLabels:
k8s-app: oauth2-proxy
template:
metadata:
labels:
k8s-app: oauth2-proxy
spec:
containers:
- args:
- --provider=oidc
#- --provider=azure
- --azure-tenant=xxxxxxxxxxxxxx
- --skip-jwt-bearer-tokens=true
- --skip-auth-preflight=true
- --email-domain=*
- --http-address=0.0.0.0:4180
- --cookie-domain=.example.com
- --whitelist-domain=.example.com
- --oidc-issuer-url=https://login.microsoftonline.com/xxxxxxxx/v2.0
env:
- name: OAUTH2_PROXY_CLIENT_ID
valueFrom:
secretKeyRef:
name: client-id
key: oauth2_proxy_client_id
- name: OAUTH2_PROXY_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: client-secret
key: oauth2_proxy_client_secret
- name: OAUTH2_PROXY_COOKIE_SECRET
valueFrom:
secretKeyRef:
name: cookie-secret
key: oauth2_proxy_cookie_secret
image: quay.io/oauth2-proxy/oauth2-proxy:latest
imagePullPolicy: Always
name: oauth2-proxy
ports:
- containerPort: 4180
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
labels:
k8s-app: oauth2-proxy
name: oauth2-proxy
namespace: dev
spec:
ports:
- name: http
port: 4180
protocol: TCP
targetPort: 4180
selector:
k8s-app: oauth2-proxy
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: oauth2-proxy
namespace: dev
spec:
ingressClassName: nginx
rules:
- host: dev.example.com
http:
paths:
- path: /oauth2
pathType: Prefix
backend:
service:
name: oauth2-proxy
port:
number: 4180
- host: api.dev.example.com
http:
paths:
- path: /oauth2
pathType: Prefix
backend:
service:
name: oauth2-proxy
port:
number: 4180
You seem to have an SPA and API architecture here. Therefore you need to deal with CORS on the API side of things. As a first step you could implement CORS in your api-service, by plugging in a library.
A more elegant solution is to do so in the ingress, via a CORS plugin. This example is from Kong but maybe there is an equivalent plugin in NGINX. The approach is to build a custom ingress controller image and then add a configuration snippet to inject settings into the nginx.conf file, though it may prove difficult.
Be aware that in oauth2-proxy you are using a website security solution, which is unlikely to be SPA friendly during Ajax calls to APIs, eg when you need to refresh an expired access token. A backend for frontend (BFF) with a more complete design is preferred.

AKS with LetsEncrypt and multiple certs for different containers

I'm looking for any working samples of applying different certificates on AKS with Application Gateway as Ingress Controller.
I have Key Vault with a certificate that is used imported in ApGw/Ingress as sitecomcert and here is Ingress manifest:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: site-agic-ig
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: sitecomcert
appgw.ingress.kubernetes.io/ssl-redirect: "true"
appgw.ingress.kubernetes.io/request-timeout: "180"
appgw.ingress.kubernetes.io/cookie-based-affinity: "true"
spec:
rules:
- host: "site.com"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: site-svc
port:
number: 80
...
Everything works perfect here.
Now I have a second certificate in Key Vault for site2.com and this cert is already imported in Ap Gw as site2comcert and I have container that should serve requests coming to site2.com which point to Ap Gw Public IP.
So I'm about to add
- host: "site2.com" <--- How can I attach **site2comcert** cert?
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: site2-svc
port:
number: 80
but with this setup I receive Untrusted Connection warning in browser because sitecomcert is used. How to configure ApGw / Ingress in a way that allows to use site2comcert for site2.com host specified above?
You can have multiple ingress resource definitions (snipped for brevity):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: site-agic-ig
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: sitecomcert
spec:
rules:
- host: "site.com"
and
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: site-agic-ig-site2
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: site2comcert
spec:
rules:
- host: "site2.com"

SSL Error: Unable to verify the first certificate for ingress kubernates service

I am facing ssl certificate issue in ingress AKS
.yaml file is as below
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: #########
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
tls:
- hosts:
- ##########
secretName: ########
rules:
- host: ##########
http:
paths:
- path: /(.*)
pathType: Prefix
backend:
service:
name: #######
port:
number: 8000
It is working on the web browser but in when try to access from postman or in node js code gives "SSL Error: Unable to verify the first certificate " error
Please help us!
Check the image attachment
On postman go to settings => general
and disabel SSL certificate verification

ingress configuration doesnt connect to service api in azure

I have below configuration with one path for ui and another path for web api but api path doesnt work.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myingress
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/use-private-ip: "true"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
appgw.ingress.kubernetes.io/backend-path-prefix: /
spec:
tls:
- secretName: mysecret
rules:
- host: dev-ingress.com
- http:
paths:
- path: /
backend:
serviceName: uidev
servicePort: 80
- path: /gateway
backend:
serviceName: ocelotapigatewaydev
servicePort: 80
uidev path works but ocelotapigatewaydev dot net web api is not working when i test the api with gateway/api.
Strangely if i have html at the root of the ocelotapigatewaydev web api project then it loads it..Not sure how to fix it.
Changing api calls path to below has fixed the issue
- path: /gateway/.*
courtesy:
https://kubernetes.github.io/ingress-nginx/user-guide/ingress-path-matching/

Invalid host header and default backend 404 with Kubernetes ingress controller

Accessing my nodejs/react site using the URL displays "Invalid Host header". Accessing it through the public IP displays "default backend - 404".
I am using Kubernetes nginx controller with Azure cloud and load balancer.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myrule
namespace: mynamespace
annotations:
kubernetes.io/ingress.class: nginx
certmanager.k8s.io/cluster-issuer: letsencrypt-staging
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
tls:
- hosts:
- mysite.uknorth.cloudapp.azure.com
secretName: tls-secret
rules:
- host: mysite.uknorth.cloudapp.azure.com
http:
paths:
- backend:
serviceName: service-ui
servicePort: 8080
path: /
- backend:
serviceName: service-api
servicePort: 8999
path: /api
Any guidance appreciated.
So let's assume the SSL part is ok (link) since you can reach the nginx ingress controller.
Your rewrite annotation is not necessary for what you need. Take a look at these rules:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myrule
namespace: mynamespace
annotations:
kubernetes.io/ingress.class: nginx
certmanager.k8s.io/cluster-issuer: letsencrypt-staging
spec:
tls:
- hosts:
- mysite.uknorth.cloudapp.azure.com
secretName: tls-secret
rules:
- host: mysite.uknorth.cloudapp.azure.com
http:
paths:
- backend:
serviceName: service-ui
servicePort: 8080
path: /
- backend:
serviceName: service-api
servicePort: 8999
path: /api
Whatever you send to /api/.* will be redirected to service-api. And whatever you send to / will be send to service-ui.
Thanks for your feedback. It turns out the problem was not with the ingress rule above. The service-ui was running the incorrect command parameters thus not acknowledging the request. I missed the fact that the service-api was responding correctly.
In short, check the endpoints and running services are configured correctly - more a lesson for me than anyone else. I received a response by curling the service locally but that didn't mean it could handle https requests over ingress as the service was configured incorrectly.
Also, another lesson for me, ask the developers if the correct image is being used for the build. And ask them again if they say yes.

Resources