NGINX Ingress external oauth with Azure Active Directory - azure

Question

I want to use Azure Active Directory as an external oauth2 provider to protect my services on the ingress level. In the past, I used basic ouath and everything worked like expected. But nginx provides the extern ouath methode which sounds much more confortable!
For that I created an SP:
$ az ad sp create-for-rbac --skip-assignment --name test -o table
AppId DisplayName Name Password Tenant
<AZURE_CLIENT_ID> test http://test <AZURE_CLIENT_SECRET> <TENANT_ID>
My ingress ressource:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress
namespace: ingress-nginx
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/auth-url: "https://\$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://\$host/oauth2/start?rd=$escaped_request_uri"
# nginx.ingress.kubernetes.io/auth-type: basic
# nginx.ingress.kubernetes.io/auth-secret: basic-auth
# nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'
And the externel-oauth:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: oauth2-proxy
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: oauth2-proxy
template:
metadata:
labels:
app: oauth2-proxy
spec:
containers:
- args:
- --provider=azure
- --email-domain=microsoft.com
- --upstream=file:///dev/null
- --http-address=0.0.0.0:4180
- --azure-tenant=$AZURE_TENANT_ID
env:
- name: OAUTH2_PROXY_CLIENT_ID
value: $API_CLIENT_ID
- name: OAUTH2_PROXY_CLIENT_SECRET
value: $API_CLIENT_SECRET
- name: OAUTH2_PROXY_COOKIE_SECRET
value: $API_COOKIE_SECRET
# created by docker run -ti --rm python:3-alpine python -c 'import secrets,base64; print(base64.b64encode(base64.b64encode(secrets.token_bytes(16))));
image: docker.io/colemickens/oauth2_proxy:latest
imagePullPolicy: Always
name: oauth2-proxy
ports:
- containerPort: 4180
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: oauth2-proxy
namespace: kube-system
spec:
ports:
- name: http
port: 4180
protocol: TCP
targetPort: 4180
selector:
app: oauth2-proxy
It looks like something wents wrong but I have no idea what I missed.
When i try to enter the page it loads up to one minute and ends in a '500 internal server error'.
The logs of the ingress controller show an infinity loop of following:
10.244.2.1 - - [16/Jan/2020:15:32:30 +0000] "GET /oauth2/auth HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/xxxxx Safari/xxxx" 727 0.003 [upstream-default-backend] [] - - - - <AZURE_CLIENT_ID>

Answer 1

So you need another ingress for the oAuth deployment as well. here's how my setup looks like:
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: grafana-ingress-oauth
namespace: grafana
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: xxx
http:
paths:
- path: /oauth2
backend:
serviceName: oauth2-proxy
servicePort: 4180
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: grafana-ingress
namespace: grafana
annotations:
kubernetes.io/ingress.class: "nginx"
kubernetes.io/tls-acme: "true"
certmanager.k8s.io/cluster-issuer: letsencrypt-production
ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
spec:
rules:
- host: xxx
http:
paths:
- path: /
backend:
serviceName: grafana
servicePort: 80
this way second ingress redirects to first and first does the auth and redirects back