How can I debug oauth2_proxy when connecting to Azure B2C? - azure

Question

I'm new to Kubernetes, and I've been learning about Ingress. I'm quite impressed by the idea of handling TLS certificates and authentication at the point of Ingress. I've added a simple static file server, and added cert-manager, so I basically have a HTTPS static website.
I read that NGINX Ingress Controller can be used with oauth2 proxy to handle authentication at the ingress. The problem is that I can't get this working at all. I can confirm that my oauth2-proxy Deployment Service and Deployment are present and correct - in the Pod's log, I can see the requests coming through from NGINX, but I can't see what uri it is actually calling at Azure B2C. Whenever I try and access my service I get a 500 Internal error - if I put my /oath2/auth address in the browser, I get "The scope 'openid' specified in the request is not supported.". However if I Test run the user Flow in Azure, the test URL also specifies "openid" and it functions as expected.
I think that I could work through this if I could find out how to monitor what oauth2-proxy requests from Azure (i.e. work out where my config is wrong by observing it's uri) - otherwise, maybe somebody that has done this can tell me where I went wrong in the config.
My config is as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
k8s-app: oauth2-proxy
name: oauth2-proxy
namespace: default
spec:
replicas: 1
selector:
matchLabels:
k8s-app: oauth2-proxy
template:
metadata:
labels:
k8s-app: oauth2-proxy
spec:
containers:
- args:
- -provider=oidc
- -email-domain=*
- -upstream=file:///dev/null
- -http-address=0.0.0.0:4180
- -redirect-url=https://jwt.ms/
- -oidc-issuer-url=https://<tenant>.b2clogin.com/tfp/<app-guid>/b2c_1_manager_signup/
- -cookie-secure=true
- -scope="openid"
# Register a new application
# https://github.com/settings/applications/new
env:
- name: OAUTH2_PROXY_CLIENT_ID
value: <app-guid>
- name: OAUTH2_PROXY_CLIENT_SECRET
value: <key-base64>
- name: OAUTH2_PROXY_COOKIE_SECRET
value: <random+base64>
image: quay.io/pusher/oauth2_proxy:latest
imagePullPolicy: Always
name: oauth2-proxy
ports:
- containerPort: 4180
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
labels:
k8s-app: oauth2-proxy
name: oauth2-proxy
namespace: default
spec:
ports:
- name: http
port: 4180
protocol: TCP
targetPort: 4180
selector:
k8s-app: oauth2-proxy
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: static1-oauth2-proxy
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
spec:
rules:
- host: cloud.<mydomain>
http:
paths:
- backend:
serviceName: oauth2-proxy
servicePort: 4180
path: /oauth2
tls:
- hosts:
- cloud.<mydomain>
secretName: cloud-demo-crt
In my static site ingress I have the following added to metadata.annotations:
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$request_uri"
I'm not 100% sure whether these annotations should always be set as such, or whether I should have varies these for B2C/OIDC, but they seem to go off to the proxy, it's just what the proxy does next that fails.
Note that the log does indicate that oauth2-proxy connected to B2C, indeed if the issuer uri changes, then it goes into a crash fallback loop.
There seem to be anumber of articles about how to set this up, so I'm sure it's possible, but I got a little lost. If somebody could help with the setup or ideas for debugging, that would be wonderful.
Thanks.
Now I'm able to reliably get a ?state= and code= to display in the browser window on the /oauth2/callback page, but the page reports Internal Error. oauth2_proxy is logging when it should now, and the log says:
[2020/06/03 21:18:07] [oauthproxy.go:803] Error redeeming code during OAuth2 callback: token exchange: oauth2: server response missing access_token
My Azure B2C Audit log howwever says that it is issuing id_tokens.
When I look at the source code to oauth2_proxy, it looks as though the problem occurs during oauth2.config.Exchange() - which is in the goloang library - I don't know what that does, but I don't think that it works properly with Azure B2c. Does anybody have an idea how I can progress from here?
Thanks.
Mark

Answer 1

I resorted to compiling and debugging the proxy app in VSCode. I ran a simple NGINX proxy to supply TLS termination to the proxy to allow the Azure B2C side to function. It turns out that I had got a lot of things wrong. Here are a list of problems that I resolved in the hope that somebody else might be able to use this to run their own oauth_proxy with Azure B2C.
When attached to a debugger, it is clear that oauth2_proxy reads the token and expects to fin, in turn access_token, then id_token, it then requires (by default) the "email" claim.
To get an "access_token" to return, you have to request access to some resource. Initially I didn't have this. In my yaml file I had:
- --scope=openid
Note: do not put quotation marks around your scope value in YAML, because they are treaded as a part of the requested scope value!
I had to set up a "read" scope in Azure B2C via "App Registrations" and "Expose an API". My final scope that worked was of the form:
- --scope=https://<myspacename>.onmicrosoft.com/<myapiname>/read openid
You have to make sure that both scopes (read and openid) go through together, otherwise you don't get an id_token. If you get an error saying that you don't have an id_token in the server response, make sure that both values are going through in a single use of the --scope flag.
Once you have access_token and id_token, oauth2_proxy fails because there is no "email" claim. Azure B2C has an "emails" claim, but I don't think that can be used. To get around this, I used the object id instead, I set:
- --user-id-claim=oid
The last problem I had was that no cookies were being set in the browser. I did see an error that the cookie value itself was too long in the oauth2-proxy output, and I removed the "offline_access" scope and that message went away. There were still not cookies in the browser however.
My NGinX ingress log did however have a message that the Headers were more than 8K, and NGinX was reporting a 503 error because of this.
In the oauth2-proxy documents, there is a description that a Redis store should be used if your cookie is long - it specifically identifies Azure AD cookies as being long enough to warrant a Redis solution.
I installed a single node Redis to test (unhardened) using a YAML config from this answer https://stackoverflow.com/a/53052122/2048821 - The --session-store-type=redis and --redis-connection-url options must be used.
The final Service/Deployment for my oauth2_proxy look like this:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
k8s-app: oauth2-proxy
name: oauth2-proxy
namespace: default
spec:
replicas: 1
selector:
matchLabels:
k8s-app: oauth2-proxy
template:
metadata:
labels:
k8s-app: oauth2-proxy
spec:
containers:
- args:
- --provider=oidc
- --email-domain=*
- --upstream=file:///dev/null
- --http-address=0.0.0.0:4180
- --redirect-url=https://<myhost>/oauth2/callback
- --oidc-issuer-url=https://<mynamespane>.b2clogin.com/tfp/<my-tenant>/b2c_1_signin/v2.0/
- --cookie-secure=true
- --cookie-domain=<myhost>
- --cookie-secret=<mycookiesecret>
- --user-id-claim=oid
- --scope=https://<mynamespace>.onmicrosoft.com/<myappname>/read openid
- --reverse-proxy=true
- --skip-provider-button=true
- --client-id=<myappid>
- --client-secret=<myclientsecret>
- --session-store-type=redis
- --redis-connection-url=redis://redis:6379
# Register a new application
image: quay.io/pusher/oauth2_proxy:latest
imagePullPolicy: Always
name: oauth2-proxy
ports:
- containerPort: 4180
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
labels:
k8s-app: oauth2-proxy
name: oauth2-proxy
namespace: default
spec:
ports:
- name: http
port: 4180
protocol: TCP
targetPort: 4180
selector:
k8s-app: oauth2-proxy
Hope that this saves somebody a lot of time.
Mark

Answer 2

I tried to follow the Mark Rabjohn’s answer , but was getting errors like
oidc: issuer did not match the issuer returned by provider, expected
"https://your-tenant-name.b2clogin.com/tfp/c5b28ff6-f360-405b-85d0-8a87b5783d3b/B2C_1A_signin/v2.0/"
got
"https://your-tenant-name.b2clogin.com/c5b28ff6-f360-405b-85d0-8a87b5783d3b/v2.0/“
( no policy name in the url)
It is a known issue (https://security.stackexchange.com/questions/212724/oidc-should-the-provider-have-the-same-address-as-the-issuer)
I'm aware that a few of the mainstream providers such as Microsoft
doesn't strictly follow this pattern but you'll have to take it up
with them, or consider the workarounds given by the OIDC library.
Fortunately oauth2-proxy supports
--skip-oidc-discovery parameter:
bypass OIDC endpoint discovery. --login-url, --redeem-url and --oidc-jwks-url must be configured in this case.
The example of parameters is the following:
- --skip-oidc-discovery=true
- --login-url=https://<mynamespace>.b2clogin.com/<mynamespace>.onmicrosoft.com/
b2c_1a_signin/oauth2/v2.0/authorize
- --redeem-url=https://<mynamespace>.b2clogin.com/<mynamespace>.onmicrosoft.com/
b2c_1a_signin/oauth2/v2.0/token
- --oidc-jwks-url=https://<mynamespace>.b2clogin.com/<mynamespace>.onmicrosoft.com/
b2c_1a_signin/discovery/v2.0/keys
To create scope I had to set up Application ID URI in Azure B2C via "App Registrations" and "Expose an API".
(e.g see https://learn.microsoft.com/en-us/azure/active-directory-b2c/tutorial-web-api-dotnet?tabs=app-reg-ga#configure-scopes).
I also have to grant admin permissions as described in https://learn.microsoft.com/en-us/azure/active-directory-b2c/add-web-api-application?tabs=app-reg-ga#grant-permissions (see also Application does not have sufficient permissions against this web resource to perform the operation in Azure AD B2C)
- —scope=https://<mynamespace>.onmicrosoft.com/<myappname>/<scopeName> openid
You also should specify
- —oidc_issuer_url=https://<mynamespace>.b2clogin.com/<TenantID>/v2.0/
Using Directory/tenantId in oidc_issuer_url satifies validation in callback/redeem stage, and you don't need to set useinsecure_oidc_skip_issuer_verification=true.
Also note that redirect-url=https:///oauth2/callback should be registered in AAD B2C as application Redirect URI( navigate in Application Overview pane to Redirect URIs link)

Answer 3

I am not clear about debugging, but according to your issue it's looking you are not passing header param.
On static site ingress please add this one as well and try
nginx.ingress.kubernetes.io/auth-response-headers: X-Auth-Request-Access-Token, Authorization
Or this one
nginx.ingress.kubernetes.io/auth-response-headers: Authorization

Answer 4

Based on the information by Mark Rabjohn and Michael Freidgeim I also got (after hours of trying) a working integration with Azure AD B2C. Here is a configuration to reproduce a working setup, using docker-compose for testing it out locally:
Local setup
version: "3.7"
services:
oauth2proxy:
image: quay.io/oauth2-proxy/oauth2-proxy:latest
command: >
--provider=oidc
--email-domain=*
--upstream=http://web
--http-address=0.0.0.0:9000
--redirect-url=http://localhost:9000
--reverse-proxy=true
--skip-provider-button=true
--session-store-type=redis
--redis-connection-url=redis://redis:6379
--oidc-email-claim=oid
--scope="https://<mynamepsace>.onmicrosoft.com/<app registration uuid>/read openid"
--insecure-oidc-skip-issuer-verification=true
--oidc-issuer-url=https://<mynamespace>.b2clogin.com/<mynamepsace>.onmicrosoft.com/<policy>/v2.0/
environment:
OAUTH2_PROXY_CLIENT_ID: "<app registration client id>"
OAUTH2_PROXY_CLIENT_SECRET: "<app registration client secret>"
OAUTH2_PROXY_COOKIE_SECRET: "<secret follow oauth2-proxy docs to create one>"
ports:
- "9000:9000"
links:
- web
web:
image: kennethreitz/httpbin
ports:
- "8000:80"
redis:
image: redis:latest
The important bits here are these options:
--oidc-email-claim=oid
--scope="https://<mynamepsace>.onmicrosoft.com/<app registration uuid>/read openid"
--insecure-oidc-skip-issuer-verification=true
--oidc-issuer-url=https://<mynamespace>.b2clogin.com/<mynamepsace>.onmicrosoft.com/<policy>/v2.0/
Using --insecure-oidc-skip-issuer-verification=true allows you to skip the explicit mentioning of the endpoints using --login-url --redeem-url --oidc-jwks-url.
--oidc-email-claim=oid replaces the deprecated option --user-id-claim=oid mentioned by Mark Rabjohn
The scope is needed also just as Mark is explaining.
Azure AD B2C settings
These are the steps summarized that are necessary to perform in Azure AD B2C portal:
In the user flow, go to "application claims" and enable "User's Object ID". This is required to make the --oidc-email-claim=oid setting work
In the app registration, in "API permissions", create a new permission with the name read. The url for this permission is the value that you need to fill in for --scope="...".