JupyterHub "500 : Internal Server Error Redirect loop detected." - jupyter-lab

Question

When running JupyterHub via docker-compose up and logging in for the first time as some_user created via web GUI of JupyterHub I can't seem to spawn the ser
Logs:
sudo docker-compose up
[+] Running 1/0
⠿ Container jh_container Created 0.0s
Attaching to jh_container
jh_container | [D 2022-07-17 12:25:07.224 JupyterHub application:837] Looking for /etc/jupyterhub/jupyterhub_config in /srv/jupyterhub
jh_container | [D 2022-07-17 12:25:07.229 JupyterHub application:858] Loaded config file: /etc/jupyterhub/jupyterhub_config.py
jh_container | [I 2022-07-17 12:25:07.235 JupyterHub app:2771] Running JupyterHub version 2.3.1
jh_container | [I 2022-07-17 12:25:07.235 JupyterHub app:2801] Using Authenticator: nativeauthenticator.nativeauthenticator.NativeAuthenticator
jh_container | [I 2022-07-17 12:25:07.235 JupyterHub app:2801] Using Spawner: dockerspawner.dockerspawner.DockerSpawner-12.1.0
jh_container | [I 2022-07-17 12:25:07.236 JupyterHub app:2801] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-2.3.1
jh_container | [I 2022-07-17 12:25:07.239 JupyterHub app:1606] Loading cookie_secret from /srv/jupyterhub/jupyterhub_cookie_secret
jh_container | [D 2022-07-17 12:25:07.240 JupyterHub app:1775] Connecting to db: sqlite:///jupyterhub.sqlite
jh_container | [D 2022-07-17 12:25:07.248 JupyterHub orm:953] database schema version found: 833da8570507
jh_container | [I 2022-07-17 12:25:07.275 JupyterHub proxy:496] Generating new CONFIGPROXY_AUTH_TOKEN
jh_container | [D 2022-07-17 12:25:07.275 JupyterHub app:2024] Loading roles into database
jh_container | [I 2022-07-17 12:25:07.283 JupyterHub app:1926] Not using allowed_users. Any authenticated user will be allowed.
jh_container | [D 2022-07-17 12:25:07.285 JupyterHub app:2283] Purging expired APITokens
jh_container | [D 2022-07-17 12:25:07.287 JupyterHub app:2283] Purging expired OAuthCodes
jh_container | [D 2022-07-17 12:25:07.288 JupyterHub app:2116] Loading role assignments from config
jh_container | [D 2022-07-17 12:25:07.297 JupyterHub app:2429] Initializing spawners
jh_container | [D 2022-07-17 12:25:07.298 JupyterHub app:2560] Loaded users:
jh_container |
jh_container | [I 2022-07-17 12:25:07.298 JupyterHub app:2840] Initialized 0 spawners in 0.001 seconds
jh_container | [W 2022-07-17 12:25:07.299 JupyterHub proxy:687] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else...
jh_container | [I 2022-07-17 12:25:07.299 JupyterHub proxy:691] Starting proxy # http://:8000
jh_container | [D 2022-07-17 12:25:07.299 JupyterHub proxy:692] Proxy cmd: ['configurable-http-proxy', '--ip', '', '--port', '8000', '--api-ip', '127.0.0.1', '--api-port', '8001', '--error-target', 'http://192.168.64.2:8081/hub/error']
jh_container | [D 2022-07-17 12:25:07.302 JupyterHub proxy:610] Writing proxy pid file: jupyterhub-proxy.pid
jh_container | 12:25:07.614 [ConfigProxy] info: Proxying http://*:8000 to (no default)
jh_container | 12:25:07.615 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
jh_container | [D 2022-07-17 12:25:07.645 JupyterHub proxy:728] Proxy started and appears to be up
jh_container | [D 2022-07-17 12:25:07.646 JupyterHub proxy:821] Proxy: Fetching GET http://127.0.0.1:8001/api/routes
jh_container | 12:25:07.652 [ConfigProxy] info: 200 GET /api/routes
jh_container | [I 2022-07-17 12:25:07.652 JupyterHub app:3089] Hub API listening on http://0.0.0.0:8081/hub/
jh_container | [I 2022-07-17 12:25:07.652 JupyterHub app:3091] Private Hub API connect url http://192.168.64.2:8081/hub/
jh_container | [D 2022-07-17 12:25:07.653 JupyterHub proxy:343] Fetching routes to check
jh_container | [D 2022-07-17 12:25:07.653 JupyterHub proxy:821] Proxy: Fetching GET http://127.0.0.1:8001/api/routes
jh_container | 12:25:07.653 [ConfigProxy] info: 200 GET /api/routes
jh_container | [D 2022-07-17 12:25:07.653 JupyterHub proxy:346] Checking routes
jh_container | [I 2022-07-17 12:25:07.653 JupyterHub proxy:431] Adding route for Hub: / => http://192.168.64.2:8081
jh_container | [D 2022-07-17 12:25:07.653 JupyterHub proxy:821] Proxy: Fetching POST http://127.0.0.1:8001/api/routes/
jh_container | 12:25:07.654 [ConfigProxy] info: Adding route / -> http://192.168.64.2:8081
jh_container | 12:25:07.654 [ConfigProxy] info: Route added / -> http://192.168.64.2:8081
jh_container | 12:25:07.654 [ConfigProxy] info: 201 POST /api/routes/
jh_container | [I 2022-07-17 12:25:07.655 JupyterHub app:3156] JupyterHub is now running at http://:8000
jh_container | [D 2022-07-17 12:25:07.655 JupyterHub app:2764] It took 0.435 seconds for the Hub to start
jh_container | [I 2022-07-17 12:25:19.270 JupyterHub log:189] 302 GET / -> /hub/ (#192.168.64.1) 0.75ms
jh_container | [D 2022-07-17 12:25:19.279 JupyterHub base:326] Refreshing auth for myadmin
jh_container | [D 2022-07-17 12:25:19.281 JupyterHub user:399] Creating <class 'dockerspawner.dockerspawner.DockerSpawner'> for myadmin:
jh_container | [I 2022-07-17 12:25:19.283 JupyterHub log:189] 302 GET /hub/ -> /hub/spawn (myadmin#192.168.64.1) 10.58ms
jh_container | [D 2022-07-17 12:25:19.285 JupyterHub scopes:491] Checking access via scope servers
jh_container | [D 2022-07-17 12:25:19.285 JupyterHub scopes:389] Unrestricted access to /hub/spawn via servers
jh_container | [D 2022-07-17 12:25:19.285 JupyterHub pages:215] Triggering spawn with default options for myadmin
jh_container | [D 2022-07-17 12:25:19.285 JupyterHub base:934] Initiating spawn for myadmin
jh_container | [D 2022-07-17 12:25:19.285 JupyterHub base:938] 0/100 concurrent spawns
jh_container | [D 2022-07-17 12:25:19.285 JupyterHub base:943] 0 active servers
jh_container | [D 2022-07-17 12:25:19.288 JupyterHub roles:477] Checking token permissions against requested role server
jh_container | [I 2022-07-17 12:25:19.289 JupyterHub roles:482] Adding role server to token: <APIToken('3dde...', user='myadmin', client_id='jupyterhub')>
jh_container | [I 2022-07-17 12:25:19.295 JupyterHub provider:607] Creating oauth client jupyterhub-user-myadmin
jh_container | [D 2022-07-17 12:25:19.307 JupyterHub user:728] Calling Spawner.start for myadmin
jh_container | [D 2022-07-17 12:25:19.319 JupyterHub dockerspawner:982] Getting container 'jupyter-myadmin'
jh_container | [I 2022-07-17 12:25:19.320 JupyterHub dockerspawner:988] Container 'jupyter-myadmin' is gone
jh_container | [D 2022-07-17 12:25:19.323 JupyterHub dockerspawner:1148] Starting host with config: {'auto_remove': True, 'binds': {}, 'links': {}, 'mounts': [], 'mem_limit': 0, 'cpu_period': 100000, 'cpu_quota': 0, 'network_mode': 'jupyterhub-network'}
jh_container | [I 2022-07-17 12:25:19.337 JupyterHub dockerspawner:1272] Created container jupyter-myadmin (id: d5b54b4) from image jupyter/scipy-notebook:lab-3.4.3
jh_container | [I 2022-07-17 12:25:19.337 JupyterHub dockerspawner:1296] Starting container jupyter-myadmin (id: d5b54b4)
jh_container | [D 2022-07-17 12:25:19.631 JupyterHub spawner:1258] Polling subprocess every 30s
jh_container | [I 2022-07-17 12:25:20.286 JupyterHub log:189] 302 GET /hub/spawn -> /hub/spawn-pending/myadmin (myadmin#192.168.64.1) 1002.05ms
jh_container | [D 2022-07-17 12:25:20.291 JupyterHub scopes:491] Checking access via scope servers
jh_container | [D 2022-07-17 12:25:20.291 JupyterHub scopes:389] Unrestricted access to /hub/spawn-pending/myadmin via servers
jh_container | [I 2022-07-17 12:25:20.291 JupyterHub pages:401] myadmin is pending spawn
jh_container | [I 2022-07-17 12:25:20.310 JupyterHub log:189] 200 GET /hub/spawn-pending/myadmin (myadmin#192.168.64.1) 20.61ms
jh_container | [D 2022-07-17 12:25:20.468 JupyterHub scopes:491] Checking access via scope read:servers
jh_container | [D 2022-07-17 12:25:20.468 JupyterHub scopes:389] Unrestricted access to /hub/api/users/myadmin/server/progress via read:servers
jh_container | [I 2022-07-17 12:25:20.527 JupyterHub log:189] 200 GET /hub/api (#192.168.64.3) 0.53ms
jh_container | [D 2022-07-17 12:25:20.532 JupyterHub base:281] Recording first activity for <APIToken('3dde...', user='myadmin', client_id='jupyterhub')>
jh_container | [D 2022-07-17 12:25:20.539 JupyterHub scopes:301] Authenticated with token <APIToken('3dde...', user='myadmin', client_id='jupyterhub')>
jh_container | [D 2022-07-17 12:25:20.541 JupyterHub scopes:491] Checking access via scope users:activity
jh_container | [D 2022-07-17 12:25:20.541 JupyterHub scopes:402] Argument-based access to /hub/api/users/myadmin/activity via users:activity
jh_container | [D 2022-07-17 12:25:20.542 JupyterHub users:859] Activity for user myadmin: 2022-07-17T12:25:20.513412Z
jh_container | [D 2022-07-17 12:25:20.542 JupyterHub users:877] Activity on server myadmin/: 2022-07-17T12:25:20.513412Z
jh_container | [I 2022-07-17 12:25:20.545 JupyterHub log:189] 200 POST /hub/api/users/myadmin/activity (myadmin#192.168.64.3) 14.99ms
jh_container | [D 2022-07-17 12:25:20.653 JupyterHub utils:230] Server at http://192.168.64.3:8888/user/myadmin/ responded with 302
jh_container | [D 2022-07-17 12:25:20.653 JupyterHub _version:74] jupyterhub and jupyterhub-singleuser both on version 2.3.1
jh_container | [I 2022-07-17 12:25:20.653 JupyterHub base:963] User myadmin took 1.368 seconds to start
jh_container | [I 2022-07-17 12:25:20.653 JupyterHub proxy:286] Adding user myadmin to proxy /user/myadmin/ => http://192.168.64.3:8888
jh_container | [D 2022-07-17 12:25:20.653 JupyterHub proxy:821] Proxy: Fetching POST http://127.0.0.1:8001/api/routes/user/myadmin
jh_container | 12:25:20.655 [ConfigProxy] info: Adding route /user/myadmin -> http://192.168.64.3:8888
jh_container | 12:25:20.655 [ConfigProxy] info: Route added /user/myadmin -> http://192.168.64.3:8888
jh_container | 12:25:20.655 [ConfigProxy] info: 201 POST /api/routes/user/myadmin
jh_container | [I 2022-07-17 12:25:20.656 JupyterHub users:753] Server myadmin is ready
jh_container | [I 2022-07-17 12:25:20.656 JupyterHub log:189] 200 GET /hub/api/users/myadmin/server/progress (myadmin#192.168.64.1) 189.29ms
jh_container | [D 2022-07-17 12:25:20.662 JupyterHub scopes:491] Checking access via scope servers
jh_container | [D 2022-07-17 12:25:20.662 JupyterHub scopes:389] Unrestricted access to /hub/spawn-pending/myadmin via servers
jh_container | [I 2022-07-17 12:25:20.663 JupyterHub log:189] 302 GET /hub/spawn-pending/myadmin -> /user/myadmin/ (myadmin#192.168.64.1) 2.32ms
jh_container | [I 2022-07-17 12:25:20.666 JupyterHub log:189] 302 GET /user/myadmin/ -> /hub/user/myadmin/ (#192.168.64.1) 0.27ms
jh_container | [D 2022-07-17 12:25:20.668 JupyterHub scopes:491] Checking access via scope access:servers
jh_container | [D 2022-07-17 12:25:20.668 JupyterHub scopes:389] Unrestricted access to /hub/user/myadmin/ via access:servers
jh_container | [I 2022-07-17 12:25:20.669 JupyterHub log:189] 302 GET /hub/user/myadmin/ -> /user/myadmin/?redirects=1 (myadmin#192.168.64.1) 1.35ms
jh_container | [I 2022-07-17 12:25:20.671 JupyterHub log:189] 302 GET /user/myadmin/?redirects=1 -> /hub/user/myadmin/?redirects=1 (#192.168.64.1) 0.25ms
jh_container | [D 2022-07-17 12:25:20.673 JupyterHub scopes:491] Checking access via scope access:servers
jh_container | [D 2022-07-17 12:25:20.673 JupyterHub scopes:389] Unrestricted access to /hub/user/myadmin/ via access:servers
jh_container | [W 2022-07-17 12:25:20.673 JupyterHub base:1615] Redirect loop detected on /hub/user/myadmin/?redirects=1
jh_container | [I 2022-07-17 12:25:22.675 JupyterHub log:189] 302 GET /hub/user/myadmin/?redirects=1 -> /user/myadmin/?redirects=2 (myadmin#192.168.64.1) 2003.39ms
jh_container | [I 2022-07-17 12:25:22.679 JupyterHub log:189] 302 GET /user/myadmin/?redirects=2 -> /hub/user/myadmin/?redirects=2 (#192.168.64.1) 0.67ms
jh_container | [D 2022-07-17 12:25:22.683 JupyterHub scopes:491] Checking access via scope access:servers
jh_container | [D 2022-07-17 12:25:22.683 JupyterHub scopes:389] Unrestricted access to /hub/user/myadmin/ via access:servers
jh_container | [W 2022-07-17 12:25:22.684 JupyterHub base:1615] Redirect loop detected on /hub/user/myadmin/?redirects=2
jh_container | [I 2022-07-17 12:25:26.685 JupyterHub log:189] 302 GET /hub/user/myadmin/?redirects=2 -> /user/myadmin/?redirects=3 (myadmin#192.168.64.1) 4003.62ms
jh_container | [I 2022-07-17 12:25:26.689 JupyterHub log:189] 302 GET /user/myadmin/?redirects=3 -> /hub/user/myadmin/?redirects=3 (#192.168.64.1) 0.66ms
jh_container | [D 2022-07-17 12:25:26.693 JupyterHub scopes:491] Checking access via scope access:servers
jh_container | [D 2022-07-17 12:25:26.693 JupyterHub scopes:389] Unrestricted access to /hub/user/myadmin/ via access:servers
jh_container | [W 2022-07-17 12:25:26.694 JupyterHub base:1615] Redirect loop detected on /hub/user/myadmin/?redirects=3
jh_container | [I 2022-07-17 12:25:34.695 JupyterHub log:189] 302 GET /hub/user/myadmin/?redirects=3 -> /user/myadmin/?redirects=4 (myadmin#192.168.64.1) 8003.82ms
jh_container | [I 2022-07-17 12:25:34.698 JupyterHub log:189] 302 GET /user/myadmin/?redirects=4 -> /hub/user/myadmin/?redirects=4 (#192.168.64.1) 0.41ms
jh_container | [D 2022-07-17 12:25:34.701 JupyterHub scopes:491] Checking access via scope access:servers
jh_container | [D 2022-07-17 12:25:34.701 JupyterHub scopes:389] Unrestricted access to /hub/user/myadmin/ via access:servers
jh_container | [W 2022-07-17 12:25:34.701 JupyterHub web:1787] 500 GET /hub/user/myadmin/?redirects=4 (192.168.64.1): Redirect loop detected.
jh_container | [D 2022-07-17 12:25:34.702 JupyterHub base:1342] No template for 500
jh_container | [E 2022-07-17 12:25:34.719 JupyterHub log:181] {
jh_container | "Host": "192.168.64.2:8081",
jh_container | "Connection": "keep-alive",
jh_container | "Cache-Control": "max-age=0",
jh_container | "Upgrade-Insecure-Requests": "1",
jh_container | "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.53 Safari/537.36",
jh_container | "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
jh_container | "Referer": "http://192.168.64.2:8081/hub/spawn-pending/myadmin",
jh_container | "Accept-Encoding": "gzip, deflate",
jh_container | "Accept-Language": "pl-PL,pl;q=0.9,en-US;q=0.8,en;q=0.7",
jh_container | "Cookie": "jupyterhub-hub-login=[secret]; jupyterhub-session-id=[secret]"
jh_container | }
jh_container | [E 2022-07-17 12:25:34.719 JupyterHub log:189] 500 GET /hub/user/myadmin/?redirects=4 (myadmin#192.168.64.1) 19.43ms
^CGracefully stopping... (press Ctrl+C again to force)
[+] Running 1/1
⠿ Container jh_container Stopped
with follwoing docker-compose.yml:
version: "3.9"
services:
jupyterhub: # Configuration for Hub+Proxy
build:
context: ./jupyterhub/ # path to the Dockerfile
args:
JUPYTERHUB_VERSION: 2.3.1
image: mc_jupyterhub_img:v1 # image name
container_name: jh_container # container name
restart: always
tty: true
stdin_open: true
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:rw"
ports:
- 8000:8000
- 8081:8081
networks:
- ${DOCKER_NETWORK_NAME}
environment:
DOCKER_JUPYTER_IMAGE: jupyter/scipy-notebook:lab-3.4.3
DOCKER_NETWORK_NAME: ${DOCKER_NETWORK_NAME}
env_file:
- .env
command: >
jupyterhub --debug -f /etc/jupyterhub/jupyterhub_config.py
networks:
jupyterhub-network:
name: ${DOCKER_NETWORK_NAME}
and jupyter_config.py:
import os
c = get_config()
import nativeauthenticator
c.JupyterHub.authenticator_class = 'nativeauthenticator.NativeAuthenticator'
c.JupyterHub.template_paths = [f"{os.path.dirname(nativeauthenticator.__file__)}/templates/"]
c.Authenticator.admin_users = {"myadmin"}
import netifaces
docker0 = netifaces.ifaddresses('eth0')
docker0_ipv4 = docker0[netifaces.AF_INET][0]
c.JupyterHub.hub_ip = '0.0.0.0'
c.JupyterHub.hub_connect_ip = docker0_ipv4['addr']
c.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner'
c.DockerSpawner.image = os.environ['DOCKER_JUPYTER_IMAGE']
network_name = os.environ['DOCKER_NETWORK_NAME']
c.DockerSpawner.use_internal_ip = True
c.DockerSpawner.network_name = network_name
c.DockerSpawner.extra_host_config = { 'network_mode': network_name }
c.DockerSpawner.remove = True
How can I fix this? What am I doing wrong?