Proxy Outpost no longer works after changing server URL

Question

Proxy Outpost no longer works after changing server URL

rdmchr opened this issue 4 months ago · 0 comments

Describe the bug
I have two Authentik Proxy outposts deployed. One on the server that is hosting Authentik and one on another server. During a migration I noticed that changing my TLD would cause both proxies to malfunction (see screenshot below). Both of them seem to be able to connect to my Authentik server fine (see screenshot below), but handling requests does not work.

To Reproduce
Steps to reproduce the behavior:

Create a new Authentik instance with a proxy outpost (I used Traefik as my reverse proxy)
Verify that you can access a service that is protected with the Authentik middleware, you can use the Traefik dashboard
Stop Authentik & Traefik, change the domain (I switched TLD), start the services again
You won't be able to access the dashboard

Expected behavior
There should be some kind of way to migrate domains, or this should be handled automatically.

Screenshots
Opening a misbehaving page:

Outposts seem to be connected:

Logs
Proxy log output:

authentik-proxy  | {"event":"Loaded config","level":"debug","path":"inbuilt-default","timestamp":"2024-06-09T21:32:01Z"}
authentik-proxy  | {"event":"Loaded config from environment","level":"debug","timestamp":"2024-06-09T21:32:01Z"}
authentik-proxy  | {"event":"not enabling debug server, set `AUTHENTIK_DEBUG` to `true` to enable it.","level":"info","logger":"authentik.go_debugger","timestamp":"2024-06-09T21:32:01Z"}
authentik-proxy  | {"event":"Successfully connected websocket","level":"info","logger":"authentik.outpost.ak-ws","outpost":"b1ff61cd-076e-40c8-85e4-50a35edbd164","timestamp":"2024-06-09T21:32:01Z"}
authentik-proxy  | {"event":"Starting Metrics server","level":"info","listen":"0.0.0.0:9300","logger":"authentik.outpost.metrics","timestamp":"2024-06-09T21:32:02Z"}
authentik-proxy  | {"event":"Starting HTTP server","level":"info","listen":"0.0.0.0:9000","logger":"authentik.outpost.proxyv2","timestamp":"2024-06-09T21:32:02Z"}
authentik-proxy  | {"event":"Starting HTTPS server","level":"info","listen":"0.0.0.0:9443","logger":"authentik.outpost.proxyv2","timestamp":"2024-06-09T21:32:02Z"}
authentik-proxy  | {"event":"Starting authentik outpost","hash":"tagged","level":"info","logger":"authentik.outpost","timestamp":"2024-06-09T21:32:03Z","version":"2024.4.2"}
authentik-proxy  | {"event":"No state saved in session","level":"warning","logger":"authentik.outpost.proxyv2.application","name":"Provider for dns.XXXX.dev","timestamp":"2024-06-09T21:32:06Z"}
authentik-proxy  | {"event":"/outpost.goauthentik.io/auth/traefik","host":"dns.XXXX.dev","level":"info","logger":"authentik.outpost.proxyv2.application","method":"GET","name":"Provider for dns.XXXX.dev","remote":"172.18.0.2:34254","runtime":"0.515","scheme":"http","size":49,"status":302,"timestamp":"2024-06-09T21:32:06Z","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"}
authentik-proxy  | {"event":"/outpost.goauthentik.io/auth/traefik","host":"dns.XXXX.dev","level":"info","logger":"authentik.outpost.proxyv2.application","method":"GET","name":"Provider for dns.XXXX.dev","remote":"172.18.0.2:34254","runtime":"0.809","scheme":"http","size":364,"status":302,"timestamp":"2024-06-09T21:32:06Z","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"}
authentik-proxy  | {"error":"oauth2: cannot fetch token: 404 Not Found\nResponse: 404 page not found\n","event":"failed to redeem code","level":"warning","logger":"authentik.outpost.proxyv2.application","name":"Provider for dns.XXXX.dev","timestamp":"2024-06-09T21:32:07Z"}
authentik-proxy  | {"event":"/outpost.goauthentik.io/auth/traefik","host":"dns.XXXX.dev","level":"info","logger":"authentik.outpost.proxyv2.application","method":"GET","name":"Provider for dns.XXXX.dev","remote":"172.18.0.2:34254","runtime":"1.090","scheme":"http","size":0,"status":400,"timestamp":"2024-06-09T21:32:07Z","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"}

Version and Deployment (please complete the following information):

authentik version: 2024.4.2
Traefik: v3.0
Deployment: Docker Compose version v2.27.0

Client: Docker Engine - Community
 Version:           26.1.3
 API version:       1.45
 Go version:        go1.21.10
 Git commit:        b72abbb
 Built:             Thu May 16 08:33:25 2024
 OS/Arch:           linux/arm64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.3
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.10
  Git commit:       8e96db1
  Built:            Thu May 16 08:33:25 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.32
  GitCommit:        8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Additional context
Authentik Proxy snippet from my docker-compose.yml (the one that runs on the same server as Authentik)

  authentik-proxy:
    image: ghcr.io/goauthentik/proxy
    container_name: authentik-proxy
    networks:
      - authentik
      - traefik
    environment:
      AUTHENTIK_HOST: https://sso.XXXX.church
      AUTHENTIK_INSECURE: "true"
      AUTHENTIK_TOKEN: XXXX
    labels:
      - "traefik.enable=true"
      - "traefik.port=9000"
      - "traefik.http.routers.authentik.rule=HostRegexp(`{subdomain:[A-Za-z0-9](?:[A-Za-z0-9\\-]{0,61}[A-Za-z0-9])?}.XXXX.cool`) && PathPrefix(`/outpost.goauthentik.io/`)" <-- Tweaking this domain didn't help
      - "traefik.http.routers.authentik.entrypoints=websecure"
      - "traefik.http.routers.authentik.tls.certresolver=le"
      # `authentik-proxy` refers to the service name in the compose file.
      - "traefik.http.middlewares.authentik.forwardauth.address=http://authentik-proxy:9000/outpost.goauthentik.io/auth/traefik"
      - "traefik.http.middlewares.authentik.forwardauth.trustForwardHeader=true"
      - "traefik.http.middlewares.authentik.forwardauth.authResponseHeaders=X-authentik-username,X-authentik-groups,X-authentik-email,X-authentik-name,X-authentik-uid,X-authentik-jwt,X-authentik-meta-jwks,X-authentik-meta-outpost,X-authentik-meta-provider,X-authentik-meta-app,X-authentik-meta-version"
    restart: unless-stopped

Authentik Server snippet from my docker-compose.yml

  server:
    image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2024.4.2}
    container_name: authentik-server
    networks:
      - authentik
      - traefik
    restart: unless-stopped
    command: server
    environment:
      AUTHENTIK_REDIS__HOST: redis
      AUTHENTIK_POSTGRESQL__HOST: postgresql
      AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
      AUTHENTIK_POSTGRESQL__NAME: ${PG_DB:-authentik}
      AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
    volumes:
      - ./media:/media
      - ./custom-templates:/templates
    env_file:
      - .env
    depends_on:
      - postgresql
      - redis
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=traefik"
      - "traefik.http.services.authentik-server.loadbalancer.server.port=9000"

      - "traefik.http.routers.authentik-server.rule=Host(`sso.XXXX.church`)"
      - "traefik.http.routers.authentik-server.entrypoints=websecure"
      - "traefik.http.routers.authentik-server.tls.certresolver=le"
      - "traefik.http.routers.authentik-server.service=authentik-server"