Ngninx ingress port conflict 8181

[robvadai]

Overview

Nginx ingress default server port is 8181 which conflicts with kiam.

See documentation here and look for --default-server-port.

How I found out the error

Set up KIAM using Helm chart version 6.0.0, helmfile config:

- name: kiam
  namespace: kube-system
  chart: uswitch/kiam
  version: 6.0.0
  values:
  - fullnameOverride: kiam
    agent:
      fullnameOverride: kiam-agent
      # gatewayTimeoutCreation: 120s
      log:
        level: debug
      # keepaliveParams:
      #   time: 120s
      #   timeout: 120s
      extraEnv:
      - name: GRPC_GO_LOG_SEVERITY_LEVEL
        value: debug
      - name: GRPC_GO_LOG_VERBOSITY_LEVEL
        value: "10"
      host:
        iptables: true
      prometheus:
        scrape: false
      resources:
        limits:
          cpu: 800m
          memory: 512Mi
        requests:
          cpu: 100m
          memory: 256Mi
    server:
      fullnameOverride: kiam-server
      useHostNetwork: false
      deployment:
        enabled: true
        replicas: 1
      probes:
        serverAddress: localhost
      keepaliveParams:
          maxConnectionAge: 1s
          maxConnectionAgeGrace: 1s
      log:
        level: info
      assumeRoleArn: {{ .Environment.Values.applications.kiam.server.assumeRoleArn }}
      sslCertHostPath: {{ .Environment.Values.applications.kiam.server.sslCertHostPath }}
      resources:
        limits:
          cpu: 800m
          memory: 512Mi
        requests:
          cpu: 100m
          memory: 256Mi

agent and server both crashing:

kiam-agent-rwx4c                           0/1     CrashLoopBackOff   6          10m
kiam-server-58d946f7b8-knlqx               0/1     CrashLoopBackOff   7          13m

agent log:

{"level":"error","msg":"error creating server gateway: error dialing grpc server: context deadline exceeded","time":"2021-05-16T09:49:54Z"}
{"level":"fatal","msg":"fatal error: error dialing grpc server: context deadline exceeded","time":"2021-05-16T09:49:54Z"}

server log:

{"level":"info","msg":"starting server","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"started prometheus metric listener 0.0.0.0:9620","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"will serve on 0.0.0.0:443","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"starting credential manager process 0","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"starting credential manager process 1","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"starting credential manager process 2","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"starting credential manager process 3","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"starting credential manager process 4","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"starting credential manager process 5","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"starting credential manager process 6","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"starting credential manager process 7","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"started cache controller","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"started namespace cache controller","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"listening","time":"2021-05-16T09:50:39Z"}
{"level":"info","msg":"stopping server","time":"2021-05-16T09:51:19Z"}
{"level":"info","msg":"stopped","time":"2021-05-16T09:51:19Z"}

Solution

Use a different agent port, in my case I set it to agent.host.port: 18181 and it works.

Proposal

Probably good idea to use a different default port or at least update the README.