Ingress is not created

Question

Ingress is not created

kivio opened this issue 5 years ago · 7 comments

I have install concourse using helm chart.
I have add custom ingress data but two things happen:

first of all ingress isn't created
second - web pod is ignoring externalUrl flag.

Service is created using helm command:

helm install builder -f concourse-settings.yaml concourse/concourse --namespace builder

My config file:

concourse:
  web:
    enabled: true
    externalUrl: builder.my.domain.name
    bindPort: 80
    ingress:
      hosts:
        - builder.my.domain.name
      enabled: true
      annotations:
        external-dns.alpha.kubernetes.io/hostname: builder.my.domain.name
        kubernetes.io/ingress.class: alb
        alb.ingress.kubernetes.io/scheme: internet-facing
        alb.ingress.kubernetes.io/target-type: ip
        alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
        alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
        alb.ingress.kubernetes.io/certificate-arn: <arn-here>
  worker:
    baggageclaim:
      driver: btrfs
  persistence:
    worker:
      storageClass: gp2
  postgresql:
    persistence:
      storageClass: gp2
  secrets:
    bitbucketCloudClientId: <token>
    bitbucktetCloudClientSecret: <secret>

Because ingress creation isn't working I have made ingress myself:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: "concourse-ingress"
  namespace: "builder"
  annotations:
    external-dns.alpha.kubernetes.io/hostname: builder.my.domain.name
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    alb.ingress.kubernetes.io/certificate-arn: <arn-here>
spec:
  rules:
    - http:
        paths:
          - path: /*
            backend:
              serviceName: "builder-web"
              servicePort: 80

Everything is working - i can access dashboard on: https://builder.my.domain.name - but when I click login button it redirects me to 127.0.0.1

I have checked and pod web have configured enviornment variable CONCOURSE_EXTERNAL_URL: builder.my.domain.name.

Then as i supposed it should work correctly.
Have you any advises why it doesn't work correctly?

Answer 1 · 2020-05-22T17:38:18.000Z

Can it be related with Helm3 usage?

Answer 2 · 2020-05-25T07:42:13.000Z

Ok, my fault with externalURL - it should contain protocol https in this case.
But Ingress is still not created

Answer 3 · 2020-05-29T21:20:51.000Z

Tried reproducing using helm template and indeed the web-ingress.yaml is not generated :(

Answer 4 · 2020-05-29T21:32:17.000Z

nvm, I was able to generate it! Annoying thing about helm is that there's no good way of finding out if the keys you passed are used or not. You placed everything in your config under the concourse key but all the settings you wanted to change were not under the concourse key, they were under other top-level keys.

---
web:
  enabled: true
  externalUrl: builder.my.domain.name
  bindPort: 80
  ingress:
    hosts:
      - builder.my.domain.name
    enabled: true
    annotations:
      external-dns.alpha.kubernetes.io/hostname: builder.my.domain.name
      kubernetes.io/ingress.class: alb
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/target-type: ip
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
      alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
      alb.ingress.kubernetes.io/certificate-arn: <arn-here>

I saved that to a file and did helm template: helm template -f ./issue107.yaml ./ | vim

and it generated the web-ingress

# Source: concourse/templates/web-ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: release-name-web
  labels:
    app: release-name-web
    chart: "concourse-11.0.0"
    release: "release-name"
    heritage: "Tiller"
  annotations:
    alb.ingress.kubernetes.io/actions.ssl-redirect: "{\"Type\": \"redirect\", \"RedirectConfig\": { \"Protocol\": \"HTTPS\", \"Port\": \"443\", \"StatusCode\": \"HTTP_301\"}}"
    alb.ingress.kubernetes.io/certificate-arn: "<arn-here>"
    alb.ingress.kubernetes.io/listen-ports: "[{\"HTTP\": 80}, {\"HTTPS\":443}]"
    alb.ingress.kubernetes.io/scheme: "internet-facing"
    alb.ingress.kubernetes.io/target-type: "ip"
    external-dns.alpha.kubernetes.io/hostname: "builder.my.domain.name"
    kubernetes.io/ingress.class: "alb"
spec:
  rules:
    - host: builder.my.domain.name
      http:
        paths:
          - backend:
              serviceName: release-name-web
              servicePort: 8080

I know the values.yaml is really large and hard to navigate, easy to get lost and figure out which keys fall under what.

Answer 5 · 2020-05-29T21:35:22.000Z

More context:

The structure of the values.yaml is this:

concourse top-level key: all settings specific to configuring the concourse binary
All other top-level keys: configuring a k8s resource

Hope that helps :)

Answer 6 · 2020-06-04T08:04:41.000Z

I try to install concourse helm chart on a local k3s cluster and having dnsmasq pointing the externalUrl to the traefik ingress controller (TYPE: LoadBalancer) external IP.

On browsing to https://concourse.cluster.local I can get to the welcome page, but on clicking login I get a "too many redirects" error

my values.yml snippet

concourse:
  web:
    kubernetes:
      namespacePrefix: concourse
    bindPort: 80
    localAuth:
      enabled: true
    auth:
      mainTeam:
        localUser: admin 
    externalUrl: https://concourse.cluster.local
    tls:
      enabled: true
      bindPort: 443
...
web:
  service:
    type: ClusterIP
...
secrets:
  create: true
  localUsers: test:test
<snip certsAndKeys>

the ingress I get from that is:

$ k get ingress concourse-web -o yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: traefik
    meta.helm.sh/release-name: concourse
    meta.helm.sh/release-namespace: default
...
spec:
  rules:
  - host: concourse.cluster.local
    http:
      paths:
      - backend:
          serviceName: concourse-web
          servicePort: 80
        pathType: ImplementationSpecific
status:
  loadBalancer:
    ingress:
    - ip: 192.168.64.3

on browsing to https://concourse.cluster.local I can get to the welcome page, but on clicking login I get a "too many redirects" error

whithin terminal fly --target local login --concourse-url http://concourse.cluster.local -u test -p test succeeds.

how do I have to configure ingress (annotations??) to get the concourseUI working in my (chrome) browser? (problem seems to be my dnsmasq address=/.cluster.local/192.168.64.3)

any ideas?

Answer 7 · 2020-08-30T10:53:41.000Z

I think README.md should be more consistent about "internal app file settings" and k8s resources settings. Second file created ingress:

minikube@vukuber:~/builds/concourse$ diff values.yaml values-iss107.yaml
19,26d18
<     ingress:
<       enabled: true
<       annotations:
<         kubernetes.io/ingress.class: nginx
<         kubernetes.io/tls-acme: 'true'
<       hosts:
<         - cicd.example.ru
<
28a21,28
> web:
>   ingress:
>     enabled: true
>     annotations:
>       kubernetes.io/ingress.class: nginx
>       kubernetes.io/tls-acme: 'true'
>     hosts:
>       - cicd.example.ru

The difference - I created separate 'web' block with ingress desc on the top level. My suggestion to maintainers - either put to README.md all the app keys with 'concourse' prefix(as a top key you have in templates) or put everything to the same level. As of now it looks a bit messy IMO...