kubernetes-sigs/cloud-provider-kind

How to get ingress working?

gary-archer opened this issue · 4 comments

gary-archer commented

Great to see this repo, to enable ingress into the cluster on a development computer in a more real-world manner.
I want to use it to demonstrate APIs deployed behind an ingress that acts as an API gateway.

I have been having trouble getting ingress connectivity working though:

kind create cluster
kubectl label node kind-control-plane node.kubernetes.io/exclude-from-external-load-balancers-

helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
helm install ingress-nginx nginx-stable/nginx-ingress \
  --namespace ingress-nginx \
  --create-namespace \
  --set serviceNameOverride='ingress-controller-svc' \
  --set fullnameOverride='ingress-controller' \
  --set controller.name=''

EXTERNAL_IP=$(kubectl -n ingress-nginx get svc ingress-controller-svc -o jsonpath="{.status.loadBalancer.ingress[0].ip}")
curl --head "http://$EXTERNAL_IP"
curl: (52) Empty reply from server

I can run your service example from the README successfully.
With the above commands I get the same service and external IP behavior.
The haproxy docker image for the ingress also runs in an equivalent way.
The ingress works if I use MetalLB or port mapping and I get the same behavior with NGINX / Kong / Istio ingresses.

So I think I am missing something at the networking or haproxy level???
Any hints on what I should do differently would be appreciated - regards.

aojea commented

Let me try to reproduce this locally

aojea commented

ok, found the problem

aojea commented

@gary-archer this should fix it #39

  • 👍1
gary-archer commented

Yep - that works fine now - great update.
In case useful to anyone else, here is how I adapted your lb-service example to use an ingress:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: policy-local
  labels:
    app: MyLocalApp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: MyLocalApp
  template:
    metadata:
      labels:
        app: MyLocalApp
    spec:
      containers:
      - name: agnhost
        image: registry.k8s.io/e2e-test-images/agnhost:2.40
        args:
          - netexec
          - --http-port=8080
          - --udp-port=8080
---
apiVersion: v1
kind: Service
metadata:
  name: lb-service-local
spec:
  selector:
    app: MyLocalApp
  ports:
    - name: http
      protocol: TCP
      port: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: lb-service-local-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: myservice
    http:
      paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: lb-service-local
              port:
                number: 8080

And this is the full script to deploy resources and then call an incorrect route and then a correct route:

kind create cluster
kubectl label node kind-control-plane node.kubernetes.io/exclude-from-external-load-balancers-

helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
helm install ingress-nginx nginx-stable/nginx-ingress \
  --namespace ingress-nginx \
  --create-namespace \
  --set serviceNameOverride='ingress-controller-svc' \
  --set fullnameOverride='ingress-controller' \
  --set controller.name=''

DOCKER_IMAGE='local/cloud-provider-kind:v0.1'
git clone https://github.com/kubernetes-sigs/cloud-provider-kind
cd cloud-provider-kind
docker build . -t $DOCKER_IMAGE
docker run --rm --network kind -d -v /var/run/docker.sock:/var/run/docker.sock $DOCKER_IMAGE

kubectl create namespace applications
kubectl -n applications apply -f lb-service.yaml

EXTERNAL_IP=$(kubectl -n ingress-nginx get svc ingress-controller-svc -o jsonpath="{.status.loadBalancer.ingress[0].ip}")
curl --head "http://$EXTERNAL_IP"
curl "http://$EXTERNAL_IP/hostname" -H 'Host: myservice'
  • 👍2