ingress-nginx: 400 Bad Request The plain HTTP request was sent to HTTPS port

[shinebayar-g]

I'm getting 400 Bad Request The plain HTTP request was sent to HTTPS port error with Kubernetes ingress-nginx.

my nginx config:

controller:
  config:
    use-proxy-protocol: "true"

  service:
    enabled: true

    annotations:
      service.beta.kubernetes.io/do-loadbalancer-name: "my-ingress-controller"
      service.beta.kubernetes.io/do-loadbalancer-algorithm: "least_connections"
      service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
      service.beta.kubernetes.io/do-loadbalancer-protocol: "https"
      service.beta.kubernetes.io/do-loadbalancer-certificate-id: "aa536df2-6e76-4811-b56f-07e82a2cf123"
      service.beta.kubernetes.io/do-loadbalancer-http-ports: "80"
      service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443"
      service.beta.kubernetes.io/do-loadbalancer-size-slug: "lb-small"
      service.beta.kubernetes.io/do-loadbalancer-enable-backend-keepalive: "true"
      service.beta.kubernetes.io/do-loadbalancer-redirect-http-to-https: "true"

My Ingress config:

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: test-nginx
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - host: mydomain.com
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: test-nginx
                port:
                  number: 80

What am I missing?

[shinebayar-g]

If I change the backend protocol of 443 port of the load balancer, from HTTP to HTTPS it's working as expected.

[timoreimann]

👋 are you saying that the annotations above configured the LB to use HTTP for port 443? That's odd, it shouldn't be the case. 🤔

Let me know for sure and I'll try to reproduce.

[shinebayar-g]

Yeah. Oops looks like I forgot to add screenshot.

[timoreimann]

The forwarding rules are configured correctly on a test cluster/LB of mine, with port 443 being exposed on HTTPS:

Could you please share the Service resource that Helm(?) rendered?

[shinebayar-g]

That's not correct. It should be:

HTTPS on port 443 -> HTTPS on port 30811

Now if you create test ingress resource on k8s with your configuration and try to access it using https. You'll get the error.

[timoreimann]

HTTPS-to-HTTPS on the target/backend part of the request path is currently not supported. #367 tracks work to support this, though I'd be curious to learn what your specific use case is given that the backend traffic runs through the VPC. Understanding this better would help prioritizing the request. Thanks!

[shinebayar-g]

I mean I don't mind having http on nginx and https on DO LB. But problem is it's giving error 400. Did you deploy basic Ingress to confirm the error? If it didn't give 400 error I don't mind. I don't have specific requirement. That's why I'm setting https on backend, because it works.

[timoreimann]

@shinebayar-g I did not deploy an Ingress, though I did test it by sending traffic directly from the LB to a default Nginx instance. I also received a 400 initially, but that was because the default Nginx configuration does not expect Proxy Protocol. Once I flipped the Proxy Protocol annotation to "false", things would work.

Just to be sure, is Proxy Protocol correctly set up on your Nginx? And if so, are you testing things by hitting the LB address from outside the cluster (routing to the public LB IP address from inside the cluster is not supported due to Kubernetes limitations)?

[shinebayar-g]

Do we need proxy protocol? (I thought it only needed if we have HTTPS backend? (nginx) If HTTPS backend isn't supported, then we may don't need this? If that's the case, I don't know why it's working for me? I don't fully know what it does). It was configured on by default.
I'm trying public IP address of the Load balancer outside of the cluster (from my browser)

[timoreimann]

Proxy Protocol is often used to relay the source IP address to the target. (See also DO's blog post when we added support.) Depending on whether that's something you want, you may or may not want to enable the option.

I'd suggest you first try without proxy protocol (i.e., disable both the Helm option and LB annotation) and see if it works in general.

[timoreimann]

Closing since the issue hasn't been updated in a while, assuming things have resolved.