Application gateway redirects to value set in backend-hostname (used for SSL)

[jekahn]

Describe the bug
For SSL enablement, I need to set the annotation:
appgw.ingress.kubernetes.io/backend-hostname: backend

This is the CN of the certificate returned by the backend server

The health probes passes but when using the browser redirects to this value (which does not resolve to an IP)

I have TLS enabled -- please see my annotations here:

 appgw.ingress.kubernetes.io/backend-protocol: https
 appgw.ingress.kubernetes.io/health-probe-hostname: backend
 appgw.ingress.kubernetes.io/request-timeout: "120"
 cert-manager.io/cluster-issuer: letsencrypt-production
kubernetes.io/ingress.class: azure/application-gateway

I had this working, but after upgrading to k8s 1.28, it doesnt work.