Use DNS name from the Service instead of the Pod IP

[juldrixx]

I've a PodMonitoring resource using mTLS to authenticate to an application, the issue is that GMP is using the IP address of the pod to call the application but the IP is not in DNS names of my certificate.

Get "https://10.4.1.93:8443/nifi-api/flow/metrics/prometheus": tls: failed to verify certificate: x509: cannot validate certificate for 10.4.1.93 because it doesn't contain any IP SANs

Is there a way to make it call the DNS name of the Service exposing the pod instead of the IP address?

[TheSpiritXIII]

Thanks for the bug report! If you were using vanilla Prometheus, you may be able to use a relabel config. Replace __address__, changing only the hostname and keeping the URL path and port numbers.

Unfortunately, it seems like GMP forbids replacing the special __address__ label (see documentation https://github.com/GoogleCloudPlatform/prometheus-engine/blob/main/doc/api.md#scrapeendpoint). GMP tries to be opinionated to prevent footguns.

In this case, I can envision something like this becoming a flag in the future, where setting it automatically adds the relabeling config.