[prometheus-json-exporter] Expand DNS SRV records when defined as targets url

[ProtossGP32]

Is your feature request related to a problem ?

I'm trying to scrape JSON payloads from multiple replicas of a Pod, exposed through a Headless Service so all Pod IPs can be at least retrieved through Kubernetes DNS:

# kubectl exec dnsutils -- nslookup -type=SRV _http._tcp.headless-service.namespace.svc.cluster.local
Server:         10.43.0.10
Address:        10.43.0.10#53

_http._tcp.headless-service.namespace.svc.cluster.local     service = 0 50 8080 10-42-0-85.headless-service.namespace.svc.cluster.local.  
_http._tcp.headless-service.namespace.svc.cluster.local     service = 0 50 8080 10-42-0-83.headless-service.namespace.svc.cluster.local.  

This is the simplified diagram of the involved components:

flowchart TB;
  subgraph APP_NS["App Namespace"]
    HS["Headless service"]
    POD1["Pod 1<br/>10.42.0.85"]
    POD2["Pod 2<br/>10.42.0.83"]
  end
  SM["Service Monitor"]
  JS["JSON Exporter<br/>service"]
  JP["JSON Exporter<br/>pod"]

  HS ---->|"Only one Pod<br/>scraped each interval"| JP
  linkStyle 0 stroke:red,color:red;
  SM -->|"Uses"| JS --> JP -->|"Scrapes SRV as target uri<br/>every interval time"| HS
  HS -->|"Round robin"| POD1 
  HS -->|"Round robin"| POD2

Loading

This causes the data to be retrieved in an alternate way, thus creating incongruent metrics (each scrape interval the data comes from a different pod):

Right now I can't compose each scrape target based on each Pod IP programmatically before the application deployment, and even after its deployment the Helm Chart doesn't have tools to properly retrieve them.

Describe the solution you'd like.

I'd expect either the Helm chart to be able to compose a new target for each available IP or the JSON exporter to retrieve all resolved IPs by the headless service and use them, so all pods are scraped at the same time:

flowchart TB;
  subgraph APP_NS["App Namespace"]
    HS["Headless service"]
    POD1["Pod 1<br/>10.42.0.85"]
    POD2["Pod 2<br/>10.42.0.83"]
  end
  SM["Service Monitor"]
  JS["JSON Exporter<br/>service"]
  JP["JSON Exporter<br/>pod"]

  HS ---->|"All Pods<br/>scraped each interval"| JP
  linkStyle 0 stroke:green,color:green;
  SM -->|"Uses"| JS --> JP -->|"Scrapes SRV as target uri<br/>every interval time"| HS
  HS -->|"Round robin"| POD1 
  HS -->|"Round robin"| POD2

Loading

Describe alternatives you've considered.

I've considered defining all targets beforehand using GoTemplate( {{ -range ip : headless_service_ips }} or something along the lines ), but I don't really know how to retrieve this Kubernetes information on Helm release time.

Additional context.

None