ArgoCD have a problem with labels from Kube-State-Metrics
rosariofe opened this issue · 1 comments
We encountered a issue in the template from k8s-monitoring here related to the kube-state-metrics, when I deploy using the ArgoCD, the template force used the k8s-monitoring helm in label kubernetes.io/instance, with this if you need a change to use another label in selector ArgoCD don't identify the another label same using the SelectorOverride or LabelMatchers.
For example:
The ArgoCD will generate the label using the name of cluster but the KSM don't scrape the metrics because in the template it expected the label kubernetes.io/instance: k8s-monitoring (without cluster name)
Argo CD generate like this: kubernetes.io/instance: k8s-monitoring-integration-us-east-1
K8s-monitoring-helm Kube-State-Metrics expected is: kubernetes.io/instance: k8s-monitoring
I tried to use selectorOverride like this:
kube-state-metrics:
selectorOverride:
app.kubernetes.io/name: k8s-monitoring
app.kubernetes.io/instance: k8s-monitoring
Or labelMatchers like this:
labelMatchers:
app.kubernetes.io/name: kube-state-metrics
I tried to use both but they don’t work for me, and looking at the template I found this line here: https://github.com/grafana/k8s-monitoring-helm/blob/main/charts/k8s-monitoring/templates/alloy_config/_kube_state_metrics.alloy.txt#L16
That forces the label Release.Name in the labels because this here using the ArgoCD didn’t work, I think it can be removed of the template to avoid this kind of problem. Because it doesn't make sense to use labelMatchers if the template forces a label, there is no flexibility.