rancher/prometheus-federator

Project Registration Namespaces are NOT imported into correspondent Projects when deploying prometheus-federator with fleet

Closed this issue · 5 comments

Labidi-github commented

Cluster Setup

  • Kubernetes version:
  • Type of Cluster (RKE1, RKE2, k3s, EKS, GKE, AKS, OKE, etc): EKS
  • Installation option (Helm Chart / Custom Installation):
    • If Helm Chart, did you supply any non-default values? If so, what values did you supply: default values
    • If Custom Installation, how did you install it (kustomize, k8s manifest, etc.):

Describe the bug
When installing prometheus-federator with fleet, keeping default helm chart values, Project Registration Namespaces (cattle-project-id) are NOT imported into the Project it is tied to. Therefore, project-owners can't create "Project Monitors" in the Rancher UI. Solution becomes in-exploitable!

To Reproduce
fleet.yaml file:

defaultNamespace: cattle-monitoring-system
helm:
  releaseName: prometheus-federator
  repo: https://charts.rancher.io
  chart: prometheus-federator
  version: 0.1.0

Result
"cattle-project-id" NOT in project

Expected Result
"cattle-project-id" shoud be imported in the project "project-id"

Screenshots

Additional context

Labidi-github commented

The only way found to deploy correctly prometheus-federator is to not use default values !
"clusterId", "systemProjectId" and "url" should be fixed in fleet.yaml file. exp:
fleet.yaml file:

defaultNamespace: cattle-monitoring-system
helm:
  releaseName: prometheus-federator
  repo: https://charts.rancher.io
  chart: prometheus-federator
  version: 0.1.0
  values: 
    global:
      cattle:
        clusterId: global.fleet.clusterLabels.management.cattle.io/cluster-name 
        clusterName: global.fleet.clusterLabels.management.cattle.io/cluster-display-name
        systemProjectId: p-ft5v6
        url: https://my.local.cluster.url.com

How can we avoid hard coding "systemProjectId" and "url" values ?

  • 👍3
MKlimuszka commented

The rancher UI normally fills in the URL value. But in this case there's no way to know that because it is configured with Fleet. Please read the documentation on namespaces to provide the appropriate configurations.
https://github.com/rancher/prometheus-federator/blob/main/docs/design.md#namespaces

aiyengar2 commented

FYI the logic for how the annotation is set that auto-imports namespaces into projects is defined in https://github.com/rancher/helm-project-operator/blob/557114ccbc4137b9272a54705416b03043f6a11a/pkg/controllers/namespace/resources.go#L24 and https://github.com/rancher/helm-project-operator/blob/557114ccbc4137b9272a54705416b03043f6a11a/pkg/controllers/common/operatorlabels.go#L63-L73

As you suggested, you would need to provide the clusterId value or else the annotation will not be set correctly to auto-import @Labidi-github.

aiyengar2 commented

With respect to the following question:

How can we avoid hard coding "systemProjectId" and "url" values ?

Since the Helm chart is installed in the downstream cluster, afaik there are no Kubernetes resources stored within it today that provides access to "global" Rancher information such as:

  • the cluster ID that the management (Rancher) cluster recognizes this cluster as
  • the Projects that the management (Rancher) cluster is organizing downstream namespaces using

As a result, there's no way from a Helm chart level today for the chart to introspect what those values are (i.e. with lookup calls) without getting it through values that are set on helm install (normally provided via the Apps & Marketplace UI).

Perhaps a valid feature request that could be proposed in another ticket would be to introduce such an object to each downstream cluster so that Helm charts installed onto a downstream cluster can reference those contents to be configured without requiring UI intervention to provide those values.

aiyengar2 commented

cc: @manno @olblak @mattfarina, this may be a feature request for the Fleet team to pursue to make it easier to deploy Rancher feature charts with Fleet