practo/k8s-worker-pod-autoscaler

Install WPA on namespace scope

caiohasouza opened this issue · 5 comments

caiohasouza commented

I'm trying install the WPA on Namespace scope, so i change the ClusterRole to Role and ClusterRoleBinding to RoleBinding, but when the WPA starts i have this error:

Error creating crd: customresourcedefinitions.apiextensions.k8s.io is forbidden: User "system:serviceaccount:NAMESPACE:workerpodautoscaler" cannot create resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope: RBAC: clusterrole.rbac.authorization.k8s.io "workerpodautoscaler" not found

It's possible run the WPA on namespace scope?

Regards

justjkk commented

There are two parts to this issue:

  1. WPA uses CRDs to read the data definitions dynamically and since CRDs cannot be namespaced currently and the controller is creating the CRD regardless of whether it already exists, this causes the above error. We can resolve this by creating the CRD manually and not attempt to create it through the controller. Then the controller doesn't need a ClusterRole and can be per-namespace specific.
  2. When I tested the above solution, I found that the controller is still trying to list the deployments and the CRDs at the cluster scope and hence it is still requires a ClusterRole for now.
E0310 16:12:43.627987       1 reflector.go:125] pkg/generated/informers/externalversions/factory.go:117: Failed to list *v1alpha1.WorkerPodAutoScaler: workerpodautoscalers.k8s.practo.dev is forbidden: User "system:serviceaccount:NAMESPACE:workerpodautoscaler" cannot list resource "workerpodautoscalers" in API group "k8s.practo.dev" at the cluster scope

@alok87 please check if it is possible to change the informers logic to listen to only a specified namespace and if it would take a lot of effort to make the necessary changes. Running individual WPA controllers at namespace scope will be useful to isolate failures and also make it more responsive.

caiohasouza commented

@justjkk yes, i found the same error that you, i appreciate that possible run WPA on namespace, in my case i have a project with stage/production environemnts separated in different namespaces, would be perfect deploy wpa in each environment isolated.

alok87 commented

CRD creation is moving out of WPA code to yamls in this #84

justjkk commented

This still doesn't solve the problem completely.

2. the controller is still trying to list the deployments and the CRDs at the cluster scope and hence it is still requires a ClusterRole for now.

^ is still pending to be solved.

nqkdev commented

Any update on this? This feature would be very useful.