About K8s RBAC and its potential to enforce isolation

[marshtompsxd]

Context: we were discussing whether we should set RBAC policies to enforce the isolation properties of k8s controllers, i.e., the controller does not create/update/delete any object that it is not supposed to.

In short, the k8s RBAC policies are not expressive enough to enforce the level of isolation properties we are expecting.

RBAC is the common mechanism used by k8s to grant and restrict permissions. The detailed documentation can be found here: https://kubernetes.io/docs/reference/access-authn-authz/rbac/. By default, each controller deployed on k8s has no permissions on any resources. The user needs to specify what type of operations (e.g., update, delete, etc.) the controller can perform on what types of resources (e.g., pods, configmaps, etc.) using the field verbs and resources of the k8s RBAC. Besides, there is a resource_names field that allows users to specify the names of the resources that the controller is allowed to read/write as an allowlist.

In conclusion, using annotations seems to be a better way for us to express and prove the isolation properties.