Add annotations and labels to generated secrets

[arnestorksen]

In our organization, different teams have different namespaces, but we want them to be able to use the same Kafka cluster. The plan is to use SystemUsers for this purpose. This will generate secrets in the same namespace as the ServiceUser, where the aiven-token secret is accessible. In order to distribute the generated secret to other namespaces, something like reflector can be used. But this uses annotations to allow secrets to be copied to other namespaces. The only way to do this now is to run something like this, after the secret has been generated:

kubectl annotate secret kafka-crab-connection reflector.v1.k8s.emberstack.com/reflection-allowed=true

A better way would be to be able to specify the annotation i want directly on the ServiceUser resource. I can imagine this functionality can be useful for labels too.

Example ServiceUser with annotations/labels:

apiVersion: aiven.io/v1alpha1
kind: ServiceUser
metadata:
  # the name of our user 🦀
  name: crab
spec:
  authSecretRef:
    name: aiven-token
    key: token

  # the Secret name we will store the users' connection information
  # looks exactly the same as the Secret generated when creating the Kafka cluster
  # we will use this Secret to produce and consume events later!
  connInfoSecretTarget:
    name: kafka-crab-connection
    annotations:
      reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
    labels:
      team: "red"

  # the Aiven project the user is related to
  project: my-project

  # the name of our Kafka Service
  serviceName: my-kafka-service

[mortenlj]

Just another user here, but it sounds like your use case could be solved with a couple different approaches:

First of all, if #183 is solved, your teams could create their ServiceUser in their own namespace and no copying would be needed. We are considering starting work on a PR for that issue in the near future.

Further afield, your use case sounds to be close to how we use Aiven Kafka in nais, in which case our operator for provisioning service users and secrets might be an option: https://github.com/nais/aivenator

[Serpentiel]

hey, @arnestorksen! 👋

thanks for bringing this issue, we are currently looking into adding support for this in the next release :)

[Serpentiel]

hey everyone! 👋

this was just released :)