mittwald/kubernetes-replicator

Secrets not being updated after patch

ErikTMA opened this issue · 4 comments

ErikTMA commented

Describe the bug
We use ArgoCD to deploy applications which have the destination secrets. If a secret is patched through Argo, it is not being updated by the replicator.
ie:
We install a new application - secret is correctly updated by the replicator.
We make an update to the app causing ArgoCD to reconcile, which patches the secret. The secret is not being updated by the replicator.

To Reproduce
Update any destination secret in a pull configuration. ie kubectl apply -f secret.yaml

Expected behavior
I expect the secret to be updated whenever it is patched.

Environment:

  • Kubernetes version: 1.22.15-gke.100
  • kubernetes-replicator version: 2.7.3
eugenberend commented

@ErikTMA hi, what is your secret type?
I can confirm this issue with kubernetes.io/tls secret type.

ErikTMA commented

@ErikTMA hi, what is your secret type? I can confirm this issue with kubernetes.io/tls secret type.

It's a tls secret as well.

yevgeniyo commented

Seeing the same problem. We are using external secrets for accessing ECR (ECR token TTL is about 12 hours), it means that we have to regenerate token every few hours. We wanted to copy secret with kubernetes-replicator, but we are seeing now that if original secret was updated, cloned ones are not.

strowi commented

Same here, deploying a secret:

apiVersion: v1
data:
  .dockerconfigjson:  e30K
kind: Secret
metadata:
  annotations:
    replicator.v1.mittwald.de/replicate-from: app/registry.gitlab.com
  name: registry.gitlab.com
  namespace: frontend-develop
type: kubernetes.io/dockerconfigjson

The first time works, and gets updated to:

apiVersion: v1
data:
  .dockerconfigjson:  ......
kind: Secret
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{".dockerconfigjson":"e30K"},"kind":"Secret","metadata":{"annotations":{"replicator.v1.mittwald.de/replicate-from":"app/registry.gitlab.com"},"name":"registry.gitlab.com","namespace":"frontend-develop"},"typ
e":"kubernetes.io/dockerconfigjson"}
    replicator.v1.mittwald.de/replicate-from: app/registry.gitlab.com
    replicator.v1.mittwald.de/replicated-at: "2024-07-19T14:34:27Z"
    replicator.v1.mittwald.de/replicated-from-version: "3717"
    replicator.v1.mittwald.de/replicated-keys: .dockerconfigjson
  creationTimestamp: "2024-07-18T09:54:35Z"
  name: registry.gitlab.com
  namespace: frontend-develop
  resourceVersion: "1108217"
  uid: 841a62f5-ad03-4472-8dae-63b34cb630d1
type: kubernetes.io/dockerconfigjson

The re-applying it show the first one with some annotations:

apiVersion: v1
data:
  .dockerconfigjson: e30K
kind: Secret
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{".dockerconfigjson":"e30K"},"kind":"Secret","metadata":{"annotations":{"replicator.v1.mittwald.de/replicate-from":"app/registry.gitlab.com"},"name":"registry.gitlab.com","namespace":"frontend-develop"},"type":"kubernetes.io/dockerconfigjson"}
    replicator.v1.mittwald.de/replicate-from: app/registry.gitlab.com
    replicator.v1.mittwald.de/replicated-at: "2024-07-19T14:34:27Z"
    replicator.v1.mittwald.de/replicated-from-version: "3717"
    replicator.v1.mittwald.de/replicated-keys: .dockerconfigjson
  creationTimestamp: "2024-07-18T09:54:35Z"
  name: registry.gitlab.com
  namespace: frontend-develop
  resourceVersion: "1108756"
  uid: 841a62f5-ad03-4472-8dae-63b34cb630d1
type: kubernetes.io/dockerconfigjson

Removing the version-annotation, it gets updated correctly again.

Not sure how kubernetes-replicator exactly works, but this might be related to a kubectl's that won't update fields missing the patch..

PS: I might have found a workaround - adding an annotation replicator.v1.mittwald.de/replicated-from-version: '' to the secret on the pull-end will update the secret each time.