piraeusdatastore/helm-charts

snapshot-controller: caBundle field changes on every `helm upgrade` or `helm diff`

gclawes opened this issue · 2 comments

gclawes commented

The caBundle field introduced in 2.0.0 for snapshot-validation-webhook changes on every helm diff or helm upgrade. This causes unnecessary deploys with continuous reconcilliation gitops tools and drift detection workflows.

Full `helm diff` (click to expand) 
kube-system, snapshot-validation-webhook, ValidatingWebhookConfiguration (admissionregistration.k8s.io) has changed:
  # Source: snapshot-controller/templates/webhook.yaml
  apiVersion: admissionregistration.k8s.io/v1
  kind: ValidatingWebhookConfiguration
  metadata:
    name: snapshot-validation-webhook
    labels:
      helm.sh/chart: snapshot-controller-2.0.0
      app.kubernetes.io/name: snapshot-validation-webhook
      app.kubernetes.io/instance: snapshot-controller
      app.kubernetes.io/version: "v6.3.1"
      app.kubernetes.io/managed-by: Helm
  webhooks:
    - name: snapshot-validation-webhook.snapshot.storage.k8s.io
      rules:
        - apiGroups:
          - snapshot.storage.k8s.io
          apiVersions:
          - v1
          - v1beta1
          operations:
          - CREATE
          - UPDATE
          resources:
          - volumesnapshots
          - volumesnapshotclasses
          - volumesnapshotcontents
          scope: "*"
      clientConfig:
        service:
          namespace: kube-system
          name: snapshot-validation-webhook
          path: "/volumesnapshot"
-       caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURiVENDQWxXZ0F3SUJBZ0lRQVVYZTZRcEhYK041Q0hIYTBCdGNBakFOQmdrcWhraUc5dzBCQVFzRkFEQTIKTVRRd01nWURWUVFERXl0emJtRndjMmh2ZEMxMllXeHBaR0YwYVc5dUxYZGxZbWh2YjJzdWEzVmlaUzF6ZVhOMApaVzB1YzNaak1CNFhEVEl6TVRBeU9URTNNRGN3TlZvWERUTXpNVEF5TmpFM01EY3dOVm93TmpFME1ESUdBMVVFCkF4TXJjMjVoY0hOb2IzUXRkbUZzYVdSaGRHbHZiaTEzWldKb2IyOXJMbXQxWW1VdGMzbHpkR1Z0TG5OMll6Q0MKQVNJd0RRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFKZkswUncyc3ZhckhqaStMYzJ4VzBiNwpsdHdkSDlBUWlwRndxQXdySUljaGJjM3BoRkhUVnJHTVhGQXlUQWxJdXl1MlQvb0w5QmxtNmt4ZTBNdlpoRCtiCjBYaUx0YVNHY1lvOUJwRn
BCbi9WQWhreDB1QzM3WDNrZzFvUmthS09ZMUZGb0l1MDRPL2FCQk5PV3pVb3Y2ekEKRGYyK1ZYTkV4NnhDY0tmYVlObU5mWXlYeXBlQTJkREM2MXRHN3hCaE9JNTdNUEFMQmRpUklOYUJkcjNqdjhpcgpzMXhBMlY1NURSRElzcXp6N2diNUJzSUwvR1Zwdjh0Z1VWc0hQMGdSNVJWZ0g5ZjFSQzlPVzBKd0REUzRVSFlQClZFam5hRGUzSDJOMWJRT0ZwN0VHYXpjaTUydXJoOFJHdUJ2VmpvSjRWUnR6akZWaE5vQzhxbk5SLzJxbDU1RUMKQXdFQUFhTjNNSFV3RGdZRFZSMFBBUUgvQkFRREFnV2dNQjBHQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01CQmdncgpCZ0VGQlFjREFqQU1CZ05WSFJNQkFmOEVBakFBTURZR0ExVWRFUVF2TUMyQ0szTnVZWEJ6YUc5MExYWmhiR2xrCllYUnBiMjR0ZDJWaWFHOXZheTVyZFdKbExYTjVjM1JsYlM1emRtTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUIKQUdVMlgycX
pKZHVDU0JRUCtuOFpmUGt2ZW9qc3ZQWWV1dEFWdXlnYVMvRGJobzhoN1gzTlNmSkJuRUl1TWFaYwpqcjJ2bFZwTU11U2tScncyKzBXKzhEeHBieUZrUVhTNm1jMUV5aS9lOGZkTUFlV25DZ2hxRDAzYU5CRE5ienBHClROYmliNHBESDQrZi82Q3B4eWVXMkJqODlHb0tLNTIrR1NkRGFSSUJXbTVYQzIrUXdpZ2FLVHNZTTlIRmdqUkoKUXlNMkVrQU5vbXkrdm93Y0RuSG0veFJSbHlXTU5VSVo1cmc1cTZrODNab2UxWjZDVE0zNFJENGhoQklJMHkrRApzN0NGRCtBdXRNSWxSRE4rcGhkZEl5b0dSRk5mQnp4dDlVdmx1OWthRXhqQVQ5a0d6cFZYdFhXeVhobkhURlNWCmZTSXNORXgxSXlPME54ZGpOTktQNlRvPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
+       caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURiakNDQWxhZ0F3SUJBZ0lSQUlnbzlDd2huZzZ1NVNpT0FhL1MyMlV3RFFZSktvWklodmNOQVFFTEJRQXcKTmpFME1ESUdBMVVFQXhNcmMyNWhjSE5vYjNRdGRtRnNhV1JoZEdsdmJpMTNaV0pvYjI5ckxtdDFZbVV0YzNsegpkR1Z0TG5OMll6QWVGdzB5TXpFd01qa3hOelUzTURGYUZ3MHpNekV3TWpZeE56VTNNREZhTURZeE5EQXlCZ05WCkJBTVRLM051WVhCemFHOTBMWFpoYkdsa1lYUnBiMjR0ZDJWaWFHOXZheTVyZFdKbExYTjVjM1JsYlM1emRtTXcKZ2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLQW9JQkFRQzlBd0dkbFYxUjV2aW05ZWQ5NVgxRgpxMTVnUzE4U1FPN0JUZ1VLVDVTNDVqMi9pVG53aFpHV1Avdm5SaVdJVVYyOXBaTS9GSzBNZXQzeG0rZG1POXhUCndxNEZUYkNnK29KdE9pZE
VMOHBIWVVCYm5HVExER3krajVnQmNBeXBhS0tTRmtRUEZ1eVZweXQ3b1pvb1IvcHYKdEFiNXpHelNwL2tpeXArU2RkWklqUGxVdDJNV3Q1VkxMcDMzdFEraXhMWGhucVlUaUxLZE9Ea0h3dVZUMyt5TgpmbXczU2hSRi80UzZzdHZ0RnVjMHl0cXI5UmxJalhnelh0M3pmN2JlMWRYaElOeUlpVkpYSGVoN2I0S1hVd3FKCjZpd1p6ai9yVk5xd2pKWW1NaXNsK0x0REhOWGVwbkRVZ0dEOWF6K09NWklHWDYrN0lEeVhvQXNNQkFqUjEyUkwKQWdNQkFBR2pkekIxTUE0R0ExVWREd0VCL3dRRUF3SUZvREFkQmdOVkhTVUVGakFVQmdnckJnRUZCUWNEQVFZSQpLd1lCQlFVSEF3SXdEQVlEVlIwVEFRSC9CQUl3QURBMkJnTlZIUkVFTHpBdGdpdHpibUZ3YzJodmRDMTJZV3hwClpHRjBhVzl1TFhkbFltaHZiMnN1YTNWaVpTMXplWE4wWlcwdWMzWmpNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUIKQVFBZjBTUG
5ET0dVUVVQY3JPcmRwVjNCR0VPakt3QjFPdjRpL0RNNUJOcFRUN2JOL29NcmFtSXczK3JKeERNNApwTkViWFdwTWwrd2VvcVRKM05yNVJ4azVESVBBV0RJbTQzQlpvejIxcW95SlVMQ3RlRTF6aHhocm1rcjRjb2IzCm1hS3ZReHdZK1VuK01QM2dFSDNuT0dEVFNMNFpicThPSHpZd3FSQklMKzFIc3lLSThocUNuYUlEcUdlK0lYbE0KWWloSmJjNEdLcW4yaHFiaGpSblh6WjE2eDhpZjhlcWZycDJoQjlmT0U0SW5yRjJuVlVGbG0xWTUvZGlBTXRpcQpCR1JjeURlSWFIZWpQdUV2VWdZQWJTWlhreUZEUnltREtHbUNwNUt6V1JibVd4bnozWFcyb3VweDdaS1pJNm1rCnh1MnMxeTNROEp2VURKT2lCRktvdytxUgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
      admissionReviewVersions:
        - v1
        - v1beta1
      sideEffects: None
      failurePolicy: Fail
      timeoutSeconds: 2
    - name: snapshot-validation-webhook.groupsnapshot.storage.k8s.io
      rules:
        - apiGroups:
            - groupsnapshot.storage.k8s.io
          apiVersions:
            - v1alpha1
          operations:
            - CREATE
            - UPDATE
          resources:
            - volumegroupsnapshots
            - volumegroupsnapshotcontents
            - volumegroupsnapshotclasses
          scope: "*"
      clientConfig:
        service:
          namespace: kube-system
          name: snapshot-validation-webhook
          path: "/volumegroupsnapshot"
-       caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURiVENDQWxXZ0F3SUJBZ0lRQVVYZTZRcEhYK041Q0hIYTBCdGNBakFOQmdrcWhraUc5dzBCQVFzRkFEQTIKTVRRd01nWURWUVFERXl0emJtRndjMmh2ZEMxMllXeHBaR0YwYVc5dUxYZGxZbWh2YjJzdWEzVmlaUzF6ZVhOMApaVzB1YzNaak1CNFhEVEl6TVRBeU9URTNNRGN3TlZvWERUTXpNVEF5TmpFM01EY3dOVm93TmpFME1ESUdBMVVFCkF4TXJjMjVoY0hOb2IzUXRkbUZzYVdSaGRHbHZiaTEzWldKb2IyOXJMbXQxWW1VdGMzbHpkR1Z0TG5OMll6Q0MKQVNJd0RRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFKZkswUncyc3ZhckhqaStMYzJ4VzBiNwpsdHdkSDlBUWlwRndxQXdySUljaGJjM3BoRkhUVnJHTVhGQXlUQWxJdXl1MlQvb0w5QmxtNmt4ZTBNdlpoRCtiCjBYaUx0YVNHY1lvOUJwRn
BCbi9WQWhreDB1QzM3WDNrZzFvUmthS09ZMUZGb0l1MDRPL2FCQk5PV3pVb3Y2ekEKRGYyK1ZYTkV4NnhDY0tmYVlObU5mWXlYeXBlQTJkREM2MXRHN3hCaE9JNTdNUEFMQmRpUklOYUJkcjNqdjhpcgpzMXhBMlY1NURSRElzcXp6N2diNUJzSUwvR1Zwdjh0Z1VWc0hQMGdSNVJWZ0g5ZjFSQzlPVzBKd0REUzRVSFlQClZFam5hRGUzSDJOMWJRT0ZwN0VHYXpjaTUydXJoOFJHdUJ2VmpvSjRWUnR6akZWaE5vQzhxbk5SLzJxbDU1RUMKQXdFQUFhTjNNSFV3RGdZRFZSMFBBUUgvQkFRREFnV2dNQjBHQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01CQmdncgpCZ0VGQlFjREFqQU1CZ05WSFJNQkFmOEVBakFBTURZR0ExVWRFUVF2TUMyQ0szTnVZWEJ6YUc5MExYWmhiR2xrCllYUnBiMjR0ZDJWaWFHOXZheTVyZFdKbExYTjVjM1JsYlM1emRtTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUIKQUdVMlgycX
pKZHVDU0JRUCtuOFpmUGt2ZW9qc3ZQWWV1dEFWdXlnYVMvRGJobzhoN1gzTlNmSkJuRUl1TWFaYwpqcjJ2bFZwTU11U2tScncyKzBXKzhEeHBieUZrUVhTNm1jMUV5aS9lOGZkTUFlV25DZ2hxRDAzYU5CRE5ienBHClROYmliNHBESDQrZi82Q3B4eWVXMkJqODlHb0tLNTIrR1NkRGFSSUJXbTVYQzIrUXdpZ2FLVHNZTTlIRmdqUkoKUXlNMkVrQU5vbXkrdm93Y0RuSG0veFJSbHlXTU5VSVo1cmc1cTZrODNab2UxWjZDVE0zNFJENGhoQklJMHkrRApzN0NGRCtBdXRNSWxSRE4rcGhkZEl5b0dSRk5mQnp4dDlVdmx1OWthRXhqQVQ5a0d6cFZYdFhXeVhobkhURlNWCmZTSXNORXgxSXlPME54ZGpOTktQNlRvPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
+       caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURiakNDQWxhZ0F3SUJBZ0lSQUlnbzlDd2huZzZ1NVNpT0FhL1MyMlV3RFFZSktvWklodmNOQVFFTEJRQXcKTmpFME1ESUdBMVVFQXhNcmMyNWhjSE5vYjNRdGRtRnNhV1JoZEdsdmJpMTNaV0pvYjI5ckxtdDFZbVV0YzNsegpkR1Z0TG5OMll6QWVGdzB5TXpFd01qa3hOelUzTURGYUZ3MHpNekV3TWpZeE56VTNNREZhTURZeE5EQXlCZ05WCkJBTVRLM051WVhCemFHOTBMWFpoYkdsa1lYUnBiMjR0ZDJWaWFHOXZheTVyZFdKbExYTjVjM1JsYlM1emRtTXcKZ2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLQW9JQkFRQzlBd0dkbFYxUjV2aW05ZWQ5NVgxRgpxMTVnUzE4U1FPN0JUZ1VLVDVTNDVqMi9pVG53aFpHV1Avdm5SaVdJVVYyOXBaTS9GSzBNZXQzeG0rZG1POXhUCndxNEZUYkNnK29KdE9pZE
VMOHBIWVVCYm5HVExER3krajVnQmNBeXBhS0tTRmtRUEZ1eVZweXQ3b1pvb1IvcHYKdEFiNXpHelNwL2tpeXArU2RkWklqUGxVdDJNV3Q1VkxMcDMzdFEraXhMWGhucVlUaUxLZE9Ea0h3dVZUMyt5TgpmbXczU2hSRi80UzZzdHZ0RnVjMHl0cXI5UmxJalhnelh0M3pmN2JlMWRYaElOeUlpVkpYSGVoN2I0S1hVd3FKCjZpd1p6ai9yVk5xd2pKWW1NaXNsK0x0REhOWGVwbkRVZ0dEOWF6K09NWklHWDYrN0lEeVhvQXNNQkFqUjEyUkwKQWdNQkFBR2pkekIxTUE0R0ExVWREd0VCL3dRRUF3SUZvREFkQmdOVkhTVUVGakFVQmdnckJnRUZCUWNEQVFZSQpLd1lCQlFVSEF3SXdEQVlEVlIwVEFRSC9CQUl3QURBMkJnTlZIUkVFTHpBdGdpdHpibUZ3YzJodmRDMTJZV3hwClpHRjBhVzl1TFhkbFltaHZiMnN1YTNWaVpTMXplWE4wWlcwdWMzWmpNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUIKQVFBZjBTUG
5ET0dVUVVQY3JPcmRwVjNCR0VPakt3QjFPdjRpL0RNNUJOcFRUN2JOL29NcmFtSXczK3JKeERNNApwTkViWFdwTWwrd2VvcVRKM05yNVJ4azVESVBBV0RJbTQzQlpvejIxcW95SlVMQ3RlRTF6aHhocm1rcjRjb2IzCm1hS3ZReHdZK1VuK01QM2dFSDNuT0dEVFNMNFpicThPSHpZd3FSQklMKzFIc3lLSThocUNuYUlEcUdlK0lYbE0KWWloSmJjNEdLcW4yaHFiaGpSblh6WjE2eDhpZjhlcWZycDJoQjlmT0U0SW5yRjJuVlVGbG0xWTUvZGlBTXRpcQpCR1JjeURlSWFIZWpQdUV2VWdZQWJTWlhreUZEUnltREtHbUNwNUt6V1JibVd4bnozWFcyb3VweDdaS1pJNm1rCnh1MnMxeTNROEp2VURKT2lCRktvdytxUgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
      admissionReviewVersions:
        - v1
        - v1beta1
      sideEffects: None
      failurePolicy: Fail
      timeoutSeconds: 2
kube-system, snapshot-validation-webhook-tls, Secret (v1) has changed:
+ Changes suppressed on sensitive content of type Secret

Other charts like ingress-nginx with validating webhooks use a patch Job instead of encoding the CA in the helm template: https://github.com/kubernetes/ingress-nginx/tree/main/charts/ingress-nginx/templates/admission-webhooks. This avoids diffs on subsequent helm runs.

WanzenBug commented

Looks like a great feature to have. Patches welcome :)

phoenix-bjoern commented

@gclawes The recommended "fix" is to use the cert-manager instead to manager the certificate, see https://github.com/piraeusdatastore/helm-charts/tree/main/charts/snapshot-controller#snapshot-validation-webhook.