airflow-helm/charts

Change in metadataSecretName does not redeploy scheduler

chillinger opened this issue · 2 comments

chillinger commented

Checks

Chart Version

1.11.0

Kubernetes Version

Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.9", GitCommit:"d15213f69952c79b317e635abff6ff4ec81475f8", GitTreeState:"archive", BuildDate:"2023-12-20T00:00:00Z", GoVersion:"go1.21.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"28", GitVersion:"v1.28.6-gke.1289000", GitCommit:"a53df8308e54c8cea2e8cf44be1842106e93d04f", GitTreeState:"clean", BuildDate:"2024-01-26T09:25:53Z", GoVersion:"go1.20.13 X:boringcrypto", Compiler:"gc", Platform:"linux/amd64"}

Helm Version

3.11.1

Description

I had a working Airflow deployment, connected to a CloudSQL Instance.
I then created a new CloudSQL Instance, restored a Backup into this new instance and changed the Kubernetes Secret with the Connection string to point to this new Instance.
The webserver correctly picked up the new secret, but the scheduler was not restarted with the new config. In the Webserver under Admin -> Configs I could see the new DB Connection being present.

Only after manually deleting the scheduler pod and re-applying the helm chart the scheduler was re-created and started with the correct DB connection.

Relevant Logs

No response

Custom Helm Values

airflow:
  executor: KubernetesExecutor
  ingress:
    web:
      enabled: True
      ingressClassName: nginx
      hosts:
        - name: airflow.OUR.TLD
          tls:
            enabled: true
            secretName: tls-cert
      annotations:
        cert-manager.io/cluster-issuer: letsencrypt
  postgresql:
    enabled: false
  data:
    metadataSecretName: connection-string
  networkPolicies:
    enabled: True
  config:
    scheduler:
      min_file_process_interval: 10
    webserver:
      expose_config: True
    providers.jdbc:
      allow_driver_class_in_extra: True
      allow_driver_path_in_extra: True
  extraEnv: |
    - name: AIRFLOW__CORE__LOAD_EXAMPLES
      value: 'True'
chillinger commented

Update: the triggerer Pod is also not redeployed