If a secret is deleted with a failed last sync, metric still exists

[piyat]

Hi - thanks for exposing so many useful metrics out of the box! I just wanted to check my understanding of something I've observed and see if there's a way around it.

I made a prometheus rule to send an alert whenever an external secret had a failed last sync with the below:

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: external-secrets-prometheus-rules
  namespace: external-secrets
spec:
  groups:
  - name: ExternalSecretsRules
    rules:
    - alert: ExternalSecretSyncFailure
      annotations:
        message: "Last Sync of {{`{{ $labels.name }}`}} in namespace {{`{{ $labels.exported_namespace }}`}} failed\n"
      expr: sum((kubernetes_external_secrets_last_sync_call_state)) by (name, exported_namespace) < 1
      labels:
        severity: warning

I made a dummy secret which would always fail (upstream secret doesn't exist), and I received the alert - all good.

I then deleted the dummy secret, but the alert still fires. It seems as though if there was never a successful sync after the most recent failed sync, this metric is always present.

Is there a way to avoid this?

[Flydiverny]

Hi piyat
please see

• #864

also duplicate of #399 😄

Don't expect this to be fixed :)