Issue with using feature HonorPVReclaimPolicy in csi-provisioner

[reenakabra]

What happened:

In my CSI controller I have enabled the feature gate HonorPVReclaimPolicy for csi-provisioner.

Based on the documentation
Finalizers can be added on a PersistentVolume to ensure that PersistentVolumes having Delete reclaim policy are deleted only after the backing storage are deleted.

The newly introduced finalizers kubernetes.io/pv-controller and external-provisioner.volume.kubernetes.io/finalizer are only added to dynamically provisioned volumes.

I can see the finalizer set on the PV while creating static Persistent Volume. PV deletion is stuck due to this finalizer

apiVersion: v1
kind: PersistentVolume
metadata:
creationTimestamp: "2023-11-08T11:39:24Z"
deletionGracePeriodSeconds: 0
deletionTimestamp: "2023-11-08T11:54:31Z"
finalizers:

external-provisioner.volume.kubernetes.io/finalizer
name: csi-infoscale-pv

What you expected to happen:

PV should be deleted when PVC is deleted.

How to reproduce it:

Persistent Volume yaml

apiVersion: v1
kind: PersistentVolume
metadata:
name: csi-infoscale-pv
spec:
storageClassName: csi-infoscale-sc-prod
persistentVolumeReclaimPolicy: Delete
capacity:
storage: 2Gi
accessModes:

• ReadWriteOnce
  csi:
  driver: org.veritas.infoscale

Please provide pre-provisioned Infoscale volume name.

volumeHandle: clust_21432/vrts_kube_dg-21432/vol1
fsType: vxfs

Persistent Volume Claim

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: csi-infoscale-pvc-static
spec:
accessModes:

ReadWriteOnce
resources:
requests:
storage: 2Gi
storageClassName: csi-infoscale-sc-prod
volumeName: csi-infoscale-pv

Anything else we need to know?:

Same thing works fine on OCP.

Environment:

• Driver version:
• Kubernetes version (use kubectl version):
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others:

WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.5", GitCommit:"93e0d7146fb9c3e9f68aa41b2b4265b2fcdb0a4c", GitTreeState:"clean", BuildDate:"2023-08-24T00:48:26Z", GoVersion:"go1.20.7", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.5", GitCommit:"93e0d7146fb9c3e9f68aa41b2b4265b2fcdb0a4c", GitTreeState:"clean", BuildDate:"2023-08-24T00:42:11Z", GoVersion:"go1.20.7", Compiler:"gc", Platform:"linux/amd64"}

[xing-yang]

This feature is alpha so the feature gate is not enabled by default. Did you enable the feature gate HonorPVReclaimPolicy on kube-controller-manager?

[reenakabra]

Yes, I have enabled this feature.

[xing-yang]

cc @deepakkinni