Stuck in "Still connecting to unix:///csi/csi.sock" loop

[sjinks]

What did you do? (required. The issue will be closed when not provided.)

Followed the installation instructions :-)

apiVersion: v1
kind: Secret
metadata:
  name: digitalocean
  namespace: kube-system
stringData:
  access-token: "2…d"

kubectl apply -fhttps://raw.githubusercontent.com/digitalocean/csi-digitalocean/master/deploy/kubernetes/releases/csi-digitalocean-v2.1.1/{crds.yaml,driver.yaml,snapshot-controller.yaml}

What did you expect to happen?

Expected kubectl get all -A to show no errors, but see this instead:

NAMESPACE              NAME                                             READY   STATUS             RESTARTS   AGE
kube-system            pod/csi-do-controller-0                          4/5     CrashLoopBackOff   21         58m
kube-system            pod/csi-do-node-r8tsb                            1/2     CrashLoopBackOff   18         58m

….

NAMESPACE       NAME                                            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
kube-system     daemonset.apps/csi-do-node                      1         1         0       1            0           <none>          58m

….

NAMESPACE     NAME                                   READY   AGE
kube-system   statefulset.apps/csi-do-controller     0/1     58m

Configuration (MUST fill this out):

• system logs:

Please provide the following logs:

kubectl cluster-info dump > kubernetes-dump.log

https://gist.github.com/sjinks/ea0ed3843dc30a46e5357f02f4d755d9

csi-provisioner / csi-attacher / csi-resizer / csi-snapshotter from csi-do-controller-0:

W0105 11:42:23.658271       1 connection.go:172] Still connecting to unix:///var/lib/csi/sockets/pluginproxy/csi.sock
…
W0105 11:58:53.658206       1 connection.go:172] Still connecting to unix:///var/lib/csi/sockets/pluginproxy/csi.sock

csi-do-plugin:

2021/01/05 11:57:50 couldn't get metadata: Get "http://169.254.169.254/metadata/v1.json": context deadline exceeded (Client.Timeout exceeded while awaiting headers) (are you running on DigitalOcean droplets?)

csi-node-driver-registrar from csi-do-node-r8tsb:

W0105 11:45:40.977092       1 connection.go:170] Still connecting to unix:///csi/csi.sock
…
W0105 12:02:10.977106       1 connection.go:170] Still connecting to unix:///csi/csi.sock

• manifests, such as pvc, deployments, etc.. you used to reproduce:

Please see above

kube-apiserver runs with --allow-privileged, --feature-gates=CSINodeInfo=true,CSIDriverRegistry=true.

Mount propagation is enabled:

# grep Mount /lib/systemd/system/docker.service
MountFlags=shared

• CSI Version: 2.1.1 (tried down to 2.0.0)

• Kubernetes Version: k3s version v1.20.0+k3s2 (2ea6b163); 1.20.0-37+7ab055a22f5440

• Cloud provider/framework version, if applicable (such as Rancher): Rancher; microk8s (the logs above are from rancher)