vCloud director CSI controller

CSI for vCloud director, it uses independent disks for volume and attach/detach it.

inspired by https://github.com/flant/yandex-csi-driver

quick start

ensure that kubectl get nodes and vm name at VDC is the same
label node for each vdc

kubectl label node nod1  failure-domain.beta.kubernetes.io/zone=some-vdc

create secret for access cloud - you can find example at deploy/secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: vcd-csi-credentials
  namespace: kube-system
stringData:
  USER: some-user
  PASSWOR: some-password
  ORG: Org-Name
  VDCS: some-vdc
  HREF: https://vcd.cloud/api

create rbac
```
kubectl apply -f deploy/rbac.yaml
```
create controller

 kubectl apply -f deploy/controller.yaml

create storage class

---
apiVersion: storage.k8s.io/v1beta1
kind: StorageClass
metadata:
 name: base-hdd
 annotations:
   storageclass.kubernetes.io/is-default-class: "true"
provisioner: vcd.csi.fght.net
parameters:
 vcd: some-vdc
 storageProfile: hdd
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

edit node daemon set deploy/ds.yaml

      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: "topology.kubernetes.io/zone"
                    operator: In
                    values:
                      - "some-vdc"

and for csi-node

            - name: VCSI_NODEVDC
              value: some-vdc

settings

csi controller uses env vars for config, you can find it at vars.MD
parameter for storage profile must be definened at storage class creation

limitations

for disk resize you must edit pvc, then delete pod from node, disk will be detached and resized
vm disk controllers cannot containt attached disk for units 8-15, it reserved for CSI
so maximum attached disk per vm is 8
tested at ubuntu 16 with Paravirtual SCSI only
computer name at vdc vapp MUST match k8s node name
permissions - csi wants admin permissions at vdc
for multiple vdcs you have to provide it as list VCSI_VDCS=vcd-1,vcd-2 at secret and tag node + vdc name topology tag
zone-aware - VDC name so common practice to run different daemonsets for each VDC with node affinity

requirements

Docker mount propagtion = shared
kubernetes 14+, --allow-privileged
feature gate KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true

known problems

Its hard to link attacheted to VM disk, currently there is filter:

list matched by unit number for PCI: bus, it must contain unit number from vm setting so unit numbers from 8 to 15 reserved for all controllers.

But there can be collisions, so whats the solution? use different vm for disk or change disk size..

Unfortunately Bus number at VM settings isnt reliable. Looking for best solution.

developing

you need go1.13+
vcd connections for integration tests

todo

leader election and clustering for controller
volume expand (seems like we need distributed lock)
more test

testing

make test

https://github.com/kubernetes-csi/csi-test/blob/master/pkg/sanity/README.md

f41gh7/vcd-csi