Fresh install and facing error DirectPV not installed
darioneto opened this issue · 10 comments
darioneto commented
I'm following the first install of this guide https://min.io/directpv
➜ ~ kubectl directpv install
███████████████████████████████████████████████████████████████████████████ 100%
┌──────────────────────────────────────┬──────────────────────────┐
│ NAME │ KIND │
├──────────────────────────────────────┼──────────────────────────┤
│ directpv │ Namespace │
│ directpv-min-io │ ServiceAccount │
│ directpv-min-io │ ClusterRole │
│ directpv-min-io │ ClusterRoleBinding │
│ directpv-min-io │ Role │
│ directpv-min-io │ RoleBinding │
│ directpvdrives.directpv.min.io │ CustomResourceDefinition │
│ directpvvolumes.directpv.min.io │ CustomResourceDefinition │
│ directpvnodes.directpv.min.io │ CustomResourceDefinition │
│ directpvinitrequests.directpv.min.io │ CustomResourceDefinition │
│ directpv-min-io │ CSIDriver │
│ directpv-min-io │ StorageClass │
│ node-server │ Daemonset │
│ controller │ Deployment │
└──────────────────────────────────────┴──────────────────────────┘
DirectPV installed successfully
➜ ~ kubectl directpv info
ERROR DirectPV not installed
➜ ~ kubectl directpv --version
directpv version v4.0.10
➜ ~
Client Version: v1.26.6
Kustomize Version: v4.5.7
Server Version: v1.26.3+k0s
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
- Kernel version:
- 5.15.0-91-generic
As far as all the pods and services are operational
➜ ~ k get all -n directpv
NAME READY STATUS RESTARTS AGE
pod/controller-867f68dfbd-8hz94 3/3 Running 0 19h
pod/controller-867f68dfbd-mw9dj 3/3 Running 0 19h
pod/controller-867f68dfbd-pdhhh 3/3 Running 0 19h
pod/node-server-4rtxd 4/4 Running 0 19h
pod/node-server-8s8qg 4/4 Running 0 19h
pod/node-server-9g6f5 4/4 Running 0 19h
pod/node-server-9zxgv 4/4 Running 0 19h
pod/node-server-b94hk 4/4 Running 0 19h
pod/node-server-cffjr 4/4 Running 0 19h
pod/node-server-d4qj2 4/4 Running 0 19h
pod/node-server-mf2kl 4/4 Running 0 19h
pod/node-server-s2k8k 4/4 Running 0 19h
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/node-server 9 9 9 9 9 19h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 3/3 3 3 19h
NAME DESIRED CURRENT READY AGE
replicaset.apps/controller-867f68dfbd 3 3 3 19h
➜ ~
balamurugana commented
Please share all controller logs and all node-server logs
darioneto commented
Here are the logs
ansible@lb:~$ k logs controller-867f68dfbd-8hz94 -n directpv
Defaulted container "csi-provisioner" out of: csi-provisioner, csi-resizer, controller
W0101 13:46:53.488394 1 feature_gate.go:241] Setting GA feature gate Topology=true. It will be removed in a future release.
I0101 13:46:53.489040 1 feature_gate.go:249] feature gates: &{map[Topology:true]}
I0101 13:46:53.489252 1 csi-provisioner.go:154] Version: v3.5.0
I0101 13:46:53.489260 1 csi-provisioner.go:177] Building kube configs for running in cluster...
W0101 13:47:03.494250 1 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:13.216981 1 common.go:111] Probing CSI driver for readiness
I0101 13:47:13.219819 1 csi-provisioner.go:230] Detected CSI driver directpv-min-io
I0101 13:47:13.221149 1 csi-provisioner.go:302] CSI driver does not support PUBLISH_UNPUBLISH_VOLUME, not watching VolumeAttachments
I0101 13:47:13.221631 1 controller.go:732] Using saving PVs to API server in background
I0101 13:47:13.221963 1 leaderelection.go:245] attempting to acquire leader lease directpv/directpv-min-io...
I0101 13:47:13.237253 1 leader_election.go:185] new leader detected, current leader: 1704116829207-794-directpv-min-io
ansible@lb:~$ k logs controller-867f68dfbd-8hz94 -c controller -n directpv
I0101 13:47:12.317471 1 controller.go:57] Identity server started
I0101 13:47:12.317546 1 controller.go:60] Controller server started
I0101 13:47:12.317786 1 ready.go:42] Serving readiness endpoint at :30443
ansible@lb:~$ k logs -n directpv controller-867f68dfbd-mw9dj
Defaulted container "csi-provisioner" out of: csi-provisioner, csi-resizer, controller
W0101 13:47:01.525932 1 feature_gate.go:241] Setting GA feature gate Topology=true. It will be removed in a future release.
I0101 13:47:01.526023 1 feature_gate.go:249] feature gates: &{map[Topology:true]}
I0101 13:47:01.526041 1 csi-provisioner.go:154] Version: v3.5.0
I0101 13:47:01.526049 1 csi-provisioner.go:177] Building kube configs for running in cluster...
W0101 13:47:11.526786 1 connection.go:183] Still connecting to unix:///csi/csi.sock
W0101 13:47:21.526790 1 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:21.927128 1 common.go:111] Probing CSI driver for readiness
I0101 13:47:21.934112 1 csi-provisioner.go:230] Detected CSI driver directpv-min-io
I0101 13:47:21.936156 1 csi-provisioner.go:302] CSI driver does not support PUBLISH_UNPUBLISH_VOLUME, not watching VolumeAttachments
I0101 13:47:21.936780 1 controller.go:732] Using saving PVs to API server in background
I0101 13:47:21.937164 1 leaderelection.go:245] attempting to acquire leader lease directpv/directpv-min-io...
I0101 13:47:21.950122 1 leader_election.go:185] new leader detected, current leader: 1704116829207-794-directpv-min-io
ansible@lb:~$
ansible@lb:~$ k logs -n directpv controller-867f68dfbd-mw9dj -c controller
I0101 13:47:21.852198 1 controller.go:57] Identity server started
I0101 13:47:21.852290 1 controller.go:60] Controller server started
I0101 13:47:21.853138 1 ready.go:42] Serving readiness endpoint at :30443
ansible@lb:~$ k logs -n directpv controller-867f68dfbd-pdhhh
Defaulted container "csi-provisioner" out of: csi-provisioner, csi-resizer, controller
W0101 13:46:53.260595 1 feature_gate.go:241] Setting GA feature gate Topology=true. It will be removed in a future release.
I0101 13:46:53.260727 1 feature_gate.go:249] feature gates: &{map[Topology:true]}
I0101 13:46:53.261262 1 csi-provisioner.go:154] Version: v3.5.0
I0101 13:46:53.261282 1 csi-provisioner.go:177] Building kube configs for running in cluster...
W0101 13:47:03.271148 1 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:09.200216 1 common.go:111] Probing CSI driver for readiness
I0101 13:47:09.206018 1 csi-provisioner.go:230] Detected CSI driver directpv-min-io
I0101 13:47:09.208849 1 csi-provisioner.go:302] CSI driver does not support PUBLISH_UNPUBLISH_VOLUME, not watching VolumeAttachments
I0101 13:47:09.209556 1 controller.go:732] Using saving PVs to API server in background
I0101 13:47:09.210079 1 leaderelection.go:245] attempting to acquire leader lease directpv/directpv-min-io...
I0101 13:47:09.262074 1 leaderelection.go:255] successfully acquired lease directpv/directpv-min-io
I0101 13:47:09.262166 1 leader_election.go:185] new leader detected, current leader: 1704116829207-794-directpv-min-io
I0101 13:47:09.262289 1 leader_election.go:178] became leader, starting
I0101 13:47:09.263693 1 reflector.go:287] Starting reflector *v1.StorageClass (1h0m0s) from k8s.io/client-go/informers/factory.go:150
I0101 13:47:09.263724 1 reflector.go:323] Listing and watching *v1.StorageClass from k8s.io/client-go/informers/factory.go:150
I0101 13:47:09.263683 1 reflector.go:287] Starting reflector *v1.CSINode (1h0m0s) from k8s.io/client-go/informers/factory.go:150
I0101 13:47:09.263709 1 reflector.go:287] Starting reflector *v1.Node (1h0m0s) from k8s.io/client-go/informers/factory.go:150
I0101 13:47:09.263737 1 reflector.go:323] Listing and watching *v1.CSINode from k8s.io/client-go/informers/factory.go:150
I0101 13:47:09.263752 1 reflector.go:323] Listing and watching *v1.Node from k8s.io/client-go/informers/factory.go:150
I0101 13:47:09.263778 1 reflector.go:287] Starting reflector *v1.PersistentVolumeClaim (15m0s) from k8s.io/client-go/informers/factory.go:150
I0101 13:47:09.263820 1 reflector.go:323] Listing and watching *v1.PersistentVolumeClaim from k8s.io/client-go/informers/factory.go:150
I0101 13:47:09.363202 1 volume_store.go:97] Starting save volume queue
I0101 13:47:09.363162 1 controller.go:811] Starting provisioner controller directpv-min-io_controller-867f68dfbd-pdhhh_cdf8c24d-07a2-4734-8b14-e0b93ab2f725!
I0101 13:47:09.363594 1 reflector.go:287] Starting reflector *v1.PersistentVolume (15m0s) from sigs.k8s.io/sig-storage-lib-external-provisioner/v9/controller/controller.go:845
I0101 13:47:09.363623 1 reflector.go:323] Listing and watching *v1.PersistentVolume from sigs.k8s.io/sig-storage-lib-external-provisioner/v9/controller/controller.go:845
I0101 13:47:09.363668 1 reflector.go:287] Starting reflector *v1.StorageClass (15m0s) from sigs.k8s.io/sig-storage-lib-external-provisioner/v9/controller/controller.go:848
I0101 13:47:09.363704 1 reflector.go:323] Listing and watching *v1.StorageClass from sigs.k8s.io/sig-storage-lib-external-provisioner/v9/controller/controller.go:848
I0101 13:47:09.464890 1 controller.go:860] Started provisioner controller directpv-min-io_controller-867f68dfbd-pdhhh_cdf8c24d-07a2-4734-8b14-e0b93ab2f725!
ansible@lb:~$
ansible@lb:~$ k logs -n directpv controller-867f68dfbd-pdhhh -c controller
I0101 13:47:08.834807 1 controller.go:57] Identity server started
I0101 13:47:08.834923 1 controller.go:60] Controller server started
I0101 13:47:08.835326 1 ready.go:42] Serving readiness endpoint at :30443
ansible@lb:~$ k logs -n directpv node-server-4rtxd
Defaulted container "node-driver-registrar" out of: node-driver-registrar, node-server, node-controller, liveness-probe
I0101 13:46:53.187377 1353264 main.go:167] Version: v2.8.0
I0101 13:46:53.187775 1353264 main.go:168] Running node-driver-registrar in mode=registration
I0101 13:46:53.189062 1353264 main.go:192] Attempting to open a gRPC connection with: "unix:///csi/csi.sock"
W0101 13:47:03.189607 1353264 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:08.098545 1353264 main.go:199] Calling CSI driver to discover driver name
I0101 13:47:08.102571 1353264 main.go:209] CSI driver name: "directpv-min-io"
I0101 13:47:08.102606 1353264 node_register.go:53] Starting Registration Server at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.102793 1353264 node_register.go:62] Registration Server started at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.103173 1353264 node_register.go:92] Skipping HTTP server because endpoint is set to: ""
ansible@lb:~$
ansible@lb:~$ k logs -n directpv node-server-4rtxd -c node-server
I0101 13:47:07.458810 1353523 node-server.go:78] Identity server started
I0101 13:47:07.458904 1353523 node-server.go:101] Node server started
I0101 13:47:07.459069 1353523 reflector.go:289] Starting reflector *v1beta1.DirectPVDrive (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.459145 1353523 reflector.go:325] Listing and watching *v1beta1.DirectPVDrive from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.459068 1353523 reflector.go:289] Starting reflector *v1beta1.DirectPVVolume (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.459211 1353523 reflector.go:325] Listing and watching *v1beta1.DirectPVVolume from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.459100 1353523 ready.go:42] Serving readiness endpoint at :30443
I0101 13:47:07.459229 1353523 metrics.go:63] Starting metrics exporter at port 10443
I0101 13:47:07.559664 1353523 controller.go:141] drive controller synced and ready
I0101 13:47:07.559675 1353523 controller.go:141] volume controller synced and ready
ansible@lb:~$ k logs -n directpv node-server-8s8qg
Defaulted container "node-driver-registrar" out of: node-driver-registrar, node-server, node-controller, liveness-probe
I0101 13:46:55.524378 1995636 main.go:167] Version: v2.8.0
I0101 13:46:55.524467 1995636 main.go:168] Running node-driver-registrar in mode=registration
I0101 13:46:55.526572 1995636 main.go:192] Attempting to open a gRPC connection with: "unix:///csi/csi.sock"
W0101 13:47:05.527400 1995636 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:08.121451 1995636 main.go:199] Calling CSI driver to discover driver name
I0101 13:47:08.130975 1995636 main.go:209] CSI driver name: "directpv-min-io"
I0101 13:47:08.131061 1995636 node_register.go:53] Starting Registration Server at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.131362 1995636 node_register.go:62] Registration Server started at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.131621 1995636 node_register.go:92] Skipping HTTP server because endpoint is set to: ""
ansible@lb:~$
ansible@lb:~$ k logs -n directpv node-server-8s8qg -c node-server
I0101 13:47:07.684174 1995893 node-server.go:78] Identity server started
I0101 13:47:07.684335 1995893 node-server.go:101] Node server started
I0101 13:47:07.684595 1995893 reflector.go:289] Starting reflector *v1beta1.DirectPVVolume (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.684621 1995893 reflector.go:325] Listing and watching *v1beta1.DirectPVVolume from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.684637 1995893 ready.go:42] Serving readiness endpoint at :30443
I0101 13:47:07.684659 1995893 reflector.go:289] Starting reflector *v1beta1.DirectPVDrive (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.684693 1995893 reflector.go:325] Listing and watching *v1beta1.DirectPVDrive from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.684871 1995893 metrics.go:63] Starting metrics exporter at port 10443
I0101 13:47:07.784674 1995893 controller.go:141] volume controller synced and ready
I0101 13:47:07.784896 1995893 controller.go:141] drive controller synced and ready
ansible@lb:~$ k logs -n directpv node-server-9g6f5
Defaulted container "node-driver-registrar" out of: node-driver-registrar, node-server, node-controller, liveness-probe
I0101 13:46:53.959194 3724845 main.go:167] Version: v2.8.0
I0101 13:46:53.959265 3724845 main.go:168] Running node-driver-registrar in mode=registration
I0101 13:46:53.959813 3724845 main.go:192] Attempting to open a gRPC connection with: "unix:///csi/csi.sock"
W0101 13:47:03.960131 3724845 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:08.341066 3724845 main.go:199] Calling CSI driver to discover driver name
I0101 13:47:08.344157 3724845 main.go:209] CSI driver name: "directpv-min-io"
I0101 13:47:08.344196 3724845 node_register.go:53] Starting Registration Server at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.344353 3724845 node_register.go:62] Registration Server started at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.345153 3724845 node_register.go:92] Skipping HTTP server because endpoint is set to: ""
ansible@lb:~$
ansible@lb:~$ k logs -n directpv node-server-9g6f5 -c node-server
I0101 13:47:07.433659 3724976 node-server.go:78] Identity server started
I0101 13:47:07.433726 3724976 node-server.go:101] Node server started
I0101 13:47:07.433866 3724976 reflector.go:289] Starting reflector *v1beta1.DirectPVDrive (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.433878 3724976 reflector.go:325] Listing and watching *v1beta1.DirectPVDrive from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.433871 3724976 ready.go:42] Serving readiness endpoint at :30443
I0101 13:47:07.433866 3724976 reflector.go:289] Starting reflector *v1beta1.DirectPVVolume (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.433972 3724976 reflector.go:325] Listing and watching *v1beta1.DirectPVVolume from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.434014 3724976 metrics.go:63] Starting metrics exporter at port 10443
I0101 13:47:07.534543 3724976 controller.go:141] drive controller synced and ready
I0101 13:47:07.534667 3724976 controller.go:141] volume controller synced and ready
ansible@lb:~$ k logs -n directpv node-server-9zxgv
Defaulted container "node-driver-registrar" out of: node-driver-registrar, node-server, node-controller, liveness-probe
I0101 13:46:55.722131 2148478 main.go:167] Version: v2.8.0
I0101 13:46:55.722163 2148478 main.go:168] Running node-driver-registrar in mode=registration
I0101 13:46:55.725163 2148478 main.go:192] Attempting to open a gRPC connection with: "unix:///csi/csi.sock"
W0101 13:47:05.726212 2148478 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:11.772328 2148478 main.go:199] Calling CSI driver to discover driver name
I0101 13:47:11.790654 2148478 main.go:209] CSI driver name: "directpv-min-io"
I0101 13:47:11.790718 2148478 node_register.go:53] Starting Registration Server at: /registration/directpv-min-io-reg.sock
I0101 13:47:11.791270 2148478 node_register.go:62] Registration Server started at: /registration/directpv-min-io-reg.sock
I0101 13:47:11.792521 2148478 node_register.go:92] Skipping HTTP server because endpoint is set to: ""
ansible@lb:~$
ansible@lb:~$ k logs -n directpv node-server-9zxgv -c node-server
I0101 13:47:11.057228 2148724 node-server.go:78] Identity server started
I0101 13:47:11.057317 2148724 node-server.go:101] Node server started
I0101 13:47:11.057404 2148724 reflector.go:289] Starting reflector *v1beta1.DirectPVVolume (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:11.057415 2148724 reflector.go:325] Listing and watching *v1beta1.DirectPVVolume from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:11.057414 2148724 reflector.go:289] Starting reflector *v1beta1.DirectPVDrive (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:11.057423 2148724 reflector.go:325] Listing and watching *v1beta1.DirectPVDrive from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:11.057463 2148724 ready.go:42] Serving readiness endpoint at :30443
I0101 13:47:11.057549 2148724 metrics.go:63] Starting metrics exporter at port 10443
I0101 13:47:11.158444 2148724 controller.go:141] volume controller synced and ready
I0101 13:47:11.158571 2148724 controller.go:141] drive controller synced and ready
ansible@lb:~$ k logs -n directpv node-server-b94hk
Defaulted container "node-driver-registrar" out of: node-driver-registrar, node-server, node-controller, liveness-probe
I0101 13:46:53.393016 1288131 main.go:167] Version: v2.8.0
I0101 13:46:53.393065 1288131 main.go:168] Running node-driver-registrar in mode=registration
I0101 13:46:53.395413 1288131 main.go:192] Attempting to open a gRPC connection with: "unix:///csi/csi.sock"
W0101 13:47:03.396282 1288131 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:08.527153 1288131 main.go:199] Calling CSI driver to discover driver name
I0101 13:47:08.530495 1288131 main.go:209] CSI driver name: "directpv-min-io"
I0101 13:47:08.530662 1288131 node_register.go:53] Starting Registration Server at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.531182 1288131 node_register.go:62] Registration Server started at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.531490 1288131 node_register.go:92] Skipping HTTP server because endpoint is set to: ""
ansible@lb:~$ k logs -n directpv node-server-b94hk -c node-server
I0101 13:47:08.001918 1288336 node-server.go:78] Identity server started
I0101 13:47:08.002012 1288336 node-server.go:101] Node server started
I0101 13:47:08.002109 1288336 reflector.go:289] Starting reflector *v1beta1.DirectPVVolume (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:08.002111 1288336 reflector.go:289] Starting reflector *v1beta1.DirectPVDrive (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:08.002123 1288336 reflector.go:325] Listing and watching *v1beta1.DirectPVDrive from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:08.002118 1288336 reflector.go:325] Listing and watching *v1beta1.DirectPVVolume from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:08.002229 1288336 ready.go:42] Serving readiness endpoint at :30443
I0101 13:47:08.002251 1288336 metrics.go:63] Starting metrics exporter at port 10443
I0101 13:47:08.102815 1288336 controller.go:141] volume controller synced and ready
I0101 13:47:08.102815 1288336 controller.go:141] drive controller synced and ready
ansible@lb:~$ k logs -n directpv node-server-cffjr
Defaulted container "node-driver-registrar" out of: node-driver-registrar, node-server, node-controller, liveness-probe
I0101 13:46:53.572791 913074 main.go:167] Version: v2.8.0
I0101 13:46:53.572850 913074 main.go:168] Running node-driver-registrar in mode=registration
I0101 13:46:53.573712 913074 main.go:192] Attempting to open a gRPC connection with: "unix:///csi/csi.sock"
W0101 13:47:03.574612 913074 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:08.996074 913074 main.go:199] Calling CSI driver to discover driver name
I0101 13:47:08.998905 913074 main.go:209] CSI driver name: "directpv-min-io"
I0101 13:47:08.998947 913074 node_register.go:53] Starting Registration Server at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.999300 913074 node_register.go:62] Registration Server started at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.999590 913074 node_register.go:92] Skipping HTTP server because endpoint is set to: ""
ansible@lb:~$
ansible@lb:~$ k logs -n directpv node-server-cffjr -c node-server
I0101 13:47:08.214595 913789 node-server.go:78] Identity server started
I0101 13:47:08.214695 913789 node-server.go:101] Node server started
I0101 13:47:08.214875 913789 ready.go:42] Serving readiness endpoint at :30443
I0101 13:47:08.214904 913789 reflector.go:289] Starting reflector *v1beta1.DirectPVVolume (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:08.214926 913789 reflector.go:325] Listing and watching *v1beta1.DirectPVVolume from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:08.214935 913789 reflector.go:289] Starting reflector *v1beta1.DirectPVDrive (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:08.214951 913789 reflector.go:325] Listing and watching *v1beta1.DirectPVDrive from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:08.215004 913789 metrics.go:63] Starting metrics exporter at port 10443
I0101 13:47:08.315561 913789 controller.go:141] volume controller synced and ready
I0101 13:47:08.315576 913789 controller.go:141] drive controller synced and ready
ansible@lb:~$ k logs -n directpv node-server-d4qj2
Defaulted container "node-driver-registrar" out of: node-driver-registrar, node-server, node-controller, liveness-probe
I0101 13:46:53.120833 274767 main.go:167] Version: v2.8.0
I0101 13:46:53.120873 274767 main.go:168] Running node-driver-registrar in mode=registration
I0101 13:46:53.123102 274767 main.go:192] Attempting to open a gRPC connection with: "unix:///csi/csi.sock"
W0101 13:47:03.124849 274767 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:08.600953 274767 main.go:199] Calling CSI driver to discover driver name
I0101 13:47:08.606361 274767 main.go:209] CSI driver name: "directpv-min-io"
I0101 13:47:08.606716 274767 node_register.go:53] Starting Registration Server at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.606839 274767 node_register.go:62] Registration Server started at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.607590 274767 node_register.go:92] Skipping HTTP server because endpoint is set to: ""
ansible@lb:~$
ansible@lb:~$ k logs -n directpv node-server-d4qj2 -c node-server
I0101 13:47:07.514455 274934 node-server.go:78] Identity server started
I0101 13:47:07.514553 274934 node-server.go:101] Node server started
I0101 13:47:07.514667 274934 reflector.go:289] Starting reflector *v1beta1.DirectPVDrive (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.514680 274934 reflector.go:325] Listing and watching *v1beta1.DirectPVDrive from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.514668 274934 reflector.go:289] Starting reflector *v1beta1.DirectPVVolume (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.514724 274934 reflector.go:325] Listing and watching *v1beta1.DirectPVVolume from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.514775 274934 ready.go:42] Serving readiness endpoint at :30443
I0101 13:47:07.514894 274934 metrics.go:63] Starting metrics exporter at port 10443
I0101 13:47:07.614820 274934 controller.go:141] drive controller synced and ready
I0101 13:47:07.714898 274934 controller.go:141] volume controller synced and ready
ansible@lb:~$ k logs -n directpv node-server-mf2kl
Defaulted container "node-driver-registrar" out of: node-driver-registrar, node-server, node-controller, liveness-probe
I0101 13:46:53.399009 9937 main.go:167] Version: v2.8.0
I0101 13:46:53.399074 9937 main.go:168] Running node-driver-registrar in mode=registration
I0101 13:46:53.399644 9937 main.go:192] Attempting to open a gRPC connection with: "unix:///csi/csi.sock"
W0101 13:47:03.400445 9937 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:08.157783 9937 main.go:199] Calling CSI driver to discover driver name
I0101 13:47:08.160593 9937 main.go:209] CSI driver name: "directpv-min-io"
I0101 13:47:08.161931 9937 node_register.go:53] Starting Registration Server at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.162015 9937 node_register.go:62] Registration Server started at: /registration/directpv-min-io-reg.sock
I0101 13:47:08.162091 9937 node_register.go:92] Skipping HTTP server because endpoint is set to: ""
ansible@lb:~$ k logs -n directpv node-server-mf2kl -c node-server
I0101 13:47:07.525129 10133 node-server.go:78] Identity server started
I0101 13:47:07.525200 10133 node-server.go:101] Node server started
I0101 13:47:07.525314 10133 reflector.go:289] Starting reflector *v1beta1.DirectPVVolume (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.525326 10133 reflector.go:325] Listing and watching *v1beta1.DirectPVVolume from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.525337 10133 reflector.go:289] Starting reflector *v1beta1.DirectPVDrive (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.525343 10133 ready.go:42] Serving readiness endpoint at :30443
I0101 13:47:07.525352 10133 reflector.go:325] Listing and watching *v1beta1.DirectPVDrive from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:07.525539 10133 metrics.go:63] Starting metrics exporter at port 10443
I0101 13:47:07.625870 10133 controller.go:141] volume controller synced and ready
I0101 13:47:07.625957 10133 controller.go:141] drive controller synced and ready
nsible@lb:~$ k logs -n directpv node-server-s2k8k
Defaulted container "node-driver-registrar" out of: node-driver-registrar, node-server, node-controller, liveness-probe
I0101 13:47:03.836638 3871521 main.go:167] Version: v2.8.0
I0101 13:47:03.836693 3871521 main.go:168] Running node-driver-registrar in mode=registration
I0101 13:47:03.838107 3871521 main.go:192] Attempting to open a gRPC connection with: "unix:///csi/csi.sock"
W0101 13:47:13.839573 3871521 connection.go:183] Still connecting to unix:///csi/csi.sock
I0101 13:47:21.733565 3871521 main.go:199] Calling CSI driver to discover driver name
I0101 13:47:21.743273 3871521 main.go:209] CSI driver name: "directpv-min-io"
I0101 13:47:21.743635 3871521 node_register.go:53] Starting Registration Server at: /registration/directpv-min-io-reg.sock
I0101 13:47:21.744608 3871521 node_register.go:62] Registration Server started at: /registration/directpv-min-io-reg.sock
I0101 13:47:21.746201 3871521 node_register.go:92] Skipping HTTP server because endpoint is set to: ""
ansible@lb:~$
ansible@lb:~$ k logs -n directpv node-server-s2k8k -c node-server
I0101 13:47:20.655393 3872361 node-server.go:78] Identity server started
I0101 13:47:20.655553 3872361 node-server.go:101] Node server started
I0101 13:47:20.655954 3872361 reflector.go:289] Starting reflector *v1beta1.DirectPVVolume (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:20.655966 3872361 ready.go:42] Serving readiness endpoint at :30443
I0101 13:47:20.655982 3872361 reflector.go:325] Listing and watching *v1beta1.DirectPVVolume from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:20.655966 3872361 reflector.go:289] Starting reflector *v1beta1.DirectPVDrive (10m0s) from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:20.656011 3872361 reflector.go:325] Listing and watching *v1beta1.DirectPVDrive from k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0101 13:47:20.656139 3872361 metrics.go:63] Starting metrics exporter at port 10443
I0101 13:47:20.756080 3872361 controller.go:141] drive controller synced and ready
I0101 13:47:20.756118 3872361 controller.go:141] volume controller synced and ready
balamurugana commented
It looks like you don't have permission to get some kubernetes resources for kubectl directpv info. Try running other commands like discover, list drives etc to check whether you are able to list DirectPV resources
darioneto commented
ansible@lb:~$ kubectl directpv info
ERROR DirectPV not installed
ansible@lb:~$ kubectl directpv discover
ERROR unable to complete the discovery; context deadline exceeded
No drives are available to initialize
ansible@lb:~$ kubectl directpv list drives
No matching resources found
balamurugana commented
It looks like you have connection and permission issue. Please fix it.
darioneto commented
Not really sure why this is the case, I see that the role and role binding for directpv-min-io in the directpv namespace are set up correctly.
darioneto commented
This is what comes with the lot. I don't think this is related to any permissions. All is set correctly.
ansible@lb:~$ kubectl describe clusterrolebinding directpv-min-io
Name: directpv-min-io
Labels: application-name=directpv.min.io
application-type=CSIDriver
directpv.min.io/created-by=kubectl-directpv
directpv.min.io/version=v1beta1
Annotations: directpv.min.io/plugin-version: v4.0.10
rbac.authorization.kubernetes.io/autoupdate: true
Role:
Kind: ClusterRole
Name: directpv-min-io
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount directpv-min-io directpv
ansible@lb:~$ kubectl describe clusterrole directpv-min-io
Name: directpv-min-io
Labels: application-name=directpv.min.io
application-type=CSIDriver
directpv.min.io/created-by=kubectl-directpv
directpv.min.io/version=v1beta1
Annotations: directpv.min.io/plugin-version: v4.0.10
rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
customresourcedefinition.apiextensions.k8s.io [] [] [create delete get list patch update watch]
customresourcedefinitions.apiextensions.k8s.io [] [] [create delete get list patch update watch]
customresourcedefinition.directpv.min.io [] [] [create delete get list patch update watch]
customresourcedefinitions.directpv.min.io [] [] [create delete get list patch update watch]
persistentvolumes [] [] [create delete get list patch watch]
endpoints [] [] [create delete get list update watch]
leases.coordination.k8s.io [] [] [create delete get list update watch]
directpvdrives.directpv.min.io [] [] [create delete get list update watch]
directpvinitrequests.directpv.min.io [] [] [create delete get list update watch]
directpvnodes.directpv.min.io [] [] [create delete get list update watch]
directpvvolumes.directpv.min.io [] [] [create delete get list update watch]
events [] [] [create list patch update watch]
persistentvolumeclaims [] [] [get list update watch]
nodes [] [] [get list watch]
pod [] [] [get list watch]
pods [] [] [get list watch]
secret [] [] [get list watch]
secrets [] [] [get list watch]
csinodes.storage.k8s.io [] [] [get list watch]
storageclasses.storage.k8s.io [] [] [get list watch]
volumeattachments.storage.k8s.io [] [] [get list watch]
volumesnapshotcontents.snapshot.storage.k8s.io [] [] [get list]
volumesnapshots.snapshot.storage.k8s.io [] [] [get list]
persistentvolumeclaims/status [] [] [patch]
podsecuritypolicies.policy [] [] [use]
I think there must be a bug somehere , maybe compatibility , not sure but this ain't working regardless of many attempts reinstalling it over etc.
In this log I have no idea what is expected result out of this or maybe this is normal?
➜ ~ k logs node-server-2btgv -n directpv node-driver-registrar
I0104 13:28:20.459130 3037068 main.go:167] Version: v2.8.0
I0104 13:28:20.459188 3037068 main.go:168] Running node-driver-registrar in mode=registration
I0104 13:28:20.459794 3037068 main.go:192] Attempting to open a gRPC connection with: "unix:///csi/csi.sock"
I0104 13:28:21.462862 3037068 main.go:199] Calling CSI driver to discover driver name
I0104 13:28:21.473629 3037068 main.go:209] CSI driver name: "directpv-min-io"
I0104 13:28:21.473697 3037068 node_register.go:53] Starting Registration Server at: /registration/directpv-min-io-reg.sock
I0104 13:28:21.474379 3037068 node_register.go:62] Registration Server started at: /registration/directpv-min-io-reg.sock
I0104 13:28:21.474560 3037068 node_register.go:92] Skipping HTTP server because endpoint is set to: ""
balamurugana commented
It is purely your local issue. We run CI with kubernetes ['v1.18.20', 'v1.19.16', 'v1.20.15', 'v1.21.14', 'v1.22.17', 'v1.23.17', 'v1.24.17', 'v1.25.16', 'v1.26.11', 'v1.27.8', 'v1.28.4', 'v1.29.0'] in minikube and k3s locally. You would need to check with your k8s admin.