MountVolume.MountDevice failed for volume "pvc": rpc error: code = Internal desc = could not format "/dev/xvdac", mount failed: exit status 32, mount(2) system call failed: File exists.
AjayEdupuganti opened this issue · 3 comments
Hi,
I have my kubernetes cluster on aws instances, spinned up using kubedadm. I am using aws-ebs to allocate persisntant volumes to my redis pods. While spinning up the pod and attach pvs to the pods, they are being created on volumes and getting attached to the instance but they are not getting mounted to the pods. looks like only one pvc is not getting attached. I am not able to understand why and how. I am using t2.xlarge instances. calico network plugin.
lsblk
kubectl describe pod redis-stack-1 -n redis
root@ip-10-0-1-10:~/manifests/redis# kubectl get statefulset -n redis -o wide
NAME READY AGE CONTAINERS IMAGES
redis-stack 1/2 14m redis redis/redis-stack-server:latest
FINALLY FOUND THE ISSUE:
Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/bb418c99aacece30013bb68ec90d5d3646149b8c04d296c6d1b66b5546a05bd3/globalmount: mount(2) system call failed:
File exists.
This part of the error is important.
So, i spun up a pod to use a pvc to test the allocation of the volume.
and then i deleted the pvc. it is in terminating stage. I never gave much thought to it.
Then after getting stressed over it for 10hrs. and restarting it after 5hrs helped me to think calm.
so i saw that a pod is still running that is using the ebs claim, I deleted the pod and then the pvc. immediately the volume got allocated to redis pod and the redis pods are running.
/close
It seems like you got it figured out, so I'm going to go ahead and close this issue. Please re-open or open a new issue if you need further assistance.
@ConnorJC3: Closing this issue.
In response to this:
/close
It seems like you got it figured out, so I'm going to go ahead and close this issue. Please re-open or open a new issue if you need further assistance.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.