Mounting CephFS is giving an Error.
saitejar opened this issue · 4 comments
Request for Help.
I am using ceph-helm, also running mds with it.
ceph -s
shows that everything is up and running.
I am unable to mount ceph fs to a pod.
I am using this ceph-test-fs.yaml
apiVersion: v1
kind: Pod
metadata:
labels:
test: cephfs
name: ceph-cephfs-test
namespace: ceph
spec:
nodeSelector:
node-type: storage
containers:
- name: cephfs-rw
image: busybox
command:
- sh
- -c
- while true; do sleep 1; done
volumeMounts:
- mountPath: "/mnt/cephfs"
name: cephfs
volumes:
- name: cephfs
cephfs:
monitors:
#This only works if you have skyDNS resolveable from the kubernetes node. Otherwise you must manually put in one or more mon pod ips.
- ceph-mon.ceph:6789
user: admin
secretRef:
name: ??
I am not sure what keyring to use to be able to mount.
and also I did try all the secrets generated by ceph-helm deployment.
Can anyone help me with mounting ceph fs?
Here is the error log:
MountVolume.SetUp failed for volume "cephfs" : CephFS: mount failed: mount failed: exit status 32 Mounting command: systemd-run Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/7d82c944-e01f-11e7-bf03-001c42d61047/volumes/kubernetes.io~cephfs/cephfs --scope -- mount -t ceph -o name=admin,secretfile=/etc/ceph/admin.secret ceph-mon.ceph:6789:/ /var/lib/kubelet/pods/7d82c944-e01f-11e7-bf03-001c42d61047/volumes/kubernetes.io~cephfs/cephfs Output: Running scope as unit run-10734.scope. mount: wrong fs type, bad option, bad superblock on ceph-mon.ceph:6789:/, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so.
- which kubernetes release you are using?
- on that node, are you able to ping
ceph-mon.ceph
(and check if your host's DNS is set to kube-dns service ip)? - what do you see in
syslog
anddmesg
?
@rootfs
Yes, I can ping ceph-mon.ceph
In dmesg, I see.
[29119.038942] libceph: resolve 'ceph-mon.ceph' (ret=0): 10.211.55.195:0
[29119.038947] libceph: bad option at 'secret'
Looks like I am using a wrong secret. But I am unsure which secret to use with admin
user.
These are the secrets that ceph-helm deployment has created.
pvc-ceph-conf-combined-storageclass
ceph-bootstrap-mgr-keyring
ceph-client-admin-keyring
ceph-bootstrap-mds-keyring
ceph-bootstrap-osd-keyring
ceph-bootstrap-rgw-keyring
ceph-mon-keyring
pvc-ceph-client-key
ceph-keystone-user-rgw
how did you provision cephfs PV? it looks mount picked up secret file over secret