Install the following:
- docker
- kubectl
- minikube
- lvm (optional for ceph)
- Linux module called RBD (Can check if exists by doing
modprobe rbd
) - helm
- Disks need to be cleaned everytime a new cluster is created
# For each disk
DISK="/dev/sdg" && \
sudo sgdisk --zap-all $DISK && \
sudo dd if=/dev/zero of="$DISK" bs=1M count=100 oflag=direct,dsync && \
sudo blkdiscard $DISK
- Start 3 node K8s cluster
minikube start --nodes 3 -p rook-project --cpus 6 --memory 8g
- Check storage devices have no parition by doing
lsblk -f
- Add helm repo for rook
helm repo add rook-release https://charts.rook.io/release
- Deploy Rook operator
helm install --debug --create-namespace --namespace rook-ceph rook-ceph rook-release/rook-ceph
- Make sure Rook operator is running
kubectl -n rook-ceph get pod
- Create the Ceph cluster. WARNING: Make sure you check all the attributes in values-override.yaml (device filters etc)
helm install --debug --create-namespace --namespace rook-ceph rook-ceph-cluster \
--set operatorNamespace=rook-ceph rook-release/rook-ceph-cluster -f values-override.yaml
- Check Ceph cluster status
kubectl -n rook-ceph get cephcluster
8.1. To delete Ceph cluster
helm delete --namespace rook-ceph rook-ceph-cluster
minikube delete --all
If using minikube just run minikube -p rook-project dashboard
to get dashboard. Otherwise manually deploy:
- Install dashboard on cluster
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.4.0/aio/deploy/recommended.yaml
- Create required user accounts
kubectl apply -f service-account.yaml && \
kubectl apply -f user-binding.yaml
- Start proxy
kubectl proxy
- Copy token to clipboard
kubectl -n kubernetes-dashboard get secret $(kubectl -n kubernetes-dashboard get sa/admin-user -o jsonpath="{.secrets[0].name}") -o go-template="{{.data.token | base64decode}}" | xclip -sel clip
- Go to dashboard http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ and paste token
- Use the YAML to deploy app:
kubectl apply -f app.yaml
- To remove deployment:
kubectl delete -f app.yaml
- Start K8s proxy:
kubectl -n rook-ceph proxy
- Go to URL: http://localhost:8001/api/v1/namespaces/rook-ceph/services/https:rook-ceph-mgr-dashboard:https-dashboard/proxy/
- Get password :
kubectl -n rook-ceph get secret rook-ceph-dashboard-password -o jsonpath="{['data']['password']}" | base64 --decode | xclip -sel clip
- Create directory for mount point:
sudo mkdir -p /mnt/drive
- Mount drive:
sudo mount /dev/sdc1 /mnt/drive