Mayastor not correctly provisioning requested PVC size
Closed this issue · 4 comments
- MK8s 4 node cluster ( 1 control plane ) 1.25 release
- Mayastor addon enabled
- ArgoCD enabling prometheus https://github.com/cloud-native-skunkworks/gitops-kubernetes-bootstrap/blob/main/templates/application_observability.yaml#L40
Available attached disk space mounted on /var/snap/microk8s
MicroK8s.img has a 20Gig sparse file for each mayastorpool
I have 1 deployment consuming a 1Gi single replica.
When attempting to create a 3 replica 50Gi PVC it fails
I believe clarification is needed on where this is failing.
I believe the answer here is to clearly document how to expand the default mayastorpool size
e.g. microk8s enable mayastor --default-pool-size=100G should be documented somewhere
Hi @AlexsJones. I think you may have already figured this out but what is happening here is the result of the way that Mayastor volume replicas are provisioned. If you try to provision a 50GiB volume with three replicas, then three separate replicas each of 50 GiB in size must be created, and each must be a) created in a different disk pool (for redundancy), b) fully accommodated within their respective pool singular and c) thick-provisioned
That is to say, replicas cannot span disk pools and are fully allocated at the time of creation (the latter will change in v2.1). To create a 50GiB replica, there must exist a disk pool with at least 50GiB of available capacity. And two more, if your StorageClass repl value is actually three.
If your Mayastor pools are only 20GiB in size, then obviously that's not possible. I would expect increasing the pool size to 100GiB to resolve the issue.
Thanks for adding to the answer here @GlennBullingham I left it open as a note for us to improve the docs, but I believe that's on the way shortly.
Documentation has been updated accordingly, and is now showing how to enable the mayastor addon and configure the default pool size. Closing the issue.