StatefulSet clustering in Kube
sholdee opened this issue · 3 comments
I have been trying to create a Gravity cluster with a 3 replica StatefulSet, but the methods documented for joining the cluster are not possible in this configuration. Various methods I have tried to manually create the cluster have all failed. I was wondering how difficult it would be to have some environment variable, for instance, that could be set that would bootstrap the cluster.
One thing I have done is set the INSTANCE_IDENTIFIER for each pod dynamically to it's DNS name from a headless service (manifest below), which works, and then clustering manually via etcdctl, but I haven't been able to get that part working.
gravity-statefulset.yml: https://pastebin.com/raw/2AgK5xGV
Most of the node joining mechanics are attempts to make it easier for etcd newcomers and hence probably don't have enough dynamic-ness for K8s. It would probably be easier to use something like https://operatorhub.io/operator/etcd to run etcd, and run 3 gravity instances without etcd
Most of the node joining mechanics are attempts to make it easier for etcd newcomers and hence probably don't have enough dynamic-ness for K8s. It would probably be easier to use something like https://operatorhub.io/operator/etcd to run etcd, and run 3 gravity instances without etcd
You are right; that was the ticket. I have it working now. Thanks!
For anyone else who wants to try and get this going, here are my manifests:
Thank you for sharing. This config worked for me with microk8s. One thing that may be useful for future readers - the cluster only showed 2 replicas (number of replicas - 1) until the statefulset was restarted.