Yolean/kubernetes-kafka

Create storage class for regional persistent disks (GKE)

Closed this issue · 3 comments

https://cloudplatform.googleblog.com/2018/05/Get-higher-availability-with-Regional-Persistent-Disks-on-Google-Kubernetes-Engine.html

Google announced the beta availability of regional persistent disks. At first glance it seems like this could be a good step toward high availability without going all the way to a fully mirrored cluster.
We should create a storage class for these regional disks and think through use cases that this might address.

Very interesting. Thanks for pointing this out. It would actually solve the zookeeper situation and remove the argument for ephemeral zk, #123.

@clintfred Have you seen any information about performance? I guess this kind of HA comes at a cost. Maybe we should start with zookeeper, as brokers are basically designed to run on local spinning disks?

I don't have direct experience (yet) with this, but the docs claim the the performance is quite similar to their HDD and SSD persistent disks. My intuition is that they are sufficient for both zookeeper and brokers.

#191 includes an example of regional-pd