bitwalker/libcluster

Use watch for Kubernetes strategy

dnsbty opened this issue · 3 comments

Thanks for creating an amazing library that makes clustering so much simpler!

Kubernetes has the ability to watch a given resource. When it is watched, the transfer encoding is set to chunked, and any updates after the specified version will be sent down in new chunks. This way the node wouldn't have to poll the endpoint every 5 seconds, but could create a long-lived connection to it and then just update whenever new chunks are received. I believe this would allow for node list updates to occur much closer to real-time.

Are there any drawbacks to supporting watch requests?

Would there be support for a PR implementing this?

If yes, would it make more sense to update the existing Cluster.Strategy.Kubernetes module or to create a new one like Cluster.Strategy.KubernetesWatched or something like that?

It seems like the high-level functionality will be exactly the same but more performant. That being said, I don't know when the watch capability was added to Kubernetes, so there is potential that it would be a breaking change for those who might be on older version. The docs reference watch in version 1.14, but I haven't looked further back than that.

I'm going to go ahead and start working on this. I'll just replace the current strategy for now, and we can discussion potential separation at the PR stage.

The watch can also be simulated with the -w flag in kubectl if you're interested in seeing how this works:

kubectl get endpoints -l app=level10 -w -o json

@dnsbty Have you made any progress on this?

@complexsplit I haven't. I ended up moving away from Kubernetes for the project I was working on. You (or someone else) is more than welcome to pick this up and I'm happy to give advice if needed though.