jgroups-extras/jgroups-kubernetes

RFE: Use a Service for discovery rather than labels

onelapahead opened this issue · 10 comments

onelapahead commented

Although a Service uses labels for selecting which Pods to forward traffic to, it also considers the liveness and readiness probes to ensure that it only creates Endpoints for Pods which are considered "Ready". Other applications, such as Apache Ignite use Services for getting Endpoints to get Pod IPs, rather than label selectors for this reason AFAIK.

With the current implementation, KUBE_PING discovers all Pods in the Running state matching the provided labels and ignores whether they are healthy or not. However, I could see the argument for letting the application using JGroups to determine whether or not a peer is healthy, but this could save it the effort of having to connect and do that check.

The Kubernetes docs themselves recommend headless Service's as a means of discovery.

onelapahead commented

Yep apologies confused phase for status.

onelapahead commented

However, I still think using a Service would be more appropriate and reduce the amount of code in the client.

belaban commented

I'm always interested in suggestions.. e.g. in the form of a pull request :-)

  • 👍1
slaskawi commented

@brix4dayz Actually, you can use DNS_PING which does exactly what you want - it can use Service (DNS entries to be precise) to obtain cluster members IPs.

  • 👍2
missedone commented

In my experience, KUBE_PING is a more generic solution as sometimes we don't have Service at all.

onelapahead commented

@slaskawi had no idea K8s made SRV records like that, very cool! Thanks, that is more along the lines of what I has thinking.

@missedone though you might not have a Service, you have to create a ServiceAccount and a RoleBinding (at least for OpenShift) in order to use KUBE_PING I think its worth noting. Depending on the policies your cluster admin is enforcing, it might be easier to make a Service.

missedone commented

@slaskawi, is there any example about using DNS_PING in kubernetes? thx

missedone commented

you have to create a ServiceAccount and a RoleBinding (at least for OpenShift) in order to use KUBE_PING I think its worth noting. Depending on the policies your cluster admin is enforcing

agreed :)

slaskawi commented

@missedone Here's an example project I used for testing: https://github.com/slaskawi/jgroups-dns-ping-example