michaelosthege/pytensor-federated

Let `ArraysToArraysServiceClient` choose from a list of possible servers

michaelosthege opened this issue · 0 comments

Instead of passing just one host, client combination, the ArraysToArraysServiceClient could take a list of servers to choose from.

Each server must, of course, behave identically.

Doing this allows for a failover mechanism (#9) but also enables client-side load-balancing in situations where the ArraysToArraysServiceClient is forked/spawned into tens or hundres of copies.

For the load balancing, we probably need some get_numer_of_connected_clients endpoint on the server so the clients can np.argmin(np.random.permutation(options)).