facebookincubator/gloo

New context type that can dispatch to N actual contexts

pietern opened this issue · 0 comments

This is what we do in PyTorch upstream today but it would be good to move the functionality into Gloo. This would be a new type of context that wraps N contexts and load balances over them. If we were to create these wrapped contexts for multiple devices, this would be a natural place to apply load balancing as well.