FairRootGroup/FairMQ

Single receive buffer limit for many connected peers for ZMQ connections

Opened this issue · 0 comments

rbx commented

Alice has a use case where many peers (700) send significant amount of data (50MB per message) to a single peer, overloading the receiver. This cannot be controlled with ZMQ queue sizes, since the configurable limits are per peer. Neither can this be controlled reliably after the ZMQ transfer, since it may well happen that many senders transfer at the same and the desired limit is broken before even single message is fully built.

Investigate and implement a solution which would allow setting such a limit for all peers - custom socket type or custom transport implementation which can talk to zmq peers and receive & build messages, while accounting for the single limit for all senders.