Parsl/parsl

performance: use unix domain sockets between htex submit side and interchange

benclifford opened this issue · 0 comments

Is your feature request related to a problem? Please describe.

ZMQ supports unix domain sockets, when both ends of the connection are on the same host. That is the case for the workflow submit process and interchange.

Raw performance for this sockets can be much higher than TCP-over-loopback, which is how these two processes communicate now. For example, see https://stackoverflow.com/questions/14973942/tcp-loopback-connection-vs-unix-domain-socket-performance

It should be straightforward to try out, and less straightforward to benchmark, if this has any noticeable effect on interchange performance.