Loop-less zmq.asyncio
f0t0n opened this issue · 3 comments
Hi guys,
Is it possible to re-work the zmq.asyncio package and remove the need of usage of zmq.asyncio.ZMQEventLoop?
Say if I use zmq in my aiohttp application in combination with uvloop I can't just import and use zmq.asyncio easily. So that this forces me to use third-party libraries like aiozmq instead of the native pyzmq.
Also please take a look at aio-libs/aiohttp#1092 (comment) describing specific issues with zmq.asyncio.ZMQEventLoop.
I'm not proficient in this particular area therefore I delegate the next discussion to @asvetlov and other guys who know this area well.
If someone is able to work on this, it would be really nice! And as @asvetlov said he can point on required changes.
The implementation of loop-less transport is here: https://github.com/aio-libs/aiozmq/blob/master/aiozmq/core.py#L667
While it's based on Transport protocol the code may be transplanted into zmq.asyncio.Socket easily.
The key point is not adding writer but getting notification file descriptor by fd = zmq_sock.getsockopt(zmq.FD) and subscribing for events from it.
The tricky part of implementation is: after every read/write operation the file descriptor should be requested again for upcoming events and if there are events to process .read_ready() should be scheduled again on the next loop iteration.
From my understanding it's because zmq.FD works on edge-triggered level, not level triggered one.
@tailhook had consulted me about the patch, he knows better how zeromq works in this mode.
Having dealt with integrating the edge-triggered FD in an eventloop in zmq.green, I don't want to be responsible for maintaining another implementation that relies on it. I've pushed for a level-triggered separate read/write FD API in libzmq for years, so if that ever happens I will happily drop the zmq_poll implementations and use it.
Fortunately, aiozmq exists and takes on that challenge, so please feel free to use that. I don't view zmq.asyncio as a replacement for it at all, just one simple implementation of AIO-compatible use of zmq.
+1 I threw out a polling suggestion in aio-libs/aiohttp#1223. I'm not an expert. Having pyzmq capable of operating with the base asyncio loop would be pretty dreamy.