aio-libs/aiozmq

Self synchronized rpc.connect_pubsub behavior?

mayfield opened this issue · 7 comments

Disclaimer: I'm a new user to zeromq.

I believe this is related to a classic zeromq sync problem for pub/sub, but given that the rpc module attempts to provide "human" abstractions I was wondering if you and the community would support
making the rpc pub/sub code more deterministic without needing external synchronization techniques?

The lack of determinism I speak of can be demonstrated with some hacky code like this...

import asyncio
from aiozmq import rpc

async def test():
    await rpc.serve_pubsub(Handler(), subscribe='topic',
                           bind='ipc://aiozmq_test')
    client = await rpc.connect_pubsub(connect='ipc://aiozmq_test')
    # Uncomment this line to make it work.
    # await asyncio.sleep(0.01)
    for i in range(1000):
        await client.publish('topic').check(i)


class Handler(rpc.AttrHandler):

    @rpc.method
    def check(self, i):
        print("CHECK", i)


loop = asyncio.get_event_loop()
loop.run_until_complete(test())
loop.run_forever()

There are two troubling aspects to this test. One is that without the arbitrary delay none of the publish messages show up; Even if the loop is allowed to run for some large length of time it will never get messages to the subscriber. And then there is the more obvious problem that things don't work without an external sync mech (or sleep hack as portrayed in this dumb test).

Ideally the connecting client, either connect_pubsub() or serve_pubsub() would perform some sync routine to only allow publish to work once it's alive and well.

Sorry, I have no idea how to perform synchronization in generic way.
It's always domain specific.

Any thoughts about the /dev/null sink when publishing without sleeps?

Sorry, I don't follow.
Please elaborate.

"One is that without the arbitrary delay none of the publish messages show up; Even if the loop is allowed to run for some large length of time it will never get messages to the subscriber."

I understand the problem but don't know what /dev/null sink is it and how can it help.

There seems to be a communication breakdown because I used jargon. I was just referring to the first "troubling aspect" described in my initial message as a "/dev/null sink". Ie. messages disappearing down a proverbial kitchen sink to /dev/null (nowhere land).

Sorry, say again I don't know how to solve your problem in aiozmq boundaries.
Synchronization may be done using additional sockets but it's out of scope of aiozmq.