[Bug]: Facing Multiple Cancellation Errors : asyncio.exceptions.CancelledError in Chain reaction, failing the orchestrator and other agents on Load.

[avi-arora]

What happened?

Facing Multiple Cancellation Errors as below like a chain reaction, breaking everything in production.

asyncio.exceptions.CancelledError

The error is happening only in case when we put minimal load on server. if i run with only 1-2 client simultaneously it works perfectly, the moment i open multiple endpoints. it starts to fails.

I've tried both inmemory and redis task store. Same error.

Code for in memory:

    agent_card = get_agent_card(host, port)  
    task_store = InMemoryTaskStore()  
    executor = JudgementCaseResearchAgentExecutor()  
     # Create push notification components (optional)
    push_config_store = InMemoryPushNotificationConfigStore()
    push_sender = BasePushNotificationSender(httpx_client, push_config_store)

Code for redis:

Initialize Redis components with better configuration

            task_store = RedisTaskStore(redis_client, prefix="...")
            config = ConsumerGroupConfig(strategy=ConsumerGroupStrategy.SHARED_LOAD_BALANCING)
            queue_manager = RedisStreamsQueueManager(
                redis_client, 
                prefix="...",
                consumer_config=config
            )
            push_config_store = RedisPushNotificationConfigStore(redis_client, prefix="...")

Logs Screenshots:

Relevant log output

Traceback (most recent call last):
  File "/tmp/8dde22529ac0237/antenv/lib/python3.13/site-packages/a2a/utils/telemetry.py", line 194, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/8dde22529ac0237/antenv/lib/python3.13/site-packages/a2a/server/events/event_queue.py", line 111, in dequeue_event
    event = await self.queue.get()
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python/3.13.5/lib/python3.13/asyncio/queues.py", line 186, in get
    await getter
asyncio.exceptions.CancelledError

Code of Conduct

• I agree to follow this project's Code of Conduct