Clarification Needed on Thread Context Switch in writer() Function (example streaming-server.cpp)

[bronfo1]

Hello,

I'm currently reviewing the server-side code for handling gRPC requests, specifically the use of co_await rpc.write(response); in the writer() coroutine function. I observed that there is a comment stating "// Now we are back on the main thread." after this line of code.

From my understanding, in Asio and C++ coroutines, the execution context or thread does not automatically switch back to the original context (such as the main thread of grpc_context) after an await operation, unless explicitly specified. However, in this instance, I don't see an explicit switch back to the grpc_context after the write operation in a multi-threaded environment (using asio::thread_pool).

Could you please clarify if the comment accurately reflects the behavior of the code? If there is indeed an automatic switch back to the main grpc_context thread happening, could you point out where and how this context switch is being handled in the code?

This clarification will greatly help in understanding the correct thread behavior and context management in asynchronous operations within the application.

Thank you for your time and assistance.

[Tradias]

Hi,

the comment in the code is in fact accurate. Asio describes the thread-switching behavior in their documentation. Although I find it rather difficult to understand, let me try to explain it in my own words.

Take the following example:

    asio::io_context io_context1;
    asio::steady_timer timer{io_context1};
    asio::io_context io_context2;
    asio::co_spawn(
        io_context2,
        [&]() -> asio::awaitable<void>
        {
            // on io_context2 thread

            // Timer is handled on io_context1 thread
            // But after completion it will:
            // `asio::dispatch(asio::get_associated_executor(use_awaitable_completion_handler,
            // timer.get_executor()), use_awaitable_completion_handler)`
            // which causes the switch back to io_context2 thread
            co_await timer.async_wait(asio::use_awaitable);

            // on io_context2 thread
        },
        asio::detached);
    auto g = asio::make_work_guard(io_context1);
    std::thread t{[&]
                  {
                      io_context1.run();
                  }};
    io_context2.run();
    g.reset();
    t.join();

Asio performs the following steps on this line co_await timer.async_wait(asio::use_awaitable);:

• Convert the completion token (asio::use_awaitable) into a completion handler using asio::async_result machinery. The details do not matter much, but the result is a callable object (called completion handler) that is to be invoked when the timer completes.
• Get the I/O executor by calling timer.get_executor() and use that to do whatever is necessary to initiate and wait for the timer (system calls, epoll, etc.)
• Meanwhile create the completion handler executor by calling asio::get_associated_executor(completion_handler, timer.get_executor()) and establish work-tracking (call asio::make_work_guard()) until the timer completes. use_awaitable's completion handler actually has an associated executor, it is the one used in the call to co_spawn (io_context2).
• When the timer completes, invoke the completion handler as if by calling asio::dispatch(completion_handler_executor, completion_handler). This is where the thread switching is happening, going from timer.get_executor() (io_context1) to the completion handler executor (io_context2).
• The completion handler will then resume the coroutine.

Since asio-grpc tries to mimic Asio's behavior as much as possible, replacing asio::steady_timer with agrpc::ServerRPC and io_context with GrpcContext yields the same thread-switching result.

I hope that helps, let me know if something is still unclear.

[bronfo1]

Hi,

Thank you for the clear explanation. I tested it and everything worked just as you described. Your guidance made the thread-switching behavior in Asio much clearer.

Also, I want to say that asio-grpc is an amazing project. Really appreciate all the work you've put into it.

Best

[Tradias]

Thanks, glad I could help.

Btw, you can disable the thread-switching this way:

inline void use_inline_awaitable() { return asio::bind_executor(asio::system_executor(), asio::use_awaitable); }


co_await timer.async_wait(use_inline_awaitable());

This will cause async_wait to resume the coroutine on whatever thread the timer's I/O executor is running on.

[bronfo1]

Hi again,

I revisited your example and noticed that after co_await timer.async_wait(), the execution returns to the io_context2 thread, the same thread as before the co_await. So, in the writer code, before co_await rpc.write(), it's in the thread_pool, right? By the same logic, does this mean that after co_await rpc.write(), the execution also returns to the thread_pool?

[Tradias]

It doesn't matter what thread it was on before the co_await, what matters is the first argument passed to asio::co_spawn. By default any co_await ... asio::use_awaitable asynchronous operation started within the co_spawned coroutine will switch back to that first argument (provided the asynchronous operation follows Asio's completion handler executor rules explained earlier). I know this can be a bit unfortunate because the co_spawn call can be far away in some other source file.

[bronfo1]

Great, it's finally clear now. Thank you so much.
Have a good day!

[bronfo1]

writer is (just a little far away) from the co_spawn that first argument is grpc_context or grpc_context's executor, right?

[Tradias]

Correct, the co_spawn call is actually hidden inside agrpc::register_awaitable_rpc_handler. It actually invokes co_spawn like so:

asio::co_spawn(asio::get_associated_executor(completion_handler, first_argument_passed_to_register_awaitable_rpc_handler())

Asio internally always works with executors. There is a co_spawn overload that takes an execution context, like GrpcContext, and essentially forwards the call to co_spawn(grpc_context.get_executor(), ...).