microsoft/demikernel

[runtime] Remove result from Yielder and create condition variables

iyzhang opened this issue · 0 comments

Context

This issue tracks a design change to our Yielder abstraction. We introduced this abstraction in #556 and have now had some time to see its flaws.

The primary issue with the Yielder is that it conflates two mechanisms: 1. the mechanism to wake a yielded coroutine (i.e., the waker bit) and 2. a condition variable that must be true before returning from the yield, namely the result variable must be non-empty. Having these two mechanisms in the same abstraction is problematic because the first is shared among all Yielders in a coroutine and the second is unique to the Yielder. Then we create Yielders and hand out yielder handles, which all have references to the same waker, making it difficult to track which coroutines might wake this one. Finally, it is possible to wake a coroutine after it has been completed and removed from the scheduler because the yielder handles still hold references to the waker and they are in other coroutines. This inadvertently wakes the new coroutine occupying that index in the slab.

Proposed Solution

We separate the mechanisms for yielding and waking coroutines from the conditions that cause them to wake. Specifically, yielding and waking will now be done on a per task basis and the other coroutine must wake the task by TaskId (or some other mechanism) ensuring that it is impossible to wake a completed task.

Then it is up to the libOS coroutine to decide what conditions to check for when woken and how to signal to the waking coroutine what work should be done. However, to support our use of the select! and select_bias! macros, we will create condition variables that check for specific conditions to be true before returning from yield. These conditions will be dictated by the libOSes and it will be up to the libOSes to wake the appropriate coroutine when conditions change. It is especially important the libOSes track what resources each coroutine uses and wake the coroutine when those resources should be freed. For example, all I/O operations in coroutines should track (i.e. use a conditional watched value for) the state of the underlying I/O interface (e.g., a socket) and exit when it is closed.