TimelyDataflow/timely-dataflow

Distinguish capabilities for multi-output operators

Closed this issue · 0 comments

At the moment, a Capability<T> is valid for all outputs of a custom operator, and this is enforced by copying the capability changes into the "held capabilities" for all outputs. However, one is able to describe operators without full connections between inputs and outputs. This lets us capture capabilities from an input that should not be able to reach one of the outputs, and then use the capability for that output. Even if the user does not do this, timely may report that the frontier on that output has regressed, because while the operator reports that messages cannot flow to that output the held capability implies that it could still happen.

One solution is to distinguish capabilities by output port, which should be relatively painless for all operators with only one output. For operators with multiple outputs, this would require acquiring and holding distinguished capabilities, which would be a bit of an ergonomic overhead for operators that previously wanted a capability for all outputs. The "overhead" is valuable for operators with non-standard summaries connecting their inputs and outputs, as they very much want to only hold a capability on a subset of outputs.