kubeflow/kfp-tekton

[KFP-Tekton v2] Merge DAG drivers and publishers into a sub-dag controller to reduce Tekton graph complexity.

Tomcli opened this issue · 0 comments

Tomcli commented

/kind feature

Description:
When adding driver and publisher tasks into the sub-dag, we are introducing a new layer of complexity to the Tekton graph because the driver needs to connect with all the root nodes and publisher needs to connect with all the leave node. Because driver always runs before the main sub-dag(pipelinerun in Tekton) and publisher runs after the sub-dag is done, we can have our controller handles those logic instead of embedding drivers and publishers into the Tekton pipeline graph.

This will require we implement the status-IR first as right now we don't have a way to update sub-dag and root dag pipeline status in MLMD and reflect them on the UI.

Screenshot 2023-11-22 at 9 27 59 AM

Additional information:
[Miscellaneous information that will assist in solving the issue.]