sky-uk/kfp-operator

RunConfiguration Triggers

Closed this issue · 0 comments

Overview

RunConfigurations currently can only be triggered by single Cron Schedules.
In order to allow different kinds of triggers, the existing API needs to be redesigned.

Solution proposal

  • Extend the RunConfiguration CRD by replacing the single schedule with an array of triggers whose elements can be of type schedule (more to be defined later).
  • Move run spec into its own spec.run block separate from triggers
  • The existing RunConfiguration would be redefined into RunSchedule which becomes a managed dependent resource of RunConfiguration
apiVersion: pipelines.kubeflow.org/v1alpha5
kind: RunConfiguration
metadata:
  name: a-recurring-run
spec:
  run:
    pipeline: a-pipeline
  triggers:
  - type: schedule
    cronExpression: 1 * * * *
  - type: schedule
    cronExpression: 2 * * * *
status:
  synchronizationState: Updating
  provider: ...
  observedPipelineVersion: ...
  observedGeneration: ...
classDiagram
RunConfiguration *-- RunShedule1
RunConfiguration *-- RunShedule2

class RunShedule1 {
    schedule: 1 * * * *
}

class RunShedule2 {
    schedule: 2 * * * *
}
Loading

Implementation Notes:

'State Machine' logic:

  • check dependencies
  • determine and set new status
  • action discrepancies

Resource Status (in order of precedence)

Dependency states RC state Example
At least one Updating/Creating Updating 🔁✔️❌
At least one Failed (no pending updates) Failed ❌✔️✔️
All Succeeded Succeeded ✔️✔️✔️