onyx-platform/onyx

Flow conditions improvements

lbradstreet opened this issue · 1 comments

Issue 1:

Short circuit exception flow condition such as:

{:flow/from :input-stream
 :flow/to [:error-task]
 :flow/short-circuit? true
 :flow/thrown-exception? true
 :flow/post-transform :my.ns/post-transform
 :flow/predicate :my.ns/handle-error?}

Should:
Only flow to :error-task for exceptions that pass that predicate.
Flow to all task other than :error-task the rest of the time?

Breaking changes in this solution:
All regular segments currently flow to :error-task.

Issue 2:
Flow conditions, windowing, trigger/emit:

Suggest we allow:

{:flow/from [:my-task :my-trigger]
;; or :flow/from-trigger :my-trigger
 :flow/to [:task1]}

When used, all other flows will be cut off from :my-task, even if :my-task is not named?

Breaking change here:
All all trigger/emit flows will be cut off if someone is currently using a flow condition + trigger/emit.

Issue 3:
Triggered segments should go through to output plugin on output tasks with windows.
Everything will look the same from users point of view, implementation tbd.

Better discussed in #819.