Streaming jobs do not cumulate datasets sent through a run
pawel-big-lebowski opened this issue · 0 comments
pawel-big-lebowski commented
I observed a streaming job that was correctly emitting OL event with input and output datasets which lead to a nice lineage graph. However, a single fail event at the end (which did not have any input / output datasets) broke the graph.
TODO:
- write an integration test which emits streaming job OL event with inputs/outputs
- emit OL event for the same run with no input / output datasets
- make sure API responds with lineage connecting the datasets