transportenergy/database

Allow multi-step processing

Opened this issue · 1 comments

The initial processing scripts for input data set have a “1-step” design pattern: a single function process() does all the work.

This must be extended to allow multiple layers of processing steps, forming a flowchart, graph, or network of processing steps that can be arbitrarily long. This is necessary because, for instance, the step of harmonizing or joining 2 or more data sets must be kept distinct from the step of cleaning/adapting the input data for each one.

This graph pattern is already implemented by other software, e.g. ixmp.reporting—code should be reused instead of reimplemented.

This graph pattern is already implemented by other software, e.g. ixmp.reporting

This is now genno.