mozilla/PRESC

Management for config options

Closed this issue · 0 comments

Many components of the PRESC computations should be customizable by a user running the tools in their own environment. Some options are currently passed around as function args, but we need a more unified way of setting config options that doesn't require code changes, eg. via a config file.

Some of the components that will be configured via options are:

  • The evaluation computations, eg. the binning scheme to use for grouping test samples
  • Plot components, eg. element sizing or plot arrangement
  • How an evaluation applies to a dataset, eg. some would be run per-feature, and we'd want to include or exclude certain features. Also, some of the per-evaluation options may be allowed to vary per-feature in this case.
  • The report generation, eg. which evaluations to include