Automatically create a command for every `Metric`
danieldeutsch opened this issue · 1 comments
It would be nice to have some command like sacrerouge rouge --n 3
or sacrerouge evaluate rouge --no-stemmer
that would run one specific metric and allow the options for that metric to be passed on the command line instead of through a config file. I think this would enhance the usability of the library.
It would be even nicer if this command could be automatically created for any object that inherits Metric
so every metric will get it for free. I believe this should be doable by inspecting the class's constructor and creating an option for every parameter. Each command line argument would be a json object that can be deserialized into the type of the corresponding parameter. Only parameters with non-default parameters would be required to pass on the command line.
Commands are mostly updated with auto-generation for each metric/dataset reader. Working on updating run methods to use argparse results rather than config files.