Make log_valid_metrics, checkpoint_metric, and validation metric compatible
jdunnmon opened this issue · 1 comments
jdunnmon commented
Currently, the user has to set each of these individually, and if they don't set log_validation_metrics
their Tuner
will validate on accuracy by default, even if they've specified something else. This should be handled automatically such that anything in checkpoint_metric
is automatically propagated to log_validation_metrics
when the Tuner
is used.
bhancock8 commented
I think the best solution will be to import the new logger and logger settings from the mmtl branch (with maybe just minor modification so that it can handle metrics that aren't prefixed with a task name for the common single-task setting). That code should be coming into master within the month.