neulab/ExplainaBoard

Separate batched/non-batched operations in Metric

odashi opened this issue · 0 comments

Metric is implemented for both batched/non-batched operations and it is very annoying to organize overall behavior around Metric. Most functions need to switch its behavior with given argument, and it looks some functions involves untested bugs with unclear operations (e.g., the caller expects to return ndarray of shape [], but the function returns [1]).

It is better to implement Metric to calculate scores for only non-batched data, and operations for batched data should be synthesized from non-batched operations.