qdrant/quaterion

[metrics] Handle metrics updates inside TrainableModel

Closed this issue · 1 comments

Currently it is required to write metrics update logic manually in the custom implementation of TrainableModel.
But there is a large group of metrics which are updated in similar way, therefore it is possible to relieve the user from it.

Proposition

Implement metrics configuration similar to loss - create an abstract function in the base class configure_metrics.
If re-refined configure_metrics returns non-empty dict, TrainableModel should register metric updates in related hooks

joein commented

Currently, every metric computes distance matrix each time compute method is called, might be reasonable to introduce some kind of cache inside a metric and some kind of metrics collection to share distance matrix between them