Different OOS evaluation metrics, user specified
Opened this issue · 3 comments
crahal commented
Pseudo R2 is the natural candidate as it applies to probabilities as opposed to categories for binary outcomes, and is nicely interpretable. Ideally a suite of things that work for both binary and continuous DVs are preferable, but we could have a flag to check and catch warn then revert to a default if the specified option isn't amenable.
dhvalden commented
to me: This means Out of Sample Evaluation Metric
crahal commented
Metrics to include:
- Default to Pseudo-R2
- Include RMSE
- Include Cross Entropy Loss.
Include in results object a string which can be used for x-axis label of the out of sample plot (top right).