SchlossLab/mikropml

Precision should be 1 when TP=1 and FP=0

Opened this issue · 0 comments

caret uses MLmetrics::PRAUC(), which reports precision as NA when TP=1 and FP=0. As a result, the prAUC value reported by caret is overinflated because it drops NA values.

Solution: use yardstick::pr_auc(), which uses the convention that precision is 1 when TP=1 and FP=0.

See tidymodels/yardstick#166 & tidymodels/yardstick#95