IntelLabs/AVUC

Brier and ECE functions

RiqiangGao opened this issue · 4 comments

RiqiangGao commented

Dear author,

I found a few codes lines commented, for example,

#brier = np.mean(calib.brier_scores(target_labels, probs=pred_mean))
#ece = calib.expected_calibration_error_multiclass(pred_mean, target_labels)

which is also useful to me. Could I ask where can find the related functions?

Thanks in advance.

ranganathkrishnan commented

Below are pointers to Brier and ECE functions from 'uncertainty-metrics' repo:
brier_score
ECE

RiqiangGao commented

Great! thank you a lot.

  • 👍1
ranganathkrishnan commented

metrics_lib from uq_benchmark might also be helpful

RiqiangGao commented

Very appreciate