Calibrate probabilities for multiclass classification?

[karllandheer]

Hello, is this package able to calibrate probabilities for multiclass classification? If not do you know how this can be achieved? Thank you very much in advance for your great package

[ptocca]

Hello, thanks for your question!
Venn-ABERS predictors are specifically for binary classification and my package therefore supports only that type of classification.
I have not thought long enough to give a proper answer for the multiclass case, but I guess that a possible approach might be to set up C classification problems (where by C I denote the number of classes) of the type one-class-vs-the-rest. You could run the calibration on each of the C classification scores separately and get calibrated probabilities for each label.
However I do not think there would be theoretical guarantees that the sum of the probabilities for the C labels of an object would always be exactly 1.