Expected-Calibration-Error

ece_score(py, y_test, n_bins=10)

'py' is the output of your model's predict. py = model.predict(X) y_test is X's label n_bins is the numbers of bins.

The paper: On Calibration of Modern Neural Networks https://arxiv.org/pdf/1706.04599.pdf