CalibrationErrors.jl
Estimation of calibration errors.
Overview
This package implements different estimators of the expected calibration error (ECE) and the squared kernel calibration error calibration error (SKCE) in the Julia language.
Example
Calibration errors can be estimated from a data set of predicted probabilities and and a set of targets by executing
calibrationerror(estimator, predictions, targets)
The predictions can be provided as a vector of n
vectors of predicted probabilities
of length m
or as a matrix of size (m, n)
, in which each of the n
columns corresponds
to predicted probabilities of the targets 1,…,m
. The corresponding targets have to be
provided as a vector of length n
, in which every element is from the set 1,…,m
.
Alternatively, it is possible to specify a tuple of predictions and targets or a vector of tuples of predictions and targets.
This package implements the estimator ECE
of the ECE and the estimators
BiasedSKCE
, UnbiasedSKCE
, and BlockUnbiasedSKCE
for the SKCE.