Communication Dans Un Congrès Année : 2024

Post-Calibration Techniques: Balancing Calibration and Score Distribution Alignment

Résumé

A binary scoring classifier can appear well-calibrated according to standard calibration metrics, even when the distribution of scores does not align with the distribution of the true events. In this paper, we investigate the impact of postprocessing calibration on the score distribution (sometimes named "recalibration"). Using simulated data, where the true probability is known, followed by real-world datasets with prior knowledge on event distributions, we compare the performance of an XGBoost model before and after applying calibration techniques. The results show that while applying methods such as Platt scaling, Beta calibration, or isotonic regression can improve the model's calibration, they may also lead to an increase in the divergence between the score distribution and the underlying event probability distribution.
Fichier principal
Vignette du fichier
Calib_NeurIPS_W_Bayesian-4.pdf (524) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04916151 , version 1 (11-02-2025)

Identifiants

  • HAL Id : hal-04916151 , version 1

Citer

Agathe Fernandes Machado, Arthur Charpentier, Emmanuel Flachaire, Ewen Gallic, François Hu. Post-Calibration Techniques: Balancing Calibration and Score Distribution Alignment. 38th Conference on Neural Information Processing Systems (NeurIPS 2024), May 2024, San Diego, United States. ⟨hal-04916151⟩
0 Consultations
0 Téléchargements

Partager

More