Multi-class probabilistic classification using inductive and cross Venn-Abers predictors

Code

• github.com/valeman/Multi-class-probabilistic-classification
  Officialnone★ 50

Abstract

Inductive (IVAP) and cross (CVAP) Venn–Abers predictors are computationally efficient algorithms for probabilistic prediction in binary classification problems. We present a new approach to multi-class probability estimation by turning IVAPs and CVAPs into multi-class probabilistic predictors. The proposed multi-class predictors are experimentally more accurate than both uncalibrated predictors and existing calibration methods. Unlike other methods such as Platt's scaler and Isotonic Regression Venn-ABERS predictors (being a form of conformal prediction) contain inbuilt mathematical guarantees of validity (lack of bias). In addition Venn-ABERS predictors are multi-output predictors that output two probability predictions of class 1. Such two probability prediction of assigning label 1 for each test objectc can be considered prediction interval. The interval width contains valuable information about degree of certainty of prediction, such information is not available when using other calibration methods such as Platt's and isotonic regression.

Tasks

Reproductions