Causal Covariate Shift Correction using Fisher information penalty

Abstract

Evolving feature densities across batches of training data bias cross-validation, making model selection and assessment unreliable (sugiyama2012machine). This work takes a distributed density estimation angle to the training setting where data are temporally distributed. Causal Covariate Shift Correction (C^3), accumulates knowledge about the data density of a training batch using Fisher Information, and using it to penalize the loss in all subsequent batches. The penalty improves accuracy by 12.9\% over the full-dataset baseline, by 20.3\% accuracy at maximum in batchwise and 5.9\% at minimum in foldwise benchmarks.

Tasks

Reproductions