Logit As Auxiliary Weak-supervision for More Reliable and Accurate Prediction

Abstract

When a person identifies objects, he or she can think by associating objects to many classes and conclude by taking inter-class relations into account. This cognitive system can make a more reliable prediction. Inspired by these observations, we propose a new network training strategy to consider inter-class relations, namely LogitMix. Specifically, we use recent data augmentation techniques (e.g., Mixup, Manifold Mixup, or Cutmix) as baselines for generating mixed samples. Then, LogitMix suggests using the mixed logit (ie., the mixture of two logits) as an auxiliary training objective. Because using logit before softmax activation preserves rich class relationships, it can serve as a weak-supervision signal concerning inter-class relations. Our experimental results demonstrate that LogitMix achieves state-of-the-art performance among recent data augmentation techniques in terms of both calibration error and prediction accuracy. The source code is attached as the supplementary material.

Tasks

Reproductions