m-mix: Generating hard negatives via multiple samples mixing for contrastive learning

Abstract

Negative pairs are essential in contrastive learning, which plays the role of avoiding degenerate solutions. Hard negatives can improve the representation ability on the basis of common negatives. Inspired by recent hard negative mining methods via mixup operation in vision, we propose m-mix, which generates hard negatives dynamically. Compared with previous methods, m-mix mainly has three advantages: 1) adaptively chooses samples to mix; 2) simultaneously mixes multiple samples; 3) automatically and comprehensively assigns different mixing weights to the selected mixing samples. We evaluate our method on two image classification datasets, five node classification datasets (PPI, DBLP, Pubmed, etc), five graph classification datasets (IMDB, PTC\_MR, etc), and downstream combinatorial tasks (graph edit distance and clustering). Results show that our method achieves state-of-the-art performance on most benchmarks under self-supervised settings.

Tasks

Reproductions