Empirical Evaluation of Topic Zero- and Few-Shot Learning for Stance Dissonance Detection

Abstract

We address stance dissonance detection, the task of detecting conflicting stance between two input statements. Computational models for traditional stance detection have typically been trained to indicate pro/cons for a given target topic (e.g. gun control) and thus do not generalize well to new topics. In this paper, we systematically evaluate the generalizability of this task to situations where examples of the topic have not been seen at all (zero-shot) or only a few times (few-shot). We first build a large-scale dataset of stance dissonance detection from an online debate platform, consisting of 23.8k pairs of statements from 34 diverse topics. We show that stance dissonance detection models trained only on a small number of non-target topics already perform as well as those trained on a target topic. We also show that adding more non-target topics further boosts the performance, indicating the generalizability of non-target topics to a target topic in the stance dissonance detection task.

Tasks

Reproductions