Class Embeddings for Improved Out-of-Scope Detection in Intent Classification

Abstract

This work presents a deep investigation of class embeddings for intent classification, which had only been timidly addressed before. We show that there are advantages in representing classes as dense vectors instead of one-hot-encoded ones, similar to the way sentence embeddings became the standard representation instead of one-hot-encoding of bags of words. Some of our results corroborate previous findings showing that class embeddings can be a better approach to deal with out-of-scope examples. But this paper goes further by arguing that there are gains by simply changing from the traditional one-hot-encoding with softmax to one-hot embeddings. We also show that there is a lot of room for the better use of class embeddings by exploring random embeddings. The results suggest that optimal class embeddings could exist, with significant improvement on classification metrics, particularly out-of-scope detection. Additionally, we observe that the dimension of the class embedding might depend on the objective function, and that can open up for research on new neural network architectures to make better use of class embeddings.

Tasks

Reproductions