Disaggregating Hops: Can We Guide a Multi-Hop Reasoning Language Model to Incrementally Learn at each Hop?

Abstract

Despite the success of state-of-the-art pre-trained language models (PLMs) on a series of multi-hop reasoning tasks, they still suffer from their limited abilities to transfer learning from simple to complex tasks and vice-versa. We argue that one step forward to overcome this limitation is to better understand the behavioral trend of PLMs at each hop over the inference chain. Our critical underlying idea is to mimic human-style reasoning: we envision the multi-hop reasoning process as a sequence of explicit one-hop incremental reasoning steps. Using the SHINRA and ConceptNet resources jointly, we provide automatically generated datasets built upon a set of inference heuristics on relevant phrases and distractors, allowing us to teach the models incremental reasoning skills. We empirically show the effectiveness of the proposed models on multiple-choice question answering (MCQA) and reading comprehension (RC), with a relative improvement of 68.4\% and 16.0\% accuracy improvement w.r.t. classic PLMs, respectively.

Tasks

Reproductions