MAFiD: Moving Average Equipped Fusion-in-Decoder for Question Answering over Tabular and Textual Data

Code

• github.com/ZIZUN/MAFiD
  pytorch★ 2

Abstract

Transformer-based models for question answering (QA) over tables and texts confront a “long” hybrid sequence over tabular and textual elements, causing long-range reasoning problems. To handle long-range reasoning, we extensively employ a fusion-in-decoder (FiD) and exponential moving average (EMA), proposing a underlineMoving underlineAverage Equipped underlineFusion-underlinein-underlineDecoder (textbfMAFiD). With FiD as the backbone architecture, MAFiD combines various levels of reasoning: textitindependent encoding of homogeneous data and textitsingle-row and textitmulti-row heterogeneous reasoning, using a textitgated cross attention layer to effectively aggregate the three types of representations resulting from various reasonings. Experimental results on HybridQA indicate that MAFiD achieves state-of-the-art performance by increasing exact matching (EM) and F1 by 1.1 and 1.7, respectively, on the blind test set.

Tasks

Reproductions