TempQuestions: A Benchmark for Temporal Question Answering
Jia, Zhen and Abujabal, Abdalghani and Saha Roy, Rishiraj and Str\"{o}tgen, Jannik and Weikum, Gerhard
Unverified — Be the first to reproduce this paper.
ReproduceAbstract
Answering complex questions is one of the challenges that question-answering (QA) systems face today. While complexity has several facets, question dimensions like temporal and spatial intents necessitate specialized treatment. Methods geared towards such questions need benchmarks that reflect the desired aspects and challenges. Here, we take a key step in this direction, and release a new benchmark, TempQuestions, containing 1,271 questions, that are all temporal in nature, paired with their answers. As a key contribution that enabled the creation of this resource, we provide a crisp definition for temporal questions. Most questions require decomposing them into sub-questions, and the questions are of a kind that they would be best evaluated on a combination of structured data and unstructured text sources. Experiments with two QA systems demonstrate the need for further research on complex questions.