SOTAVerified

Temporal Knowledge Base Completion: New Algorithms and Evaluation Protocols

2020-05-02EMNLP 2020Code Available0· sign in to hype

Prachi Jain, Sushant Rathi, Mausam, Soumen Chakrabarti

Code Available — Be the first to reproduce this paper.

Reproduce

Code

Abstract

Temporal knowledge bases associate relational (s,r,o) triples with a set of times (or a single time instant) when the relation is valid. While time-agnostic KB completion (KBC) has witnessed significant research, temporal KB completion (TKBC) is in its early days. In this paper, we consider predicting missing entities (link prediction) and missing time intervals (time prediction) as joint TKBC tasks where entities, relations, and time are all embedded in a uniform, compatible space. We present TIMEPLEX, a novel time-aware KBC method, that also automatically exploits the recurrent nature of some relations and temporal interactions between pairs of relations. TIMEPLEX achieves state-of-the-art performance on both prediction tasks. We also find that existing TKBC models heavily overestimate link prediction performance due to imperfect evaluation mechanisms. In response, we propose improved TKBC evaluation protocols for both link and time prediction tasks, dealing with subtle issues that arise from the partial overlap of time intervals in gold instances and system predictions.

Tasks

Benchmark Results

DatasetModelMetricClaimedVerifiedStatus
ICEWS05-15TimePlexMRR0.63Unverified
ICEWS14TimePlexMRR0.59Unverified
Wikidata12kTimePlexMRR0.33Unverified
Yago11kTimePlexMRR0.24Unverified

Reproductions