SOTAVerified

Ensembling Graph Predictions for AMR Parsing

2021-10-18NeurIPS 2021Code Available1· sign in to hype

Hoang Thanh Lam, Gabriele Picco, Yufang Hou, Young-suk Lee, Lam M. Nguyen, Dzung T. Phan, Vanessa López, Ramon Fernandez Astudillo

Code Available — Be the first to reproduce this paper.

Reproduce

Code

Abstract

In many machine learning tasks, models are trained to predict structure data such as graphs. For example, in natural language processing, it is very common to parse texts into dependency trees or abstract meaning representation (AMR) graphs. On the other hand, ensemble methods combine predictions from multiple models to create a new one that is more robust and accurate than individual predictions. In the literature, there are many ensembling techniques proposed for classification or regression problems, however, ensemble graph prediction has not been studied thoroughly. In this work, we formalize this problem as mining the largest graph that is the most supported by a collection of graph predictions. As the problem is NP-Hard, we propose an efficient heuristic algorithm to approximate the optimal solution. To validate our approach, we carried out experiments in AMR parsing problems. The experimental results demonstrate that the proposed approach can combine the strength of state-of-the-art AMR parsers to create new predictions that are more accurate than any individual models in five standard benchmark datasets.

Tasks

Benchmark Results

DatasetModelMetricClaimedVerifiedStatus
BioGraphene SmatchSmatch62.8Unverified
LDC2017T10Graphene Smatch (IBM)Smatch86.26Unverified
LDC2017T10Graphene Support (IBM)Smatch85.85Unverified
LDC2020T02Graphene Smatch (IBM)Smatch84.87Unverified
LDC2020T02Graphene Support (IBM)Smatch84.41Unverified
New3Graphene SmatchSmatch76.32Unverified
The Little PrinceGraphene SmatchSmatch79.52Unverified

Reproductions