Data-to-text Generation with Macro Planning
Ratish Puduppully, Mirella Lapata
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/ratishsp/data2text-macro-plan-pyOfficialIn paperpytorch★ 23
Abstract
Recent approaches to data-to-text generation have adopted the very successful encoder-decoder architecture or variants thereof. These models generate text which is fluent (but often imprecise) and perform quite poorly at selecting appropriate content and ordering it coherently. To overcome some of these issues, we propose a neural model with a macro planning stage followed by a generation stage reminiscent of traditional methods which embrace separate modules for planning and surface realization. Macro plans represent high level organization of important content such as entities, events and their interactions; they are learnt from data and given as input to the generator. Extensive experiments on two data-to-text benchmarks (RotoWire and MLB) show that our approach outperforms competitive baselines in terms of automatic and human evaluation.
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| MLB Dataset | Macro | BLEU | 12.62 | — | Unverified |
| MLB Dataset (Content Ordering) | Macro | DLD | 21.8 | — | Unverified |
| MLB Dataset (Content Ordering) | ENT | DLD | 20.7 | — | Unverified |
| MLB Dataset (Content Selection) | Macro | Precision | 40.8 | — | Unverified |
| MLB Dataset (Relation Generation) | ENT | Precision | 81.1 | — | Unverified |
| MLB Dataset (Relation Generation) | Macro | Precision | 94.4 | — | Unverified |
| RotoWire | Macro | BLEU | 15.46 | — | Unverified |
| RotoWire (Content Ordering) | Macro | DLD | 17.7 | — | Unverified |
| Rotowire (Content Selection) | Macro | Precision | 34.1 | — | Unverified |
| RotoWire (Relation Generation) | Macro | Precision | 97.6 | — | Unverified |