Extreme Summarization
Image credit: TLDR: Extreme Summarization of Scientific Documents
Papers
Showing 21–22 of 22 papers
Benchmark Results
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | EXT-ORACLE | ROUGE-1 | 44.17 | — | Unverified |
| 2 | BART-large (s=abs+title, t=TLDR) | ROUGE-1 | 42.02 | — | Unverified |
| 3 | BART-large (s=abs+disci, t=TLDR) | ROUGE-1 | 42.01 | — | Unverified |
| 4 | BART-large (s=abs, t= TLDR/Title/Disci) | ROUGE-1 | 41.89 | — | Unverified |
| 5 | BART-large (s=abs, t=TLDR) | ROUGE-1 | 41.86 | — | Unverified |
| 6 | BART-large (s=abs, t=TLDR/title) | ROUGE-1 | 41.85 | — | Unverified |
| 7 | PEGASUS (s=abs, t=TLDR) | ROUGE-1 | 41.56 | — | Unverified |
| 8 | EXT-HEURISTIC | ROUGE-1 | 29.32 | — | Unverified |
| 9 | EXT-LEAD | ROUGE-1 | 21.94 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | PEGASUS | ROUGE-2 | 23.2 | — | Unverified |
| 2 | PaLM (finetuning)-540B | ROUGE-2 | 21.2 | — | Unverified |
| 3 | T5-XXL | ROUGE-2 | 21 | — | Unverified |
| 4 | PaLM (finetuning)-62B | ROUGE-2 | 18.5 | — | Unverified |
| 5 | ByT5 | BLEU score | 15.3 | — | Unverified |
| 6 | mT5 | BLEU score | 14.3 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | ORACLE-EXT | RG-1(%) | 30.26 | — | Unverified |
| 2 | BART | RG-1(%) | 23.59 | — | Unverified |
| 3 | BERTSUMABS | RG-1(%) | 23.05 | — | Unverified |
| 4 | BERTSUMEXT | RG-1(%) | 20.94 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | PEGASUS | METEOR | 0.22 | — | Unverified |