Question Answering
Question answering can be segmented into domain-specific tasks like community question answering and knowledge-base question answering. Popular benchmark datasets for evaluation question answering systems include SQuAD, HotPotQA, bAbI, TriviaQA, WikiQA, and many others. Models for question answering are typically evaluated on metrics like EM and F1. Some recent top performing models are T5 and XLNet.
( Image credit: SQuAD )
Papers
Showing 1–10 of 10817 papers
All datasetsSQuAD2.0SQuAD1.1HotpotQAPIQABoolQCOPATriviaQASQuAD1.1 devNatural QuestionsOpenBookQATruthfulQAMultiRC
Benchmark Results
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | GPT-4 + knowledge base | Accuracy | 95.9 | — | Unverified |
| 2 | MVP-Tuning (ensemble) | Accuracy | 95.2 | — | Unverified |
| 3 | PaLM 540B (Self Improvement, Self Consistency) | Accuracy | 94.4 | — | Unverified |
| 4 | X-Reasoner | Accuracy | 94.2 | — | Unverified |
| 5 | PaLM 540B (Self Improvement, CoT Prompting) | Accuracy | 93 | — | Unverified |
| 6 | PaLM 540B (Self Improvement, Standard-Prompting) | Accuracy | 92 | — | Unverified |
| 7 | DeBERTa-xxlarge 1.5B + MVP-Tuning | Accuracy | 91.3 | — | Unverified |
| 8 | PaLM 540B (Self Consistency) | Accuracy | 90 | — | Unverified |
| 9 | GrapeQA: PEGA+CANP | Accuracy | 90 | — | Unverified |
| 10 | GenMC 11B | Accuracy | 89.8 | — | Unverified |