Reading Comprehension

Most current question answering datasets frame the task as reading comprehension where the question is about a paragraph or document and the answer often is a span in the document.

Some specific tasks of reading comprehension include multi-modal machine reading comprehension and textual machine reading comprehension, among others. In the literature, machine reading comprehension can be divide into four categories: cloze style, multiple choice, span prediction, and free-form answer. Read more about each category here.

Benchmark datasets used for testing a model's reading comprehension abilities include MovieQA, ReCoRD, and RACE, among others.

The Machine Reading group at UCL also provides an overview of reading comprehension tasks.

Figure source: A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 876–900 of 1760 papers

Title	Date	Tasks	Status	Hype
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark	Oct 29, 2020	Common Sense ReasoningDiagnostic	CodeCode Available	1
Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation	Oct 27, 2020	Knowledge DistillationMachine Reading Comprehension	—Unverified	0
QBSUM: a Large-Scale Query-Based Document Summarization Dataset from Real-world Applications	Oct 27, 2020	Document SummarizationMachine Reading Comprehension	—Unverified	0
Commonsense knowledge adversarial dataset that challenges ELECTRA	Oct 25, 2020	Reading ComprehensionWord Sense Disambiguation	—Unverified	0
Improved Synthetic Training for Reading Comprehension	Oct 24, 2020	Knowledge DistillationMachine Reading Comprehension	—Unverified	0
Towards Zero-Shot Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehension	Oct 22, 2020	Answer GenerationCross-Lingual Question Answering	—Unverified	0
Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval	Oct 22, 2020	answerability predictionLanguage Modelling	—Unverified	0
mT5: A massively multilingual pre-trained text-to-text transformer	Oct 22, 2020	Common Sense ReasoningNatural Language Inference	CodeCode Available	1
Probing and Fine-tuning Reading Comprehension Models for Few-shot Event Extraction	Oct 21, 2020	Event DetectionEvent Extraction	—Unverified	0
RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering	Oct 21, 2020	Machine Reading ComprehensionNatural Questions	CodeCode Available	1
Knowledge Distillation for Improved Accuracy in Spoken Question Answering	Oct 21, 2020	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified	0
Bi-directional Cognitive Thinking Network for Machine Reading Comprehension	Oct 20, 2020	Machine Reading ComprehensionReading Comprehension	—Unverified	0
Deriving Commonsense Inference Tasks from Interactive Fictions	Oct 19, 2020	Reading Comprehension	—Unverified	0
Technical Question Answering across Tasks and Domains	Oct 19, 2020	Question AnsweringReading Comprehension	CodeCode Available	0
Towards Interpreting BERT for Reading Comprehension Based QA	Oct 18, 2020	Question AnsweringReading Comprehension	CodeCode Available	0
A Wrong Answer or a Wrong Question? An Intricate Relationship between Question Reformulation and Answer Selection in Conversational Question Answering	Oct 13, 2020	Answer SelectionConversational Question Answering	CodeCode Available	0
Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension	Oct 13, 2020	Machine Reading ComprehensionReading Comprehension	—Unverified	0
Multi-Stage Pre-training for Low-Resource Domain Adaptation	Oct 12, 2020	Document RankingDomain Adaptation	—Unverified	0
Open-Domain Question Answering Goes Conversational via Question Rewriting	Oct 10, 2020	Conversational Question AnsweringOpen-Domain Question Answering	CodeCode Available	1
Counterfactually-Augmented SNLI Training Data Does Not Yield Better Generalization Than Unaugmented Data	Oct 9, 2020	counterfactualNatural Language Inference	CodeCode Available	0
MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics	Oct 7, 2020	Question AnsweringReading Comprehension	CodeCode Available	1
PolicyQA: A Reading Comprehension Dataset for Privacy Policies	Oct 6, 2020	Question AnsweringReading Comprehension	CodeCode Available	1
Context Modeling with Evidence Filter for Multiple Choice Question Answering	Oct 6, 2020	Machine Reading ComprehensionMultiple-choice	—Unverified	0
Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning	Oct 5, 2020	Reading Comprehensionreinforcement-learning	CodeCode Available	1
Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading	Oct 5, 2020	Decision MakingDiscourse Segmentation	CodeCode Available	1

Show:10 25 50

← PrevPage 36 of 71Next →

All datasets ReClor RACE MuSeRC AdversarialQA CrowdSource QA RadQA

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Rational Reasoner / IDOL	Test	80.6	—	Unverified
2	AMR-LE-Ensemble	Test	80	—	Unverified
3	MERIt(MERIt-deberta-v2-xxlarge )	Test	79.3	—	Unverified
4	MERIt-deberta-v2-xxlarge deberta.v2.xxlarge.path.override_True.norm_1.1.0.w2.A100.cp200.s42	Test	79.3	—	Unverified
5	Knowledge model	Test	79.2	—	Unverified
6	DeBERTa-v2-xxlarge-AMR-LE-Contraposition	Test	77.2	—	Unverified
7	LReasoner ensemble	Test	76.1	—	Unverified
8	ELECTRA and ALBERT	Test	71	—	Unverified
9	WWZ	Test	69.7	—	Unverified
10	xlnet-large-uncased [extended data]	Test	69.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALBERT (Ensemble)	Accuracy	91.4	—	Unverified
2	Megatron-BERT (ensemble)	Accuracy	90.9	—	Unverified
3	ALBERTxxlarge+DUMA(ensemble)	Accuracy	89.8	—	Unverified
4	Megatron-BERT	Accuracy	89.5	—	Unverified
5	XLNet	Accuracy (Middle)	88.6	—	Unverified
6	DeBERTalarge	Accuracy	86.8	—	Unverified
7	B10-10-10	Accuracy	85.7	—	Unverified
8	RoBERTa	Accuracy	83.2	—	Unverified
9	Orca 2-13B	Accuracy	82.87	—	Unverified
10	Orca 2-7B	Accuracy	80.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Golden Transformer	Average F1	0.94	—	Unverified
2	MT5 Large	Average F1	0.84	—	Unverified
3	ruRoberta-large finetune	Average F1	0.83	—	Unverified
4	ruT5-large-finetune	Average F1	0.82	—	Unverified
5	Human Benchmark	Average F1	0.81	—	Unverified
6	ruT5-base-finetune	Average F1	0.77	—	Unverified
7	ruBert-large finetune	Average F1	0.76	—	Unverified
8	ruBert-base finetune	Average F1	0.74	—	Unverified
9	RuGPT3XL few-shot	Average F1	0.74	—	Unverified
10	RuGPT3Large	Average F1	0.73	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RoBERTa-Large	Overall: F1	64.4	—	Unverified
2	BERT-Large	Overall: F1	62.7	—	Unverified
3	BiDAF	Overall: F1	28.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BERT	MSE	0.05	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BERT pretrained on MIMIC-III	Answer F1	63.55	—	Unverified