Reading Comprehension

Most current question answering datasets frame the task as reading comprehension where the question is about a paragraph or document and the answer often is a span in the document.

Some specific tasks of reading comprehension include multi-modal machine reading comprehension and textual machine reading comprehension, among others. In the literature, machine reading comprehension can be divide into four categories: cloze style, multiple choice, span prediction, and free-form answer. Read more about each category here.

Benchmark datasets used for testing a model's reading comprehension abilities include MovieQA, ReCoRD, and RACE, among others.

The Machine Reading group at UCL also provides an overview of reading comprehension tasks.

Figure source: A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1601–1625 of 1760 papers

Title	Date	Tasks	Status	Hype
Machine Comprehension Using Match-LSTM and Answer Pointer	Aug 29, 2016	Natural Language InferenceQuestion Answering	CodeCode Available	0
Hierarchical Attention Model for Improved Machine Comprehension of Spoken Content	Aug 28, 2016	Reading Comprehension	—Unverified	0
Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine	Aug 23, 2016	Reading ComprehensionSentence	—Unverified	0
Who did What: A Large-Scale Person-Centered Cloze Dataset	Aug 19, 2016	ArticlesMultiple-choice	—Unverified	0
Focus Annotation of Task-based Data: Establishing the Quality of Crowd Annotation	Aug 1, 2016	Reading Comprehension	—Unverified	0
Do We Really Need All Those Rich Linguistic Features? A Neural Network-Based Approach to Implicit Sense Labeling	Aug 1, 2016	AllReading Comprehension	CodeCode Available	0
Approximating Givenness in Content Assessment through Distributional Semantics	Aug 1, 2016	Reading Comprehension	—Unverified	0
Learning to Jointly Predict Ellipsis and Comparison Structures	Aug 1, 2016	Question AnsweringReading Comprehension	—Unverified	0
Specifying and Annotating Reduced Argument Span Via QA-SRL	Aug 1, 2016	Knowledge Base PopulationReading Comprehension	—Unverified	0
Creating Interactive Macaronic Interfaces for Language Learning	Aug 1, 2016	Language AcquisitionMachine Translation	—Unverified	0
Controlled and Balanced Dataset for Japanese Lexical Simplification	Aug 1, 2016	Lexical SimplificationReading Comprehension	CodeCode Available	0
Machine Comprehension using Rich Semantic Representations	Aug 1, 2016	Natural Language InferenceReading Comprehension	—Unverified	0
Attention-over-Attention Neural Networks for Reading Comprehension	Jul 15, 2016	Question AnsweringReading Comprehension	CodeCode Available	0
Separating Answers from Queries for Neural Reading Comprehension	Jul 12, 2016	Reading ComprehensionRetrieval	—Unverified	0
Consensus Attention-based Neural Networks for Chinese Reading Comprehension	Jul 8, 2016	Chinese Reading ComprehensionReading Comprehension	—Unverified	0
SQuAD: 100,000+ Questions for Machine Comprehension of Text	Jun 16, 2016	ArticlesQuestion Answering	CodeCode Available	1
Dialog state tracking, a machine reading approach using Memory Network	Jun 13, 2016	dialog state trackingManagement	—Unverified	0
A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task	Jun 9, 2016	ArticlesReading Comprehension	CodeCode Available	0
Natural Language Comprehension with the EpiReader	Jun 7, 2016	Question AnsweringReading Comprehension	—Unverified	0
Iterative Alternating Neural Attention for Machine Reading	Jun 7, 2016	ArticlesQuestion Answering	CodeCode Available	0
Generating and Exploiting Large-scale Pseudo Training Data for Zero Pronoun Resolution	Jun 6, 2016	Reading Comprehension	—Unverified	0
Gated-Attention Readers for Text Comprehension	Jun 5, 2016	Answer SelectionOpen-Domain Question Answering	CodeCode Available	0
Investigating Active Learning for Short-Answer Scoring	Jun 1, 2016	Active LearningReading Comprehension	—Unverified	0
Automatic Generation of Context-Based Fill-in-the-Blank Exercises Using Co-occurrence Likelihoods and Google n-grams	Jun 1, 2016	Reading Comprehension	—Unverified	0
Exploring the Intersection of Short Answer Assessment, Authorship Attribution, and Plagiarism Detection	Jun 1, 2016	Authorship AttributionReading Comprehension	—Unverified	0

Show:10 25 50

← PrevPage 65 of 71Next →

All datasets ReClor RACE MuSeRC AdversarialQA CrowdSource QA RadQA

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Rational Reasoner / IDOL	Test	80.6	—	Unverified
2	AMR-LE-Ensemble	Test	80	—	Unverified
3	MERIt(MERIt-deberta-v2-xxlarge )	Test	79.3	—	Unverified
4	MERIt-deberta-v2-xxlarge deberta.v2.xxlarge.path.override_True.norm_1.1.0.w2.A100.cp200.s42	Test	79.3	—	Unverified
5	Knowledge model	Test	79.2	—	Unverified
6	DeBERTa-v2-xxlarge-AMR-LE-Contraposition	Test	77.2	—	Unverified
7	LReasoner ensemble	Test	76.1	—	Unverified
8	ELECTRA and ALBERT	Test	71	—	Unverified
9	WWZ	Test	69.7	—	Unverified
10	xlnet-large-uncased [extended data]	Test	69.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALBERT (Ensemble)	Accuracy	91.4	—	Unverified
2	Megatron-BERT (ensemble)	Accuracy	90.9	—	Unverified
3	ALBERTxxlarge+DUMA(ensemble)	Accuracy	89.8	—	Unverified
4	Megatron-BERT	Accuracy	89.5	—	Unverified
5	XLNet	Accuracy (Middle)	88.6	—	Unverified
6	DeBERTalarge	Accuracy	86.8	—	Unverified
7	B10-10-10	Accuracy	85.7	—	Unverified
8	RoBERTa	Accuracy	83.2	—	Unverified
9	Orca 2-13B	Accuracy	82.87	—	Unverified
10	Orca 2-7B	Accuracy	80.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Golden Transformer	Average F1	0.94	—	Unverified
2	MT5 Large	Average F1	0.84	—	Unverified
3	ruRoberta-large finetune	Average F1	0.83	—	Unverified
4	ruT5-large-finetune	Average F1	0.82	—	Unverified
5	Human Benchmark	Average F1	0.81	—	Unverified
6	ruT5-base-finetune	Average F1	0.77	—	Unverified
7	ruBert-large finetune	Average F1	0.76	—	Unverified
8	ruBert-base finetune	Average F1	0.74	—	Unverified
9	RuGPT3XL few-shot	Average F1	0.74	—	Unverified
10	RuGPT3Large	Average F1	0.73	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RoBERTa-Large	Overall: F1	64.4	—	Unverified
2	BERT-Large	Overall: F1	62.7	—	Unverified
3	BiDAF	Overall: F1	28.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BERT	MSE	0.05	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BERT pretrained on MIMIC-III	Answer F1	63.55	—	Unverified