Reading Comprehension

Most current question answering datasets frame the task as reading comprehension where the question is about a paragraph or document and the answer often is a span in the document.

Some specific tasks of reading comprehension include multi-modal machine reading comprehension and textual machine reading comprehension, among others. In the literature, machine reading comprehension can be divide into four categories: cloze style, multiple choice, span prediction, and free-form answer. Read more about each category here.

Benchmark datasets used for testing a model's reading comprehension abilities include MovieQA, ReCoRD, and RACE, among others.

The Machine Reading group at UCL also provides an overview of reading comprehension tasks.

Figure source: A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1376–1400 of 1760 papers

Title	Date	Tasks	Status	Hype
Neural Models for Key Phrase Extraction and Question Generation	Jul 1, 2018	Question AnsweringQuestion Generation	—Unverified	0
Tackling Adversarial Examples in QA via Answer Sentence Selection	Jul 1, 2018	ArticlesQuestion Answering	—Unverified	0
Systematic Error Analysis of the Stanford Question Answering Dataset	Jul 1, 2018	Common Sense ReasoningMachine Reading Comprehension	—Unverified	0
Syntactic and Lexical Approaches to Reading Comprehension	Jul 1, 2018	Reading Comprehension	—Unverified	0
A Multi-Stage Memory Augmented Neural Network for Machine Reading Comprehension	Jul 1, 2018	Machine Reading ComprehensionQuestion Answering	—Unverified	0
Proceedings of the Workshop on Machine Reading for Question Answering	Jul 1, 2018	Question AnsweringReading Comprehension	—Unverified	0
Generating Questions for Reading Comprehension using Coherence Relations	Jul 1, 2018	Reading Comprehension	—Unverified	0
RECIPE: Applying Open Domain Question Answering to Privacy Policies	Jul 1, 2018	DescriptiveOpen-Domain Question Answering	—Unverified	0
Denoising Distantly Supervised Open-Domain Question Answering	Jul 1, 2018	DenoisingInformation Retrieval	CodeCode Available	0
Document Modeling with External Attention for Sentence Extraction	Jul 1, 2018	Answer SelectionDocument Summarization	CodeCode Available	0
Automatic Question Generation using Relative Pronouns and Adverbs	Jul 1, 2018	DescriptiveDialogue Generation	—Unverified	0
Exploring Semantic Properties of Sentence Embeddings	Jul 1, 2018	Machine TranslationReading Comprehension	—Unverified	0
Entity-Centric Joint Modeling of Japanese Coreference Resolution and Predicate Argument Structure Analysis	Jul 1, 2018	coreference-resolutionCoreference Resolution	—Unverified	0
Semantically Equivalent Adversarial Rules for Debugging NLP models	Jul 1, 2018	Data AugmentationQuestion Answering	CodeCode Available	0
Jack the Reader -- A Machine Reading Framework	Jul 1, 2018	ArticlesInformation Retrieval	CodeCode Available	0
A Spatial Model for Extracting and Visualizing Latent Discourse Structure in Text	Jul 1, 2018	Information RetrievalReading Comprehension	—Unverified	0
CNN for Text-Based Multiple Choice Question Answering	Jul 1, 2018	Multiple-choiceQuestion Answering	CodeCode Available	0
SANTO: A Web-based Annotation Tool for Ontology-driven Slot Filling	Jul 1, 2018	Knowledge Base PopulationReading Comprehension	—Unverified	0
Subword-augmented Embedding for Cloze Reading Comprehension	Jun 24, 2018	Machine Reading ComprehensionReading Comprehension	CodeCode Available	0
Jack the Reader - A Machine Reading Framework	Jun 20, 2018	ArticlesLink Prediction	CodeCode Available	1
Comparative Analysis of Neural QA models on SQuAD	Jun 18, 2018	Information RetrievalQuestion Answering	—Unverified	0
Know What You Don't Know: Unanswerable Questions for SQuAD	Jun 11, 2018	Natural Language UnderstandingQuestion Answering	CodeCode Available	1
A Co-Matching Model for Multi-choice Reading Comprehension	Jun 11, 2018	Reading Comprehension	CodeCode Available	0
Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Task	Jun 10, 2018	Machine Reading ComprehensionQuestion Answering	—Unverified	0
Learning to Search in Long Documents Using Document Structure	Jun 9, 2018	Information RetrievalQuestion Answering	CodeCode Available	0

Show:10 25 50

← PrevPage 56 of 71Next →

All datasets ReClor RACE MuSeRC AdversarialQA CrowdSource QA RadQA

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Rational Reasoner / IDOL	Test	80.6	—	Unverified
2	AMR-LE-Ensemble	Test	80	—	Unverified
3	MERIt(MERIt-deberta-v2-xxlarge )	Test	79.3	—	Unverified
4	MERIt-deberta-v2-xxlarge deberta.v2.xxlarge.path.override_True.norm_1.1.0.w2.A100.cp200.s42	Test	79.3	—	Unverified
5	Knowledge model	Test	79.2	—	Unverified
6	DeBERTa-v2-xxlarge-AMR-LE-Contraposition	Test	77.2	—	Unverified
7	LReasoner ensemble	Test	76.1	—	Unverified
8	ELECTRA and ALBERT	Test	71	—	Unverified
9	WWZ	Test	69.7	—	Unverified
10	xlnet-large-uncased [extended data]	Test	69.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALBERT (Ensemble)	Accuracy	91.4	—	Unverified
2	Megatron-BERT (ensemble)	Accuracy	90.9	—	Unverified
3	ALBERTxxlarge+DUMA(ensemble)	Accuracy	89.8	—	Unverified
4	Megatron-BERT	Accuracy	89.5	—	Unverified
5	XLNet	Accuracy (Middle)	88.6	—	Unverified
6	DeBERTalarge	Accuracy	86.8	—	Unverified
7	B10-10-10	Accuracy	85.7	—	Unverified
8	RoBERTa	Accuracy	83.2	—	Unverified
9	Orca 2-13B	Accuracy	82.87	—	Unverified
10	Orca 2-7B	Accuracy	80.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Golden Transformer	Average F1	0.94	—	Unverified
2	MT5 Large	Average F1	0.84	—	Unverified
3	ruRoberta-large finetune	Average F1	0.83	—	Unverified
4	ruT5-large-finetune	Average F1	0.82	—	Unverified
5	Human Benchmark	Average F1	0.81	—	Unverified
6	ruT5-base-finetune	Average F1	0.77	—	Unverified
7	ruBert-large finetune	Average F1	0.76	—	Unverified
8	ruBert-base finetune	Average F1	0.74	—	Unverified
9	RuGPT3XL few-shot	Average F1	0.74	—	Unverified
10	RuGPT3Large	Average F1	0.73	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RoBERTa-Large	Overall: F1	64.4	—	Unverified
2	BERT-Large	Overall: F1	62.7	—	Unverified
3	BiDAF	Overall: F1	28.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BERT	MSE	0.05	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BERT pretrained on MIMIC-III	Answer F1	63.55	—	Unverified