Reading Comprehension

Most current question answering datasets frame the task as reading comprehension where the question is about a paragraph or document and the answer often is a span in the document.

Some specific tasks of reading comprehension include multi-modal machine reading comprehension and textual machine reading comprehension, among others. In the literature, machine reading comprehension can be divide into four categories: cloze style, multiple choice, span prediction, and free-form answer. Read more about each category here.

Benchmark datasets used for testing a model's reading comprehension abilities include MovieQA, ReCoRD, and RACE, among others.

The Machine Reading group at UCL also provides an overview of reading comprehension tasks.

Figure source: A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1551–1575 of 1760 papers

Title	Date	Tasks	Status	Hype
Resource-Lean Modeling of Coherence in Commonsense Stories	Apr 1, 2017	Cloze TestReading Comprehension	—Unverified	0
Improving Neural Knowledge Base Completion with Cross-Lingual Projections	Apr 1, 2017	Knowledge Base CompletionLink Prediction	—Unverified	0
Learning and Knowledge Transfer with Memory Networks for Machine Comprehension	Apr 1, 2017	Question AnsweringReading Comprehension	—Unverified	0
Which is the Effective Way for Gaokao: Information Retrieval or Neural Networks?	Apr 1, 2017	Information RetrievalMultiple-choice	CodeCode Available	0
Reading Wikipedia to Answer Open-Domain Questions	Mar 31, 2017	ArticlesOpen-Domain Question Answering	CodeCode Available	1
Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering	Mar 14, 2017	Question AnsweringReading Comprehension	—Unverified	0
Making Neural QA as Simple as Possible but not Simpler	Mar 14, 2017	Question AnsweringReading Comprehension	CodeCode Available	1
Linguistic Knowledge as Memory for Recurrent Neural Networks	Mar 7, 2017	LAMBADAReading Comprehension	—Unverified	0
A Comparative Study of Word Embeddings for Reading Comprehension	Mar 2, 2017	BIG-bench Machine LearningReading Comprehension	—Unverified	0
Structural Embedding of Syntactic Trees for Machine Comprehension	Mar 2, 2017	Question AnsweringReading Comprehension	—Unverified	0
Understanding Image and Text Simultaneously: a Dual Vision-Language Machine Comprehension Task	Dec 22, 2016	Image CaptioningMulti-Task Learning	—Unverified	0
Machine Reading with Background Knowledge	Dec 16, 2016	Prepositional Phrase AttachmentReading Comprehension	—Unverified	0
Building Large Machine Reading-Comprehension Datasets using Paragraph Vectors	Dec 13, 2016	Dataset GenerationMachine Reading Comprehension	CodeCode Available	0
Multi-Perspective Context Matching for Machine Comprehension	Dec 13, 2016	Question AnsweringReading Comprehension	CodeCode Available	0
Reading Comprehension using Entity-based Memory Network	Dec 12, 2016	Question AnsweringReading Comprehension	—Unverified	0
Korean FrameNet Expansion Based on Projection of Japanese FrameNet	Dec 1, 2016	Machine TranslationReading Comprehension	—Unverified	0
Generating Questions and Multiple-Choice Answers using Semantic Analysis of Texts	Dec 1, 2016	coreference-resolutionCoreference Resolution	—Unverified	0
Textual complexity as a predictor of difficulty of listening items in language proficiency tests	Dec 1, 2016	Reading Comprehension	—Unverified	0
The Open Framework for Developing Knowledge Base And Question Answering System	Dec 1, 2016	Question AnsweringReading Comprehension	—Unverified	0
Chinese Hypernym-Hyponym Extraction from User Generated Categories	Dec 1, 2016	Knowledge GraphsMachine Translation	—Unverified	0
Effect of Syntactic Features in Bangla Sentence Comprehension	Dec 1, 2016	Language AcquisitionReading Comprehension	—Unverified	0
Distributed Vector Representations for Unsupervised Automatic Short Answer Grading	Dec 1, 2016	automatic short answer gradingReading Comprehension	—Unverified	0
Filling a Knowledge Graph with a Crowd	Dec 1, 2016	Knowledge GraphsQuestion Answering	—Unverified	0
QAF: Frame Semantics-based Question Interpretation	Dec 1, 2016	Knowledge Base Question AnsweringQuestion Answering	—Unverified	0
Dedicated Workflow Management for OKBQA Framework	Dec 1, 2016	ManagementQuestion Answering	—Unverified	0

Show:10 25 50

← PrevPage 63 of 71Next →

All datasets ReClor RACE MuSeRC AdversarialQA CrowdSource QA RadQA

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Rational Reasoner / IDOL	Test	80.6	—	Unverified
2	AMR-LE-Ensemble	Test	80	—	Unverified
3	MERIt(MERIt-deberta-v2-xxlarge )	Test	79.3	—	Unverified
4	MERIt-deberta-v2-xxlarge deberta.v2.xxlarge.path.override_True.norm_1.1.0.w2.A100.cp200.s42	Test	79.3	—	Unverified
5	Knowledge model	Test	79.2	—	Unverified
6	DeBERTa-v2-xxlarge-AMR-LE-Contraposition	Test	77.2	—	Unverified
7	LReasoner ensemble	Test	76.1	—	Unverified
8	ELECTRA and ALBERT	Test	71	—	Unverified
9	WWZ	Test	69.7	—	Unverified
10	xlnet-large-uncased [extended data]	Test	69.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALBERT (Ensemble)	Accuracy	91.4	—	Unverified
2	Megatron-BERT (ensemble)	Accuracy	90.9	—	Unverified
3	ALBERTxxlarge+DUMA(ensemble)	Accuracy	89.8	—	Unverified
4	Megatron-BERT	Accuracy	89.5	—	Unverified
5	XLNet	Accuracy (Middle)	88.6	—	Unverified
6	DeBERTalarge	Accuracy	86.8	—	Unverified
7	B10-10-10	Accuracy	85.7	—	Unverified
8	RoBERTa	Accuracy	83.2	—	Unverified
9	Orca 2-13B	Accuracy	82.87	—	Unverified
10	Orca 2-7B	Accuracy	80.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Golden Transformer	Average F1	0.94	—	Unverified
2	MT5 Large	Average F1	0.84	—	Unverified
3	ruRoberta-large finetune	Average F1	0.83	—	Unverified
4	ruT5-large-finetune	Average F1	0.82	—	Unverified
5	Human Benchmark	Average F1	0.81	—	Unverified
6	ruT5-base-finetune	Average F1	0.77	—	Unverified
7	ruBert-large finetune	Average F1	0.76	—	Unverified
8	ruBert-base finetune	Average F1	0.74	—	Unverified
9	RuGPT3XL few-shot	Average F1	0.74	—	Unverified
10	RuGPT3Large	Average F1	0.73	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RoBERTa-Large	Overall: F1	64.4	—	Unverified
2	BERT-Large	Overall: F1	62.7	—	Unverified
3	BiDAF	Overall: F1	28.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BERT	MSE	0.05	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BERT pretrained on MIMIC-III	Answer F1	63.55	—	Unverified