SOTAVerified

Reading Comprehension

Most current question answering datasets frame the task as reading comprehension where the question is about a paragraph or document and the answer often is a span in the document.

Some specific tasks of reading comprehension include multi-modal machine reading comprehension and textual machine reading comprehension, among others. In the literature, machine reading comprehension can be divide into four categories: cloze style, multiple choice, span prediction, and free-form answer. Read more about each category here.

Benchmark datasets used for testing a model's reading comprehension abilities include MovieQA, ReCoRD, and RACE, among others.

The Machine Reading group at UCL also provides an overview of reading comprehension tasks.

Figure source: A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1051–1100 of 1760 papers

TitleStatusHype
Why can't memory networks read effectively?—0
Why We Build Local Large Language Models: An Observational Analysis from 35 Japanese and Multilingual LLMs—0
WikiPossessions: Possession Timeline Generation as an Evaluation Benchmark for Machine Reading Comprehension of Long Texts—0
Work Smart - Reducing Effort in Short-Answer Grading—0
World Knowledge for Reading Comprehension: Rare Entity Prediction with Hierarchical LSTMs Using External Descriptions—0
XCMRC: Evaluating Cross-lingual Machine Reading Comprehension—0
XL^2Bench: A Benchmark for Extremely Long Context Understanding with Long-range Dependencies—0
XLMRQA: Open-Domain Question Answering on Vietnamese Wikipedia-based Textual Knowledge Source—0
XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking—0
Yimmon at SemEval-2019 Task 9: Suggestion Mining with Hybrid Augmented Approaches—0
Y-NQ: English-Yorùbá Evaluation dataset for Open-Book Reading Comprehension and Text Generation—0
YNU\_AI1799 at SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge of Different model ensemble—0
YNU\_Deep at SemEval-2018 Task 11: An Ensemble of Attention-based BiLSTM Models for Machine Comprehension—0
YNU Deep at SemEval-2018 Task 12: A BiLSTM Model with Neural Attention for Argument Reasoning Comprehension—0
YNU-HPCC at Semeval-2018 Task 11: Using an Attention-based CNN-LSTM for Machine Comprehension using Commonsense Knowledge—0
``You are grounded!'': Latent Name Artifacts in Pre-trained Language Models—0
Zero-Shot Estimation of Base Models' Weights in Ensemble of Machine Reading Comprehension Systems for Robust Generalization—0
Zero-shot Event Causality Identification with Question Answering—0
Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model—0
ZEROTOP: Zero-Shot Task-Oriented Semantic Parsing using Large Language Models—0
Fewer Truncations Improve Language Modeling—0
Few-shot Mining of Naturally Occurring Inputs and Outputs—0
Few-shot Policy (de)composition in Conversational Question Answering—0
Filling a Knowledge Graph with a Crowd—0
Focus Annotation in Reading Comprehension Data—0
Focus Annotation of Task-based Data: Establishing the Quality of Crowd Annotation—0
Focus Annotation of Task-based Data: A Comparison of Expert and Crowd-Sourced Annotation in a Reading Comprehension Corpus—0
ForceReader: a BERT-based Interactive Machine Reading Comprehension Model with Attention Separation—0
ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data—0
FPAI at SemEval-2020 Task 10: A Query Enhanced Model with RoBERTa for Emphasis Selection—0
FQuAD2.0: French Question Answering and knowing that you know nothing—0
FQuAD2.0: French Question Answering and Learning When You Don’t Know—0
FQuAD: French Question Answering Dataset—0
FriendsQA: Open-Domain Question Answering on TV Show Transcripts—0
From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension—0
From Light to Rich ERE: Annotation of Entities, Relations, and Events—0
Frustratingly Poor Performance of Reading Comprehension Models on Non-adversarial Examples—0
G4: Grounding-guided Goal-oriented Dialogues Generation with Multiple Documents—0
GAAMA 2.0: An Integrated System that Answers Boolean and Extractive Questions—0
Gated Self-Matching Networks for Reading Comprehension and Question Answering—0
Gaze-Driven Sentence Simplification for Language Learners: Enhancing Comprehension and Readability—0
General Embedding vs. Task-Specific Embedding: A Comparative Approach to Enhancing NLP Performance—0
Generalizing Question Answering System with Pre-trained Language Model Fine-tuning—0
Generating and Exploiting Large-scale Pseudo Training Data for Zero Pronoun Resolution—0
Generating Diagnostic Multiple Choice Comprehension Cloze Questions—0
Generating Feedback for English Foreign Language Exercises—0
Generating Questions and Multiple-Choice Answers using Semantic Analysis of Texts—0
Generating Questions for Reading Comprehension using Coherence Relations—0
Generating Training Data for Semantic Role Labeling based on Label Transfer from Linked Lexical Resources—0
Generative Large Language Models Are All-purpose Text Analytics Engines: Text-to-text Learning Is All Your Need—0
Show:102550
← PrevPage 22 of 36Next →
All datasetsReClorRACEMuSeRCAdversarialQACrowdSource QARadQA

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Rational Reasoner / IDOLTest80.6—Unverified
2AMR-LE-EnsembleTest80—Unverified
3MERIt(MERIt-deberta-v2-xxlarge )Test79.3—Unverified
4MERIt-deberta-v2-xxlarge deberta.v2.xxlarge.path.override_True.norm_1.1.0.w2.A100.cp200.s42Test79.3—Unverified
5Knowledge modelTest79.2—Unverified
6DeBERTa-v2-xxlarge-AMR-LE-ContrapositionTest77.2—Unverified
7LReasoner ensembleTest76.1—Unverified
8ELECTRA and ALBERTTest71—Unverified
9WWZTest69.7—Unverified
10xlnet-large-uncased [extended data]Test69.3—Unverified
#ModelMetricClaimedVerifiedStatus
1ALBERT (Ensemble)Accuracy91.4—Unverified
2Megatron-BERT (ensemble)Accuracy90.9—Unverified
3ALBERTxxlarge+DUMA(ensemble)Accuracy89.8—Unverified
4Megatron-BERTAccuracy89.5—Unverified
5XLNetAccuracy (Middle)88.6—Unverified
6DeBERTalargeAccuracy86.8—Unverified
7B10-10-10Accuracy85.7—Unverified
8RoBERTaAccuracy83.2—Unverified
9Orca 2-13BAccuracy82.87—Unverified
10Orca 2-7BAccuracy80.79—Unverified
#ModelMetricClaimedVerifiedStatus
1Golden TransformerAverage F10.94—Unverified
2MT5 LargeAverage F10.84—Unverified
3ruRoberta-large finetuneAverage F10.83—Unverified
4ruT5-large-finetuneAverage F10.82—Unverified
5Human BenchmarkAverage F10.81—Unverified
6ruT5-base-finetuneAverage F10.77—Unverified
7ruBert-large finetuneAverage F10.76—Unverified
8ruBert-base finetuneAverage F10.74—Unverified
9RuGPT3XL few-shotAverage F10.74—Unverified
10RuGPT3LargeAverage F10.73—Unverified
#ModelMetricClaimedVerifiedStatus
1RoBERTa-LargeOverall: F164.4—Unverified
2BERT-LargeOverall: F162.7—Unverified
3BiDAFOverall: F128.5—Unverified
#ModelMetricClaimedVerifiedStatus
1BERTMSE0.05—Unverified
#ModelMetricClaimedVerifiedStatus
1BERT pretrained on MIMIC-IIIAnswer F163.55—Unverified