SOTAVerified|Agents Browse Leaderboard About

Linguistic Acceptability

Linguistic Acceptability is the task of determining whether a sentence is grammatical or ungrammatical.

Image Source: Warstadt et al

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 21–30 of 72 papers

Title	Date	Tasks	Status	Hype
RuCoLA: Russian Corpus of Linguistic Acceptability	Oct 23, 2022	Linguistic AcceptabilityText Generation	CodeCode Available	1
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale	Aug 15, 2022	GPULanguage Modelling	CodeCode Available	5
Acceptability Judgements via Examining the Topology of Attention Maps	May 19, 2022	CoLALinguistic Acceptability	CodeCode Available	0
VALUE: Understanding Dialect Disparity in NLU	Apr 6, 2022	Linguistic AcceptabilityNatural Language Understanding	CodeCode Available	0
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language	Feb 7, 2022	image-classificationImage Classification	CodeCode Available	1
Cross-Architecture Distillation Using Bidirectional CMOW Embeddings	Sep 29, 2021	Linguistic AcceptabilityQQP	—Unverified	0
Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpus	Sep 24, 2021	CoLAdomain classification	CodeCode Available	0
Revisiting the Uniform Information Density Hypothesis	Sep 23, 2021	Linguistic AcceptabilitySentence	—Unverified	0
General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddings	Sep 17, 2021	CoLALinguistic Acceptability	CodeCode Available	0
An Automated Knowledge Mining and Document Classification System with Multi-model Transfer Learning	Jun 24, 2021	CoLADocument Classification	—Unverified	0

Show:10 25 50

← PrevPage 3 of 8Next →

All datasets CoLA RuCoLA CoLA Dev ItaCoLA DaLAJ

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	En-BERT + TDA + PCA	Accuracy	88.6	—	Unverified
2	BERT+TDA	Accuracy	88.2	—	Unverified
3	RoBERTa+TDA	Accuracy	87.3	—	Unverified
4	deberta-v3-base+tasksource	Accuracy	87.15	—	Unverified
5	RoBERTa-large 355M + Entailment as Few-shot Learner	Accuracy	86.4	—	Unverified
6	LTG-BERT-base 98M	Accuracy	82.7	—	Unverified
7	ELC-BERT-base 98M	Accuracy	82.6	—	Unverified
8	En-BERT + TDA	Accuracy	82.1	—	Unverified
9	FNet-Large	Accuracy	78	—	Unverified
10	LTG-BERT-small 24M	Accuracy	77.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Ru-RoBERTa+TDA	MCC	0.59	—	Unverified
2	ruRoBERTa	MCC	0.53	—	Unverified
3	Ru-BERT+TDA	MCC	0.48	—	Unverified
4	RemBERT	MCC	0.44	—	Unverified
5	ruBERT	MCC	0.42	—	Unverified
6	ruGPT-3	MCC	0.3	—	Unverified
7	ruT5	MCC	0.25	—	Unverified
8	mBERT	MCC	0.15	—	Unverified
9	XLM-R	MCC	0.13	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	En-BERT + TDA	Accuracy	88.6	—	Unverified
2	XLM-R (pre-trained) + TDA	Accuracy	73	—	Unverified
3	DeBERTa (large)	Accuracy	69.5	—	Unverified
4	TinyBERT-6 67M	Accuracy	54	—	Unverified
5	Synthesizer (R+V)	Accuracy	53.3	—	Unverified
6	En-BERT (pre-trained) + TDA	MCC	0.42	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	XLM-R + TDA	MCC	0.68	—	Unverified
2	XLM-R	MCC	0.52	—	Unverified
3	It-BERT (pre-trained) + TDA	MCC	0.48	—	Unverified
4	mBERT	MCC	0.36	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Sw-BERT + H0M	Accuracy	76.9	—	Unverified