Linguistic Acceptability
Linguistic Acceptability is the task of determining whether a sentence is grammatical or ungrammatical.
Image Source: Warstadt et al
Papers
Showing 1–25 of 72 papers
Benchmark Results
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | En-BERT + TDA + PCA | Accuracy | 88.6 | — | Unverified |
| 2 | BERT+TDA | Accuracy | 88.2 | — | Unverified |
| 3 | RoBERTa+TDA | Accuracy | 87.3 | — | Unverified |
| 4 | deberta-v3-base+tasksource | Accuracy | 87.15 | — | Unverified |
| 5 | RoBERTa-large 355M + Entailment as Few-shot Learner | Accuracy | 86.4 | — | Unverified |
| 6 | LTG-BERT-base 98M | Accuracy | 82.7 | — | Unverified |
| 7 | ELC-BERT-base 98M | Accuracy | 82.6 | — | Unverified |
| 8 | En-BERT + TDA | Accuracy | 82.1 | — | Unverified |
| 9 | FNet-Large | Accuracy | 78 | — | Unverified |
| 10 | LTG-BERT-small 24M | Accuracy | 77.6 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | Ru-RoBERTa+TDA | MCC | 0.59 | — | Unverified |
| 2 | ruRoBERTa | MCC | 0.53 | — | Unverified |
| 3 | Ru-BERT+TDA | MCC | 0.48 | — | Unverified |
| 4 | RemBERT | MCC | 0.44 | — | Unverified |
| 5 | ruBERT | MCC | 0.42 | — | Unverified |
| 6 | ruGPT-3 | MCC | 0.3 | — | Unverified |
| 7 | ruT5 | MCC | 0.25 | — | Unverified |
| 8 | mBERT | MCC | 0.15 | — | Unverified |
| 9 | XLM-R | MCC | 0.13 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | En-BERT + TDA | Accuracy | 88.6 | — | Unverified |
| 2 | XLM-R (pre-trained) + TDA | Accuracy | 73 | — | Unverified |
| 3 | DeBERTa (large) | Accuracy | 69.5 | — | Unverified |
| 4 | TinyBERT-6 67M | Accuracy | 54 | — | Unverified |
| 5 | Synthesizer (R+V) | Accuracy | 53.3 | — | Unverified |
| 6 | En-BERT (pre-trained) + TDA | MCC | 0.42 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | XLM-R + TDA | MCC | 0.68 | — | Unverified |
| 2 | XLM-R | MCC | 0.52 | — | Unverified |
| 3 | It-BERT (pre-trained) + TDA | MCC | 0.48 | — | Unverified |
| 4 | mBERT | MCC | 0.36 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | Sw-BERT + H0M | Accuracy | 76.9 | — | Unverified |