Natural Language Understanding
Natural Language Understanding is an important field of Natural Language Processing which contains various tasks such as text classification, natural language inference and story comprehension. Applications enabled by natural language understanding range from question answering to automated reasoning.
Source: Find a Reasonable Ending for Stories: Does Logic Relation Help the Story Cloze Test?
Papers
Showing 1–25 of 1978 papers
Benchmark Results
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | HNN | Accuracy | 90 | — | Unverified |
| 2 | UDSSM-II (ensemble) | Accuracy | 78.3 | — | Unverified |
| 3 | BERT-large 340M | Accuracy | 78.3 | — | Unverified |
| 4 | UDSSM-I (ensemble) | Accuracy | 76.7 | — | Unverified |
| 5 | DSSM | Accuracy | 75 | — | Unverified |
| 6 | UDSSM-II | Accuracy | 75 | — | Unverified |
| 7 | BERT-base 110M + MAS | Accuracy | 68.3 | — | Unverified |
| 8 | USSM + Supervised Deepnet + 3 Knowledge Bases | Accuracy | 66.7 | — | Unverified |
| 9 | Word-level CNN+LSTM (full scoring) | Accuracy | 60 | — | Unverified |
| 10 | Subword-level Transformer LM | Accuracy | 58.3 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | BERT (pred POS/lemmas) | Tags (Full) Acc | 82.5 | — | Unverified |
| 2 | BERT (none) | Tags (Full) Acc | 82 | — | Unverified |
| 3 | BERT (gold POS/lemmas) | Tags (Full) Acc | 81 | — | Unverified |
| 4 | GloVe (gold POS/lemmas) | Tags (Full) Acc | 79.3 | — | Unverified |
| 5 | RoBERTa + Linear | Full F1 (Preps) | 78.2 | — | Unverified |
| 6 | GloVe (none) | Tags (Full) Acc | 77.5 | — | Unverified |
| 7 | GloVe (pred POS/lemmas) | Tags (Full) Acc | 77.1 | — | Unverified |
| 8 | SVM (feature-rich, gold syntax) | Role F1 (Preps) | 62.2 | — | Unverified |
| 9 | BiLSTM + MLP (gold syntax) | Role F1 (Preps) | 62.2 | — | Unverified |
| 10 | SVM (feature-rich, auto syntax) | Role F1 (Preps) | 58.2 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CaseLaw-BERT | CaseHOLD | 75.6 | — | Unverified |
| 2 | Legal-BERT | CaseHOLD | 75.1 | — | Unverified |
| 3 | DeBERTa | CaseHOLD | 72.1 | — | Unverified |
| 4 | Longformer | CaseHOLD | 72 | — | Unverified |
| 5 | RoBERTa | CaseHOLD | 71.7 | — | Unverified |
| 6 | BERT | CaseHOLD | 70.7 | — | Unverified |
| 7 | BigBird | CaseHOLD | 70.4 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | ConvBERT-DG | Average | 74.6 | — | Unverified |
| 2 | ConvBERT-DG + Pre + Multi | Average | 73.8 | — | Unverified |
| 3 | mslm | Average | 73.49 | — | Unverified |
| 4 | ConvBERT + Pre + Multi | Average | 68.22 | — | Unverified |
| 5 | BanLanGen | Average | 39.16 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | ConvBERT + Pre + Multi | Average | 86.89 | — | Unverified |
| 2 | mslm | Average | 85.83 | — | Unverified |
| 3 | ConvBERT-DG + Pre + Multi | Average | 85.34 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | MT-DNN-SMART | Average | 89.9 | — | Unverified |
| 2 | BERT-LARGE | Average | 82.1 | — | Unverified |