Semantic Textual Similarity
Semantic textual similarity deals with determining how similar two pieces of texts are. This can take the form of assigning a score from 1 to 5. Related tasks are paraphrase or duplicate identification.
Image source: Learning Semantic Textual Similarity from Conversations
Papers
Showing 11–20 of 2381 papers
Benchmark Results
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | SMARTRoBERTa | Dev Pearson Correlation | 92.8 | — | Unverified |
| 2 | DeBERTa (large) | Accuracy | 92.5 | — | Unverified |
| 3 | SMART-BERT | Dev Pearson Correlation | 90 | — | Unverified |
| 4 | MT-DNN-SMART | Pearson Correlation | 0.93 | — | Unverified |
| 5 | StructBERTRoBERTa ensemble | Pearson Correlation | 0.93 | — | Unverified |
| 6 | Mnet-Sim | Pearson Correlation | 0.93 | — | Unverified |
| 7 | XLNet (single model) | Pearson Correlation | 0.93 | — | Unverified |
| 8 | ALBERT | Pearson Correlation | 0.93 | — | Unverified |
| 9 | T5-11B | Pearson Correlation | 0.93 | — | Unverified |
| 10 | RoBERTa | Pearson Correlation | 0.92 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | AnglE-UAE | Spearman Correlation | 84.54 | — | Unverified |
| 2 | ST5-XXL | Spearman Correlation | 82.63 | — | Unverified |
| 3 | ST5-Large | Spearman Correlation | 81.83 | — | Unverified |
| 4 | ST5-XL | Spearman Correlation | 81.66 | — | Unverified |
| 5 | ST5-Base | Spearman Correlation | 81.14 | — | Unverified |
| 6 | MPNet-multilingual | Spearman Correlation | 80.73 | — | Unverified |
| 7 | SGPT-5.8B-nli | Spearman Correlation | 80.53 | — | Unverified |
| 8 | MPNet | Spearman Correlation | 80.28 | — | Unverified |
| 9 | MiniLM-L12 | Spearman Correlation | 79.8 | — | Unverified |
| 10 | SimCSE-BERT-sup | Spearman Correlation | 79.12 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | MT-DNN-SMART | Accuracy | 93.7 | — | Unverified |
| 2 | ALBERT | Accuracy | 93.4 | — | Unverified |
| 3 | RoBERTa (ensemble) | Accuracy | 92.3 | — | Unverified |
| 4 | BigBird | F1 | 91.5 | — | Unverified |
| 5 | StructBERTRoBERTa ensemble | Accuracy | 91.5 | — | Unverified |
| 6 | FLOATER-large | Accuracy | 91.4 | — | Unverified |
| 7 | SMART | Accuracy | 91.3 | — | Unverified |
| 8 | RoBERTa-large 355M (MLP quantized vector-wise, fine-tuned) | Accuracy | 91 | — | Unverified |
| 9 | RoBERTa-large 355M + Entailment as Few-shot Learner | F1 | 91 | — | Unverified |
| 10 | SpanBERT | Accuracy | 90.9 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | PromCSE-RoBERTa-large (0.355B) | Spearman Correlation | 0.82 | — | Unverified |
| 2 | PromptEOL+CSE+LLaMA-30B | Spearman Correlation | 0.82 | — | Unverified |
| 3 | PromptEOL+CSE+OPT-13B | Spearman Correlation | 0.82 | — | Unverified |
| 4 | SimCSE-RoBERTalarge | Spearman Correlation | 0.82 | — | Unverified |
| 5 | PromptEOL+CSE+OPT-2.7B | Spearman Correlation | 0.81 | — | Unverified |
| 6 | SentenceBERT | Spearman Correlation | 0.75 | — | Unverified |
| 7 | SRoBERTa-NLI-base | Spearman Correlation | 0.74 | — | Unverified |
| 8 | SRoBERTa-NLI-large | Spearman Correlation | 0.74 | — | Unverified |
| 9 | Dino (STS/̄🦕) | Spearman Correlation | 0.74 | — | Unverified |
| 10 | SBERT-NLI-large | Spearman Correlation | 0.74 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | AnglE-LLaMA-7B | Spearman Correlation | 0.91 | — | Unverified |
| 2 | AnglE-LLaMA-7B-v2 | Spearman Correlation | 0.91 | — | Unverified |
| 3 | PromptEOL+CSE+LLaMA-30B | Spearman Correlation | 0.9 | — | Unverified |
| 4 | PromptEOL+CSE+OPT-13B | Spearman Correlation | 0.9 | — | Unverified |
| 5 | PromptEOL+CSE+OPT-2.7B | Spearman Correlation | 0.9 | — | Unverified |
| 6 | PromCSE-RoBERTa-large (0.355B) | Spearman Correlation | 0.89 | — | Unverified |
| 7 | Trans-Encoder-BERT-large-bi (unsup.) | Spearman Correlation | 0.89 | — | Unverified |
| 8 | Trans-Encoder-BERT-large-cross (unsup.) | Spearman Correlation | 0.88 | — | Unverified |
| 9 | Trans-Encoder-RoBERTa-large-cross (unsup.) | Spearman Correlation | 0.88 | — | Unverified |
| 10 | SimCSE-RoBERTa-large | Spearman Correlation | 0.87 | — | Unverified |