| MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NER | Aug 31, 2021 | Cross-Lingual NERData Augmentation | CodeCode Available | 1 |
| Sentence Bottleneck Autoencoders from Transformer Language Models | Aug 31, 2021 | DecoderDenoising | CodeCode Available | 1 |
| Knowledge Perceived Multi-modal Pretraining in E-commerce | Aug 20, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification | Aug 4, 2021 | ClassificationFew-Shot Text Classification | CodeCode Available | 1 |
| SPBERT: An Efficient Pre-training BERT on SPARQL Queries for Question Answering over Knowledge Graphs | Jun 18, 2021 | DecoderKnowledge Graphs | CodeCode Available | 1 |
| Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment | Jun 11, 2021 | DenoisingLanguage Modeling | CodeCode Available | 1 |
| Luna: Linear Unified Nested Attention | Jun 3, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| TreeBERT: A Tree-Based Pre-Trained Model for Programming Language | May 26, 2021 | Code SummarizationLanguage Modeling | CodeCode Available | 1 |
| KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction | Apr 15, 2021 | Dialog Relation ExtractionLanguage Modeling | CodeCode Available | 1 |
| On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies | Apr 12, 2021 | Inductive BiasLanguage Modeling | CodeCode Available | 1 |
| MMBERT: Multimodal BERT Pretraining for Improved Medical VQA | Apr 3, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation | Mar 18, 2021 | Bilingual Lexicon InductionLanguage Modeling | CodeCode Available | 1 |
| MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding | Mar 11, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| CDLM: Cross-Document Language Modeling | Jan 2, 2021 | Citation RecommendationCoreference Resolution | CodeCode Available | 1 |
| AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding | Dec 31, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| RealFormer: Transformer Likes Residual Attention | Dec 21, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| TAP: Text-Aware Pre-training for Text-VQA and Text-Caption | Dec 8, 2020 | Caption GenerationLanguage Modeling | CodeCode Available | 1 |
| Pre-training Protein Language Models with Label-Agnostic Binding Pairs Enhances Performance in Downstream Tasks | Dec 5, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling | Dec 1, 2020 | Constituency ParsingDependency Parsing | CodeCode Available | 1 |
| Cold-start Active Learning through Self-supervised Language Modeling | Oct 19, 2020 | Active LearningClassification | CodeCode Available | 1 |
| Cross-Thought for Sentence Encoder Pre-training | Oct 7, 2020 | Information RetrievalLanguage Modeling | CodeCode Available | 1 |
| SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding | Oct 5, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| XDA: Accurate, Robust Disassembly with Transfer Learning | Oct 2, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing | Sep 29, 2020 | Inductive BiasLanguage Modeling | CodeCode Available | 1 |
| Intermediate Training of BERT for Product Matching | Aug 31, 2020 | Entity ResolutionLanguage Modeling | CodeCode Available | 1 |
