| Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastings | Jun 4, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Bi-Granularity Contrastive Learning for Post-Training in Few-Shot Scene | Jun 4, 2021 | Contrastive LearningData Augmentation | —Unverified | 0 |
| Luna: Linear Unified Nested Attention | Jun 3, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| BERT-Defense: A Probabilistic Model Based on BERT to Combat Cognitively Inspired Orthographic Adversarial Attacks | Jun 2, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| MG-BERT: Multi-Graph Augmented BERT for Masked Language Modeling | Jun 1, 2021 | Knowledge GraphsLanguage Modeling | —Unverified | 0 |
| SCRIPT: Self-Critic PreTraining of Transformers | Jun 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Target-Aware Data Augmentation for Stance Detection | Jun 1, 2021 | Data AugmentationLanguage Modeling | —Unverified | 0 |
| TreeBERT: A Tree-Based Pre-Trained Model for Programming Language | May 26, 2021 | Code SummarizationLanguage Modeling | CodeCode Available | 1 |
| From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding | May 15, 2021 | intent-classificationIntent Classification | CodeCode Available | 0 |
| Larger-Scale Transformers for Multilingual Masked Language Modeling | May 2, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Understanding Chinese Video and Language via Contrastive Multimodal Pre-Training | Apr 19, 2021 | Contrastive LearningLanguage Modeling | —Unverified | 0 |
| On the Influence of Masking Policies in Intermediate Pre-training | Apr 18, 2021 | Abstractive Text SummarizationLanguage Modeling | —Unverified | 0 |
| KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction | Apr 15, 2021 | Dialog Relation ExtractionLanguage Modeling | CodeCode Available | 1 |
| Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little | Apr 14, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies | Apr 12, 2021 | Inductive BiasLanguage Modeling | CodeCode Available | 1 |
| ReCAM@IITK at SemEval-2021 Task 4: BERT and ALBERT based Ensemble for Abstract Word Prediction | Apr 4, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| MMBERT: Multimodal BERT Pretraining for Improved Medical VQA | Apr 3, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Pseudo-Label Guided Unsupervised Domain Adaptation of Contextual Embeddings | Apr 1, 2021 | Domain AdaptationLanguage Modeling | —Unverified | 0 |
| UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training | Apr 1, 2021 | Image-text matchingImage-text Retrieval | —Unverified | 0 |
| Self-supervised Image-text Pre-training With Mixed Data In Chest X-rays | Mar 30, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Variable Name Recovery in Decompiled Binary Code using Constrained Masked Language Modeling | Mar 23, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation | Mar 18, 2021 | Bilingual Lexicon InductionLanguage Modeling | CodeCode Available | 1 |
| MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding | Mar 11, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives | Feb 25, 2021 | Contrastive LearningLanguage Modeling | —Unverified | 0 |
| Bilingual Language Modeling, A transfer learning technique for Roman Urdu | Feb 22, 2021 | Cross-Lingual TransferLanguage Modeling | —Unverified | 0 |