| MeLT: Message-Level Transformer with Masked Document Representations as Pre-Training for Stance Detection | Sep 16, 2021 | AttributeLanguage Modeling | CodeCode Available | 0 |
| Split-and-Rephrase in a Cross-Lingual Manner: A Complete Pipeline | Sep 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Domain-Specific Japanese ELECTRA Model Using a Small Corpus | Sep 1, 2021 | ArticlesComputational Efficiency | —Unverified | 0 |
| Prompt-Learning for Fine-Grained Entity Typing | Aug 24, 2021 | Entity TypingKnowledge Probing | —Unverified | 0 |
| Noobs at Semeval-2021 Task 4: Masked Language Modeling for abstract answer prediction | Aug 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Fine-Grained Emotion Prediction by Modeling Emotion Definitions | Jul 26, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Learning to Sample Replacements for ELECTRA Pre-Training | Jun 25, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Winner Team Mia at TextVQA Challenge 2021: Vision-and-Language Representation Learning with Pre-trained Sequence-to-Sequence Model | Jun 24, 2021 | DecoderLanguage Modeling | —Unverified | 0 |
| SAS: Self-Augmentation Strategy for Language Model Pre-training | Jun 14, 2021 | Data AugmentationLanguage Modeling | CodeCode Available | 0 |
| MST: Masked Self-Supervised Transformer for Visual Representation | Jun 10, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Exploring Unsupervised Pretraining Objectives for Machine Translation | Jun 10, 2021 | DecoderLanguage Modeling | CodeCode Available | 0 |
| BERTnesia: Investigating the capture and forgetting of knowledge in BERT | Jun 5, 2021 | Knowledge Base CompletionLanguage Modeling | CodeCode Available | 0 |
| Bi-Granularity Contrastive Learning for Post-Training in Few-Shot Scene | Jun 4, 2021 | Contrastive LearningData Augmentation | —Unverified | 0 |
| Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastings | Jun 4, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| BERT-Defense: A Probabilistic Model Based on BERT to Combat Cognitively Inspired Orthographic Adversarial Attacks | Jun 2, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| SCRIPT: Self-Critic PreTraining of Transformers | Jun 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Target-Aware Data Augmentation for Stance Detection | Jun 1, 2021 | Data AugmentationLanguage Modeling | —Unverified | 0 |
| MG-BERT: Multi-Graph Augmented BERT for Masked Language Modeling | Jun 1, 2021 | Knowledge GraphsLanguage Modeling | —Unverified | 0 |
| From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding | May 15, 2021 | intent-classificationIntent Classification | CodeCode Available | 0 |
| Larger-Scale Transformers for Multilingual Masked Language Modeling | May 2, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Understanding Chinese Video and Language via Contrastive Multimodal Pre-Training | Apr 19, 2021 | Contrastive LearningLanguage Modeling | —Unverified | 0 |
| On the Influence of Masking Policies in Intermediate Pre-training | Apr 18, 2021 | Abstractive Text SummarizationLanguage Modeling | —Unverified | 0 |
| Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little | Apr 14, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| ReCAM@IITK at SemEval-2021 Task 4: BERT and ALBERT based Ensemble for Abstract Word Prediction | Apr 4, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training | Apr 1, 2021 | Image-text matchingImage-text Retrieval | —Unverified | 0 |
| Pseudo-Label Guided Unsupervised Domain Adaptation of Contextual Embeddings | Apr 1, 2021 | Domain AdaptationLanguage Modeling | —Unverified | 0 |
| Self-supervised Image-text Pre-training With Mixed Data In Chest X-rays | Mar 30, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Variable Name Recovery in Decompiled Binary Code using Constrained Masked Language Modeling | Mar 23, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives | Feb 25, 2021 | Contrastive LearningLanguage Modeling | —Unverified | 0 |
| Bilingual Language Modeling, A transfer learning technique for Roman Urdu | Feb 22, 2021 | Cross-Lingual TransferLanguage Modeling | —Unverified | 0 |
| MSA Transformer | Feb 13, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| SJ_AJ@DravidianLangTech-EACL2021: Task-Adaptive Pre-Training of Multilingual BERT models for Offensive Language Identification | Feb 1, 2021 | Language IdentificationLanguage Modeling | CodeCode Available | 0 |
| MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding | Jan 23, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Universal Sentence Representations Learning with Conditional Masked Language Model | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Universal Sentence Representation Learning with Conditional Masked Language Model | Dec 28, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models | Dec 1, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| XHate-999: Analyzing and Detecting Abusive Language Across Domains and Languages | Dec 1, 2020 | Abusive LanguageDisentanglement | —Unverified | 0 |
| Self-Supervised Relationship Probing | Dec 1, 2020 | Contrastive LearningLanguage Modeling | —Unverified | 0 |
| Self-Supervised learning with cross-modal transformers for emotion recognition | Nov 20, 2020 | Emotion RecognitionLanguage Modeling | —Unverified | 0 |
| A Hierarchical Multi-Modal Encoder for Moment Localization in Video Corpus | Nov 18, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Controlling the Imprint of Passivization and Negation in Contextualized Representations | Nov 1, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| POSTECH-ETRI’s Submission to the WMT2020 APE Shared Task: Automatic Post-Editing with Cross-lingual Language Model | Nov 1, 2020 | Automatic Post-EditingLanguage Modeling | —Unverified | 0 |
| Effective Decoder Masking for Transformer Based End-to-End Speech Recognition | Oct 27, 2020 | Automatic Speech RecognitionAutomatic Speech Recognition (ASR) | —Unverified | 0 |
| DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries | Oct 23, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding | Oct 23, 2020 | cross-modal alignmentLanguage Modeling | —Unverified | 0 |
| Corruption Is Not All Bad: Incorporating Discourse Structure into Pre-training via Corruption for Essay Scoring | Oct 13, 2020 | AllAutomated Essay Scoring | —Unverified | 0 |
| VECO: Variable Encoder-decoder Pre-training for Cross-lingual Understanding and Generation | Sep 28, 2020 | DecoderLanguage Modeling | —Unverified | 0 |
| Deep Transformers with Latent Depth | Sep 28, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| GraphCodeBERT: Pre-training Code Representations with Data Flow | Sep 17, 2020 | Clone DetectionCode Completion | —Unverified | 0 |
| Learning Visual Representations with Caption Annotations | Aug 4, 2020 | Image CaptioningLanguage Modeling | —Unverified | 0 |