| Misinformation Detection in Social Media Video Posts | Feb 15, 2022 | Contrastive LearningLanguage Modeling | —Unverified | 0 |
| Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling | Feb 7, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Prompt-Guided Injection of Conformation to Pre-trained Protein Model | Feb 7, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |
| MVPTR: Multi-Level Semantic Alignment for Vision-Language Pre-Training via Multi-Stage Learning | Jan 29, 2022 | Image-text matchingLanguage Modeling | CodeCode Available | 1 |
| Text Style Transfer for Bias Mitigation using Masked Language Modeling | Jan 21, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Data Augmentation for Biomedical Factoid Question Answering | Jan 16, 2022 | Data AugmentationInformation Retrieval | —Unverified | 0 |
| Causal Distillation for Language Models | Jan 16, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |
| STT: Soft Template Tuning for Few-Shot Learning | Jan 16, 2022 | Few-Shot LearningLanguage Modeling | —Unverified | 0 |
| Eliciting Knowledge from Pretrained Language Models for Prototypical Prompt Verbalizer | Jan 14, 2022 | ClassificationContrastive Learning | CodeCode Available | 1 |
| A Transfer Learning Pipeline for Educational Resource Discovery with Application in Leading Paragraph Generation | Jan 7, 2022 | Information RetrievalLanguage Modeling | —Unverified | 0 |
| EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate | Dec 29, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge | Dec 16, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoising | Dec 14, 2021 | Cross-Modal RetrievalDecoder | —Unverified | 0 |
| Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation | Dec 10, 2021 | Image-text matchingImage-text Retrieval | —Unverified | 0 |
| DIBERT: Dependency Injected Bidirectional Encoder Representations from Transformers | Dec 5, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Causal Distillation for Language Models | Dec 5, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| UFO: A UniFied TransfOrmer for Vision-Language Representation Learning | Nov 19, 2021 | Image CaptioningImage-text matching | —Unverified | 0 |
| LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model | Nov 18, 2021 | Anomaly DetectionLanguage Modeling | —Unverified | 0 |
| Prompt-Learning for Fine-Grained Entity Typing | Nov 16, 2021 | Entity TypingKnowledge Probing | —Unverified | 0 |
| TACO: Pre-training of Deep Transformers with Attention Convolution using Disentangled Positional Representation | Nov 16, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Phrase-aware Unsupervised Constituency Parsing | Nov 16, 2021 | Constituency ParsingLanguage Modeling | —Unverified | 0 |
| Temporal Language Modeling for Short Text Document Classification with Transformers | Nov 16, 2021 | ClassificationDocument Classification | —Unverified | 0 |
| How does the pre-training objective affect what large language models learn about linguistic properties? | Nov 16, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| DS-TOD: Efficient Domain Specialization for Task-Oriented Dialog | Nov 16, 2021 | dialog state trackingLanguage Modeling | —Unverified | 0 |
| Generative Prompt Tuning for Relation Classification | Nov 16, 2021 | ClassificationLanguage Modeling | —Unverified | 0 |
| Towards Unified Prompt Tuning for Few-shot Learning | Nov 16, 2021 | Few-Shot LearningLanguage Modeling | —Unverified | 0 |
| Contextual Representation Learning beyond Masked Language Modeling | Nov 16, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification | Nov 16, 2021 | Few-Shot Text ClassificationLanguage Modeling | —Unverified | 0 |
| "Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction | Nov 16, 2021 | Grammatical Error CorrectionLanguage Modeling | —Unverified | 0 |
| Predicting Attention Sparsity in Transformers | Nov 16, 2021 | DecoderLanguage Modeling | —Unverified | 0 |
| Probing BERT’s priors with serial reproduction chains | Nov 16, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| DAWSON: Data Augmentation using Weak Supervision On Natural Language | Nov 16, 2021 | Data AugmentationLanguage Modeling | —Unverified | 0 |
| Composable Sparse Fine-Tuning for Cross-Lingual Transfer | Nov 16, 2021 | Cross-Lingual TransferLanguage Modeling | —Unverified | 0 |
| A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models | Nov 16, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Unsupervised Dependency Graph Network | Nov 16, 2021 | Dependency ParsingLanguage Modeling | —Unverified | 0 |
| Joint Unsupervised and Supervised Training for Multilingual ASR | Nov 15, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| iBOT: Image BERT Pre-Training with Online Tokenizer | Nov 15, 2021 | image-classificationImage Classification | CodeCode Available | 1 |
| NICT Kyoto Submission for the WMT’21 Quality Estimation Task: Multimetric Multilingual Pretraining for Critical Error Detection | Nov 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Modeling Mathematical Notation Semantics in Academic Papers | Nov 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| JavaBERT: Training a transformer-based model for the Java programming language | Oct 20, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| NormFormer: Improved Transformer Pretraining with Extra Normalization | Oct 18, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models | Oct 16, 2021 | Image CaptioningLanguage Modeling | CodeCode Available | 1 |
| DS-TOD: Efficient Domain Specialization for Task Oriented Dialog | Oct 15, 2021 | dialog state trackingLanguage Modeling | CodeCode Available | 0 |
| Composable Sparse Fine-Tuning for Cross-Lingual Transfer | Oct 14, 2021 | Cross-Lingual TransferLanguage Modeling | CodeCode Available | 1 |
| Maximizing Efficiency of Language Model Pre-training for Learning Representation | Oct 13, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Dict-BERT: Enhancing Language Model Pre-training with Dictionary | Oct 13, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Multi-Modal Pre-Training for Automated Speech Recognition | Oct 12, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Contextualized Semantic Distance between Highly Overlapped Texts | Oct 4, 2021 | Domain AdaptationLanguage Modeling | CodeCode Available | 0 |
| Image BERT Pre-training with Online Tokenizer | Sep 29, 2021 | image-classificationImage Classification | —Unverified | 0 |
| MLIM: Vision-and-Language Model Pre-training with Masked Language and Image Modeling | Sep 24, 2021 | Image ReconstructionLanguage Modeling | —Unverified | 0 |