| Contrastive Learning for Prompt-Based Few-Shot Language Learners | May 3, 2022 | Contrastive LearningIn-Context Learning | CodeCode Available | 1 |
| Unsupervised Dependency Graph Network | May 1, 2022 | Dependency ParsingLanguage Modeling | CodeCode Available | 1 |
| Generative power of a protein language model trained on multiple sequence alignments | Apr 14, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? | Apr 12, 2022 | DecoderLanguage Modeling | CodeCode Available | 1 |
| Contextual Representation Learning beyond Masked Language Modeling | Apr 8, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| SecureBERT: A Domain-Specific Language Model for Cybersecurity | Apr 6, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| POS-BERT: Point Cloud One-Stage BERT Pre-Training | Apr 3, 2022 | Contrastive LearningLanguage Modeling | CodeCode Available | 1 |
| What to Hide from Your Students: Attention-Guided Masked Image Modeling | Mar 23, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigation | Mar 22, 2022 | Decision MakingLanguage Modeling | CodeCode Available | 1 |
| How does the pre-training objective affect what large language models learn about linguistic properties? | Mar 20, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Transformer Quality in Linear Time | Feb 21, 2022 | 8kLanguage Modeling | CodeCode Available | 1 |
| Should You Mask 15% in Masked Language Modeling? | Feb 16, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling | Feb 7, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| MVPTR: Multi-Level Semantic Alignment for Vision-Language Pre-Training via Multi-Stage Learning | Jan 29, 2022 | Image-text matchingLanguage Modeling | CodeCode Available | 1 |
| Eliciting Knowledge from Pretrained Language Models for Prototypical Prompt Verbalizer | Jan 14, 2022 | ClassificationContrastive Learning | CodeCode Available | 1 |
| EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate | Dec 29, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Causal Distillation for Language Models | Dec 5, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| iBOT: Image BERT Pre-Training with Online Tokenizer | Nov 15, 2021 | image-classificationImage Classification | CodeCode Available | 1 |
| A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models | Oct 16, 2021 | Image CaptioningLanguage Modeling | CodeCode Available | 1 |
| Composable Sparse Fine-Tuning for Cross-Lingual Transfer | Oct 14, 2021 | Cross-Lingual TransferLanguage Modeling | CodeCode Available | 1 |
| SupCL-Seq: Supervised Contrastive Learning for Downstream Optimized Sequence Representations | Sep 15, 2021 | CoLAContrastive Learning | CodeCode Available | 1 |
| CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation | Sep 13, 2021 | DecoderDenoising | CodeCode Available | 1 |
| Data Efficient Masked Language Modeling for Vision and Language | Sep 5, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Frustratingly Simple Pretraining Alternatives to Masked Language Modeling | Sep 4, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| CTAL: Pre-training Cross-modal Transformer for Audio-and-Language Representations | Sep 1, 2021 | Emotion ClassificationLanguage Modeling | CodeCode Available | 1 |
