| MC-BERT: Efficient Language Pre-Training via a Meta Controller | Jun 10, 2020 | Binary ClassificationCloze Test | CodeCode Available | 1 | 5 |
| Composable Sparse Fine-Tuning for Cross-Lingual Transfer | Oct 14, 2021 | Cross-Lingual TransferLanguage Modeling | CodeCode Available | 1 | 5 |
| Endowing Protein Language Models with Structural Knowledge | Jan 26, 2024 | Drug DesignLanguage Modeling | CodeCode Available | 1 | 5 |
| Frustratingly Simple Pretraining Alternatives to Masked Language Modeling | Sep 4, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Generative power of a protein language model trained on multiple sequence alignments | Apr 14, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Mask-Predict: Parallel Decoding of Conditional Masked Language Models | Apr 19, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| MMBERT: Multimodal BERT Pretraining for Improved Medical VQA | Apr 3, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Generate to Understand for Representation | Jun 14, 2023 | Contrastive LearningGPU | CodeCode Available | 1 | 5 |
| MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding | Mar 11, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| MicroBERT: Effective Training of Low-resource Monolingual BERTs through Parameter Reduction and Multitask Learning | Dec 23, 2022 | Dependency ParsingLanguage Modeling | CodeCode Available | 1 | 5 |
| Nonparametric Masked Language Modeling | Dec 2, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Stochastic positional embeddings improve masked image modeling | Jul 31, 2023 | Language ModellingMasked Language Modeling | CodeCode Available | 1 | 5 |
| GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding | Oct 23, 2023 | ArticlesContrastive Learning | CodeCode Available | 1 | 5 |
| Contextual Representation Learning beyond Masked Language Modeling | Apr 8, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model | May 19, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| CodeEditor: Learning to Edit Source Code with Pre-trained Models | Oct 31, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 0 | 5 |
| Measuring Social Biases in Masked Language Models by Proxy of Prediction Quality | Feb 21, 2024 | Language ModelingLanguage Modelling | CodeCode Available | 0 | 5 |
| Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More | Feb 11, 2025 | DecoderInformation Retrieval | CodeCode Available | 0 | 5 |
| Arabic Synonym BERT-based Adversarial Examples for Text Classification | Feb 5, 2024 | Adversarial TextLanguage Modeling | CodeCode Available | 0 | 5 |
| Masked Latent Semantic Modeling: an Efficient Pre-training Alternative to Masked Language Modeling | Jul 7, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 0 | 5 |
| Masked and Permuted Implicit Context Learning for Scene Text Recognition | May 25, 2023 | DecoderLanguage Modeling | CodeCode Available | 0 | 5 |
| Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers | Jun 5, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 | 5 |
| DS-TOD: Efficient Domain Specialization for Task-Oriented Dialog | May 1, 2022 | dialog state trackingLanguage Modeling | CodeCode Available | 0 | 5 |
| Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike Ways | Oct 26, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 0 | 5 |
| DS-TOD: Efficient Domain Specialization for Task Oriented Dialog | Oct 15, 2021 | dialog state trackingLanguage Modeling | CodeCode Available | 0 | 5 |
