| Generative power of a protein language model trained on multiple sequence alignments | Apr 14, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding | Oct 23, 2023 | ArticlesContrastive Learning | CodeCode Available | 1 | 5 |
| On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies | Apr 12, 2021 | Inductive BiasLanguage Modeling | CodeCode Available | 1 | 5 |
| Generative Prompt Tuning for Relation Classification | Oct 22, 2022 | ClassificationLanguage Modeling | CodeCode Available | 1 | 5 |
| Causal Distillation for Language Models | Dec 5, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| MicroBERT: Effective Training of Low-resource Monolingual BERTs through Parameter Reduction and Multitask Learning | Dec 23, 2022 | Dependency ParsingLanguage Modeling | CodeCode Available | 1 | 5 |
| GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing | Sep 29, 2020 | Inductive BiasLanguage Modeling | CodeCode Available | 1 | 5 |
| Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification | Aug 4, 2021 | ClassificationFew-Shot Text Classification | CodeCode Available | 1 | 5 |
| A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models | Oct 16, 2021 | Image CaptioningLanguage Modeling | CodeCode Available | 1 | 5 |
| CodeArt: Better Code Models by Attention Regularization When Symbols Are Lacking | Feb 19, 2024 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding | Dec 31, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Cold-start Active Learning through Self-supervised Language Modeling | Oct 19, 2020 | Active LearningClassification | CodeCode Available | 1 | 5 |
| A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER | Aug 28, 2023 | Contrastive Learningfew-shot-ner | CodeCode Available | 1 | 5 |
| Knowledge Perceived Multi-modal Pretraining in E-commerce | Aug 20, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Intermediate Training of BERT for Product Matching | Aug 31, 2020 | Entity ResolutionLanguage Modeling | CodeCode Available | 1 | 5 |
| ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators | Mar 23, 2020 | GPULanguage Modeling | CodeCode Available | 1 | 5 |
| Interpretation of Intracardiac Electrograms Through Textual Representations | Feb 2, 2024 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Labrador: Exploring the Limits of Masked Language Modeling for Laboratory Data | Dec 9, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Contrastive Learning for Prompt-Based Few-Shot Language Learners | May 3, 2022 | Contrastive LearningIn-Context Learning | CodeCode Available | 1 | 5 |
| Contextual Representation Learning beyond Masked Language Modeling | Apr 8, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs | Jul 29, 2024 | Bilevel OptimizationLanguage Modelling | CodeCode Available | 1 | 5 |
| Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking | Dec 15, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| RealFormer: Transformer Likes Residual Attention | Dec 21, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation | Sep 13, 2021 | DecoderDenoising | CodeCode Available | 1 | 5 |
| Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment | Jun 11, 2021 | DenoisingLanguage Modeling | CodeCode Available | 1 | 5 |
| CreoPep: A Universal Deep Learning Framework for Target-Specific Peptide Design and Optimization | May 5, 2025 | DiversityLanguage Modeling | CodeCode Available | 1 | 5 |
| CDLM: Cross-Document Language Modeling | Jan 2, 2021 | Citation RecommendationCoreference Resolution | CodeCode Available | 1 | 5 |
| How does the pre-training objective affect what large language models learn about linguistic properties? | Mar 20, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Cross-Thought for Sentence Encoder Pre-training | Oct 7, 2020 | Information RetrievalLanguage Modeling | CodeCode Available | 1 | 5 |
| Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training | Jun 1, 2022 | Contrastive LearningCross-Lingual Transfer | CodeCode Available | 1 | 5 |
| DomURLs_BERT: Pre-trained BERT-based Model for Malicious Domains and URLs Detection and Classification | Sep 13, 2024 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Endowing Protein Language Models with Structural Knowledge | Jan 26, 2024 | Drug DesignLanguage Modeling | CodeCode Available | 1 | 5 |
| iBOT: Image BERT Pre-Training with Online Tokenizer | Nov 15, 2021 | image-classificationImage Classification | CodeCode Available | 1 | 5 |
| Data Efficient Masked Language Modeling for Vision and Language | Sep 5, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation | Mar 18, 2021 | Bilingual Lexicon InductionLanguage Modeling | CodeCode Available | 1 | 5 |
| Debiasing the Cloze Task in Sequential Recommendation with Bidirectional Transformers | Jan 22, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Declaration-based Prompt Tuning for Visual Question Answering | May 5, 2022 | Image-text matchingLanguage Modeling | CodeCode Available | 1 | 5 |
| ECAMP: Entity-centered Context-aware Medical Vision Language Pre-training | Dec 20, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain | May 20, 2023 | De-identificationLanguage Modeling | CodeCode Available | 1 | 5 |
| EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate | Dec 29, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| FiLM: Fill-in Language Models for Any-Order Generation | Oct 15, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model | Oct 11, 2022 | Contrastive LearningImage-text matching | CodeCode Available | 1 | 5 |
| FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Data | Oct 20, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| MC-BERT: Efficient Language Pre-Training via a Meta Controller | Jun 10, 2020 | Binary ClassificationCloze Test | CodeCode Available | 1 | 5 |
| DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning | May 17, 2023 | ClusteringLanguage Modeling | CodeCode Available | 1 | 5 |
| Eliciting Knowledge from Pretrained Language Models for Prototypical Prompt Verbalizer | Jan 14, 2022 | ClassificationContrastive Learning | CodeCode Available | 1 | 5 |
| Composable Sparse Fine-Tuning for Cross-Lingual Transfer | Oct 14, 2021 | Cross-Lingual TransferLanguage Modeling | CodeCode Available | 1 | 5 |
| MMBERT: Multimodal BERT Pretraining for Improved Medical VQA | Apr 3, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning | Aug 23, 2023 | In-Context LearningLanguage Modeling | CodeCode Available | 1 | 5 |
| Mask-Predict: Parallel Decoding of Conditional Masked Language Models | Apr 19, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
