| Dynamic Masking Rate Schedules for MLM Pretraining | May 24, 2023 | Language ModelingLanguage Modelling | —Unverified | 0 |
| ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Prediction | Oct 13, 2023 | Click-Through Rate PredictionLanguage Modeling | —Unverified | 0 |
| A Progressive Transformer for Unifying Binary Code Embedding and Knowledge Transfer | Dec 15, 2024 | Feature EngineeringLanguage Modeling | —Unverified | 0 |
| Causal Distillation for Language Models | Jan 16, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |
| DS-TOD: Efficient Domain Specialization for Task-Oriented Dialog | Nov 16, 2021 | dialog state trackingLanguage Modeling | —Unverified | 0 |
| Adversarial Soft Prompt Tuning for Cross-Domain Sentiment Analysis | May 1, 2022 | Domain AdaptationLanguage Modeling | —Unverified | 0 |
| BPDec: Unveiling the Potential of Masked Language Modeling Decoder in BERT pretraining | Jan 29, 2024 | DecoderLanguage Modeling | —Unverified | 0 |
| A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives | Feb 25, 2021 | Contrastive LearningLanguage Modeling | —Unverified | 0 |
| Masked Vision and Language Modeling for Multi-modal Representation Learning | Aug 3, 2022 | cross-modal alignmentLanguage Modeling | —Unverified | 0 |
| MaskEval: Weighted MLM-Based Evaluation for Text Summarization and Simplification | May 24, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Maximizing Efficiency of Language Model Pre-training for Learning Representation | Oct 13, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Do Transformers Parse while Predicting the Masked Word? | Mar 14, 2023 | Constituency ParsingLanguage Modeling | —Unverified | 0 |
| Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling | Jan 25, 2024 | Causal Language ModelingDecoder | —Unverified | 0 |
| Capturing Topic Framing via Masked Language Modeling | Feb 7, 2023 | ArticlesLanguage Modeling | —Unverified | 0 |
| Domain-Specific Japanese ELECTRA Model Using a Small Corpus | Sep 1, 2021 | ArticlesComputational Efficiency | —Unverified | 0 |
| APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning | Dec 19, 2022 | Data AugmentationLanguage Modeling | —Unverified | 0 |
| Domain-adapted large language models for classifying nuclear medicine reports | Mar 1, 2023 | Domain AdaptationLanguage Modeling | —Unverified | 0 |
| Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge | Dec 16, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| CamemBERT 2.0: A Smarter French Language Model Aged to Perfection | Nov 13, 2024 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Adversarial Generation and Encoding of Nested Texts | Jun 1, 2019 | Language ModelingLanguage Modelling | —Unverified | 0 |
| A Pilot Study on Dialogue-Level Dependency Parsing for Chinese | May 21, 2023 | Dependency ParsingLanguage Modeling | —Unverified | 0 |
| Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little | Apr 14, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Discovering Financial Hypernyms by Prompting Masked Language Models | Jun 1, 2022 | Domain AdaptationLanguage Modeling | —Unverified | 0 |
| Improving the Reusability of Pre-trained Language Models in Real-world Applications | Jul 19, 2023 | Language ModelingLanguage Modelling | —Unverified | 0 |
| AntLM: Bridging Causal and Masked Language Models | Dec 4, 2024 | Causal Language ModelingDecoder | —Unverified | 0 |
| Abrupt Learning in Transformers: A Case Study on Matrix Completion | Oct 29, 2024 | Language ModelingLanguage Modelling | —Unverified | 0 |
| BIM: Block-Wise Self-Supervised Learning with Masked Image Modeling | Nov 28, 2023 | Contrastive LearningLanguage Modeling | —Unverified | 0 |
| Image BERT Pre-training with Online Tokenizer | Sep 29, 2021 | image-classificationImage Classification | —Unverified | 0 |
| DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries | Oct 23, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| BiLMa: Bidirectional Local-Matching for Text-based Person Re-identification | Sep 9, 2023 | Image to textLanguage Modeling | —Unverified | 0 |
| A Novel Two-Step Fine-Tuning Pipeline for Cold-Start Active Learning in Text Classification Tasks | Jul 24, 2024 | Active LearningDomain Adaptation | —Unverified | 0 |
| DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models | Oct 10, 2024 | Image GenerationLanguage Modeling | —Unverified | 0 |
| LLMcap: Large Language Model for Unsupervised PCAP Failure Detection | Jul 3, 2024 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Low-Resource Transliteration for Roman-Urdu and Urdu Using Transformer-Based Models | Mar 27, 2025 | Information RetrievalLanguage Modeling | —Unverified | 0 |
| Masked Language Modeling Becomes Conditional Density Estimation for Tabular Data Synthesis | May 31, 2024 | Density EstimationImputation | —Unverified | 0 |
| Bilingual Language Modeling, A transfer learning technique for Roman Urdu | Feb 22, 2021 | Cross-Lingual TransferLanguage Modeling | —Unverified | 0 |
| Leveraging Explicit Procedural Instructions for Data-Efficient Action Prediction | Jun 6, 2023 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Ankh3: Multi-Task Pretraining with Sequence Denoising and Completion Enhances Protein Representations | May 26, 2025 | DenoisingLanguage Modeling | —Unverified | 0 |
| Developing Language Resources and NLP Tools for the North Korean Language | Jun 1, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |
| LecPrompt: A Prompt-based Approach for Logical Error Correction with CodeBERT | Oct 10, 2024 | Language ModelingLanguage Modelling | —Unverified | 0 |
| How does the pre-training objective affect what large language models learn about linguistic properties? | Nov 16, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Developing Healthcare Language Model Embedding Spaces | Mar 28, 2024 | Contrastive LearningDocument Classification | —Unverified | 0 |
| Image as a Foreign Language: BEiT Pretraining for Vision and Vision-Language Tasks | Jan 1, 2023 | Cross-Modal RetrievalImage Captioning | —Unverified | 0 |
| ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data | Jan 22, 2020 | Image RetrievalImage-text matching | —Unverified | 0 |
| HOP+: History-enhanced and Order-aware Pre-training for Vision-and-Language Navigation | Mar 20, 2023 | Decision MakingLanguage Modeling | —Unverified | 0 |
| Improving BERT with Hybrid Pooling Network and Drop Mask | Jul 14, 2023 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Improving Low-Resource Morphological Inflection via Self-Supervised Objectives | Jun 5, 2025 | DecoderLanguage Modeling | —Unverified | 0 |
| Detecting Bias in Large Language Models: Fine-tuned KcBERT | Mar 16, 2024 | Language ModelingLanguage Modelling | —Unverified | 0 |
| CodeSSM: Towards State Space Models for Code Understanding | May 2, 2025 | Clone DetectionLanguage Modeling | —Unverified | 0 |
| Leveraging per Image-Token Consistency for Vision-Language Pre-training | Nov 20, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |