| UFO: A UniFied TransfOrmer for Vision-Language Representation Learning | Nov 19, 2021 | Image CaptioningImage-text matching | —Unverified | 0 | 0 |
| BPDec: Unveiling the Potential of Masked Language Modeling Decoder in BERT pretraining | Jan 29, 2024 | DecoderLanguage Modeling | —Unverified | 0 | 0 |
| Do Transformers Parse while Predicting the Masked Word? | Mar 14, 2023 | Constituency ParsingLanguage Modeling | —Unverified | 0 | 0 |
| On the Influence of Masking Policies in Intermediate Pre-training | Apr 18, 2021 | Abstractive Text SummarizationLanguage Modeling | —Unverified | 0 | 0 |
| OPSD: an Offensive Persian Social media Dataset and its baseline evaluations | Apr 8, 2024 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Mapping of attention mechanisms to a generalized Potts model | Apr 14, 2023 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling | Jan 25, 2024 | Causal Language ModelingDecoder | —Unverified | 0 | 0 |
| PASTA: Pretrained Action-State Transformer Agents | Jul 20, 2023 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Patton: Language Model Pretraining on Text-Rich Networks | May 20, 2023 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| UHH-LT at SemEval-2020 Task 12: Fine-Tuning of Pre-Trained Transformer Networks for Offensive Language Detection | Apr 23, 2020 | Domain AdaptationGeneral Classification | —Unverified | 0 | 0 |
| Domain-Specific Japanese ELECTRA Model Using a Small Corpus | Sep 1, 2021 | ArticlesComputational Efficiency | —Unverified | 0 | 0 |
| PerPLM: Personalized Fine-tuning of Pretrained Language Models via Writer-specific Intermediate Learning and Prompts | Sep 14, 2023 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression | Jun 1, 2023 | Contrastive LearningData Augmentation | —Unverified | 0 | 0 |
| Phrase-aware Unsupervised Constituency Parsing | Nov 16, 2021 | Constituency ParsingLanguage Modeling | —Unverified | 0 | 0 |
| Phrase-aware Unsupervised Constituency Parsing | May 1, 2022 | Constituency ParsingLanguage Modeling | —Unverified | 0 | 0 |
| Understanding Chinese Video and Language via Contrastive Multimodal Pre-Training | Apr 19, 2021 | Contrastive LearningLanguage Modeling | —Unverified | 0 | 0 |
| Understanding the Natural Language of DNA using Encoder-Decoder Foundation Models with Byte-level Precision | Nov 4, 2023 | DecoderLanguage Modeling | —Unverified | 0 | 0 |
| Domain-adapted large language models for classifying nuclear medicine reports | Mar 1, 2023 | Domain AdaptationLanguage Modeling | —Unverified | 0 | 0 |
| Position Masking for Language Models | Jun 2, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| POSTECH-ETRI’s Submission to the WMT2020 APE Shared Task: Automatic Post-Editing with Cross-lingual Language Model | Nov 1, 2020 | Automatic Post-EditingLanguage Modeling | —Unverified | 0 | 0 |
| Predicting Attention Sparsity in Transformers | Sep 24, 2021 | DecoderLanguage Modeling | —Unverified | 0 | 0 |
| Predicting Attention Sparsity in Transformers | Nov 16, 2021 | DecoderLanguage Modeling | —Unverified | 0 | 0 |
| Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge | Dec 16, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Discovering Financial Hypernyms by Prompting Masked Language Models | Jun 1, 2022 | Domain AdaptationLanguage Modeling | —Unverified | 0 | 0 |
| Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs | Jul 22, 2024 | Few-Shot LearningGraph Neural Network | —Unverified | 0 | 0 |
| Pretraining Chinese BERT for Detecting Word Insertion and Deletion Errors | Apr 26, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning | Apr 29, 2020 | AllHellaSwag | —Unverified | 0 | 0 |
| Pre-training Language Model as a Multi-perspective Course Learner | May 6, 2023 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training | Aug 16, 2019 | Image-text matchingImage-text Retrieval | —Unverified | 0 | 0 |
| DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries | Oct 23, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Ankh3: Multi-Task Pretraining with Sequence Denoising and Completion Enhances Protein Representations | May 26, 2025 | DenoisingLanguage Modeling | —Unverified | 0 | 0 |
| DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models | Oct 10, 2024 | Image GenerationLanguage Modeling | —Unverified | 0 | 0 |
| Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation | Dec 10, 2021 | Image-text matchingImage-text Retrieval | —Unverified | 0 | 0 |
| Uniform Masking Prevails in Vision-Language Pretraining | Dec 10, 2022 | Image-text matchingLanguage Modeling | —Unverified | 0 | 0 |
| Probing BERT’s priors with serial reproduction chains | Nov 16, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models | Dec 1, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| UNITER: Learning UNiversal Image-TExt Representations | Sep 25, 2019 | Image-text matchingImage-text Retrieval | —Unverified | 0 | 0 |
| A Hierarchical Multi-Modal Encoder for Moment Localization in Video Corpus | Nov 18, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Prompt-Guided Injection of Conformation to Pre-trained Protein Model | Feb 7, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Prompt-Learning for Fine-Grained Entity Typing | Aug 24, 2021 | Entity TypingKnowledge Probing | —Unverified | 0 | 0 |
| Prompt-Learning for Fine-Grained Entity Typing | Nov 16, 2021 | Entity TypingKnowledge Probing | —Unverified | 0 | 0 |
| Pseudo-Label Guided Unsupervised Domain Adaptation of Contextual Embeddings | Apr 1, 2021 | Domain AdaptationLanguage Modeling | —Unverified | 0 | 0 |
| Pseudo-perplexity in One Fell Swoop for Protein Fitness Estimation | Jul 9, 2024 | Computational EfficiencyLanguage Modeling | —Unverified | 0 | 0 |
| Universal Sentence Representation Learning with Conditional Masked Language Model | Dec 28, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Developing Language Resources and NLP Tools for the North Korean Language | Jun 1, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Universal Sentence Representations Learning with Conditional Masked Language Model | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| WordAlchemy: A transformer-based Reverse Dictionary | Apr 16, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 | 0 |
| Developing Healthcare Language Model Embedding Spaces | Mar 28, 2024 | Contrastive LearningDocument Classification | —Unverified | 0 | 0 |
| Unsupervised Dependency Graph Network | Nov 16, 2021 | Dependency ParsingLanguage Modeling | —Unverified | 0 | 0 |
| Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models | Jun 14, 2023 | DecoderLanguage Modeling | —Unverified | 0 | 0 |