SOTAVerified

Continual Pretraining

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 26–50 of 70 papers

TitleStatusHype
ChuXin: 1.6B Technical Report—0
Continual Learning for Large Language Models: A Survey—0
70B-parameter large language models in Japanese medical question-answering—0
Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective—0
Cross-sensor self-supervised training and alignment for remote sensing—0
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code—0
DD-TIG at Constraint@ACL2022: Multimodal Understanding and Reasoning for Role Labeling of Entities in Hateful Memes—0
DoPAMine: Domain-specific Pre-training Adaptation from seed-guided data Mining—0
Efficient Domain-adaptive Continual Pretraining for the Process Industry in the German Language—0
Enhance Mobile Agents Thinking Process Via Iterative Preference Learning—0
On the Robustness of Reading Comprehension Models to Entity Renaming—0
Open Generative Large Language Models for Galician—0
Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling—0
PARAMANU-AYN: Pretrain from scratch or Continual Pretraining of LLMs for Legal Domain Adaptation?—0
Pretraining and Updates of Domain-Specific LLM: A Case Study in the Japanese Business Domain—0
Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models—0
Revisiting Pretraining with Adapters—0
AfroXLMR-Social: Adapting Pre-trained Language Models for African Languages Social Media Text—0
The Construction of Instruction-tuned LLMs for Finance without Instruction Data Using Continual Pretraining and Model Merging—0
AdaPrompt: Adaptive Model Training for Prompt-based NLP—0
BAMBINO-LM: (Bilingual-)Human-Inspired Continual Pretraining of BabyLM—0
Bilingual Adaptation of Monolingual Foundation Models—0
Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content—0
Investigating Continual Pretraining in Large Language Models: Insights and Implications—0
Is Domain Adaptation Worth Your Investment? Comparing BERT and FinBERT on Financial Tasks—0
Show:102550
← PrevPage 2 of 3Next →
All datasetsACL ARCAG NewsSciERC

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1DASF1 (macro)0.69—Unverified
#ModelMetricClaimedVerifiedStatus
1CPTF1 - macro63.77—Unverified
#ModelMetricClaimedVerifiedStatus
1DASF1 (macro)0.71—Unverified