SOTAVerified

Continual Pretraining

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 26–50 of 70 papers

TitleStatusHype
Bilingual Adaptation of Monolingual Foundation Models—0
70B-parameter large language models in Japanese medical question-answering—0
Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective—0
Open Generative Large Language Models for Galician—0
BAMBINO-LM: (Bilingual-)Human-Inspired Continual Pretraining of BabyLM—0
Towards Lifelong Learning of Large Language Models: A SurveyCode2
LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models—0
Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation PretrainingCode1
MoRA: High-Rank Updating for Parameter-Efficient Fine-TuningCode3
Cross-sensor self-supervised training and alignment for remote sensing—0
ChuXin: 1.6B Technical Report—0
Retrieval Head Mechanistically Explains Long-Context FactualityCode3
Pretraining and Updates of Domain-Specific LLM: A Case Study in the Japanese Business Domain—0
CEM: A Data-Efficient Method for Large Language Models to Continue Evolving From Mistakes—0
Rho-1: Not All Tokens Are What You NeedCode3
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code—0
PARAMANU-AYN: Pretrain from scratch or Continual Pretraining of LLMs for Legal Domain Adaptation?—0
Yi: Open Foundation Models by 01.AICode9
Investigating Continual Pretraining in Large Language Models: Insights and Implications—0
Data Engineering for Scaling Language Models to 128K ContextCode3
Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical TextsCode2
Continual Learning for Large Language Models: A Survey—0
RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via RomanizationCode0
PECoP: Parameter Efficient Continual Pretraining for Action Quality AssessmentCode0
Effective Long-Context Scaling of Foundation ModelsCode2
Show:102550
← PrevPage 2 of 3Next →
All datasetsACL ARCAG NewsSciERC

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1DASF1 (macro)0.69—Unverified
#ModelMetricClaimedVerifiedStatus
1CPTF1 - macro63.77—Unverified
#ModelMetricClaimedVerifiedStatus
1DASF1 (macro)0.71—Unverified