SOTAVerified

Continual Pretraining

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–25 of 70 papers

TitleStatusHype
Yi: Open Foundation Models by 01.AICode9
Scaling Granite Code Models to 128K ContextCode4
MoRA: High-Rank Updating for Parameter-Efficient Fine-TuningCode3
Retrieval Head Mechanistically Explains Long-Context FactualityCode3
Rho-1: Not All Tokens Are What You NeedCode3
Data Engineering for Scaling Language Models to 128K ContextCode3
A Practitioner's Guide to Continual Multimodal PretrainingCode2
Towards Lifelong Learning of Large Language Models: A SurveyCode2
Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical TextsCode2
Effective Long-Context Scaling of Foundation ModelsCode2
Continual Pre-training of Language ModelsCode2
Continual Training of Language Models for Few-Shot LearningCode2
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM PretrainingCode1
Demystifying Domain-adaptive Post-training for Financial LLMsCode1
NyayaAnumana & INLegalLlama: The Largest Indian Legal Judgment Prediction Dataset and Specialized Language Model for Enhanced Decision AnalysisCode1
Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation PretrainingCode1
CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology PreservationCode1
Towards Geospatial Foundation Models via Continual PretrainingCode1
CTP:Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology PreservationCode1
Continual Pre-Training Mitigates Forgetting in Language and VisionCode1
On the Robustness of Reading Comprehension Models to Entity RenamingCode1
Efficient Contrastive Learning via Novel Data Augmentation and Curriculum LearningCode1
ECONET: Effective Continual Pretraining of Language Models for Event Temporal ReasoningCode1
Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content—0
LLaVA-c: Continual Improved Visual Instruction Tuning—0
Show:102550
← PrevPage 1 of 3Next →
All datasetsACL ARCAG NewsSciERC

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1DASF1 (macro)0.69—Unverified
#ModelMetricClaimedVerifiedStatus
1CPTF1 - macro63.77—Unverified
#ModelMetricClaimedVerifiedStatus
1DASF1 (macro)0.71—Unverified