Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1–50 of 2644 papers

Title	Date	Tasks	Status	Hype
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications	Mar 10, 2025	Continual LearningMeta-Learning	CodeCode Available	9
Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems	Mar 31, 2025	AutoMLContinual Learning	CodeCode Available	7
From RAG to Memory: Non-Parametric Continual Learning for Large Language Models	Feb 20, 2025	Continual LearningKnowledge Graphs	CodeCode Available	7
On the Computation of the Fisher Information in Continual Learning	Feb 17, 2025	Continual Learning	CodeCode Available	5
Continual Learning of Large Language Models: A Comprehensive Survey	Apr 25, 2024	Continual LearningSurvey	CodeCode Available	4
The Ideal Continual Learner: An Agent That Never Forgets	Apr 29, 2023	Continual LearningGeneralization Bounds	CodeCode Available	4
The Thousand Brains Project: A New Paradigm for Sensorimotor Intelligence	Dec 24, 2024	Continual Learning	CodeCode Available	4
Continual Learning with Pre-Trained Models: A Survey	Jan 29, 2024	Continual LearningFairness	CodeCode Available	4
Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities	Aug 14, 2024	Continual LearningFew-Shot Learning	CodeCode Available	4
River: machine learning for streaming data in Python	Dec 8, 2020	BIG-bench Machine LearningContinual Learning	CodeCode Available	3
Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis	May 16, 2025	Continual LearningRepresentation Learning	CodeCode Available	3
OmDet: Large-scale vision-language multi-dataset pre-training with multimodal detection network	Sep 10, 2022	Continual LearningObject	CodeCode Available	3
TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree	Jun 12, 2025	Continual Learning	CodeCode Available	3
Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters	Mar 18, 2024	Continual LearningIncremental Learning	CodeCode Available	3
SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training	Aug 15, 2024	Continual Learningimage-classification	CodeCode Available	2
Recent Advances of Multimodal Continual Learning: A Comprehensive Survey	Oct 7, 2024	Continual LearningSurvey	CodeCode Available	2
Spurious Forgetting in Continual Learning of Language Models	Jan 23, 2025	Continual Learning	CodeCode Available	2
PILOT: A Pre-Trained Model-Based Continual Learning Toolbox	Sep 13, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	2
Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Models	Jul 7, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	2
MR-GDINO: Efficient Open-World Continual Object Detection	Dec 20, 2024	Continual Learningobject-detection	CodeCode Available	2
One-for-More: Continual Diffusion Model for Anomaly Detection	Feb 27, 2025	Anomaly Detectioncontinual anomaly detection	CodeCode Available	2
Learning without Forgetting	Jun 29, 2016	Class Incremental LearningContinual Learning	CodeCode Available	2
KAN or MLP: A Fairer Comparison	Jul 23, 2024	Continual Learning	CodeCode Available	2
A Practitioner's Guide to Continual Multimodal Pretraining	Aug 26, 2024	Continual LearningContinual Pretraining	CodeCode Available	2
Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptation	Nov 8, 2024	Continual Learning	CodeCode Available	2
LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error	Mar 7, 2024	Continual LearningIn-Context Learning	CodeCode Available	2
InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning	Mar 30, 2024	Continual Learningparameter-efficient fine-tuning	CodeCode Available	2
Renate: A Library for Real-World Continual Learning	Apr 24, 2023	Continual Learning	CodeCode Available	2
SegViTv2: Exploring Efficient and Continual Semantic Segmentation with Plain Vision Transformers	Jun 9, 2023	Continual LearningContinual Semantic Segmentation	CodeCode Available	2
SHINE-Mapping: Large-Scale 3D Mapping Using Sparse Hierarchical Implicit Neural Representations	Oct 5, 2022	3D ReconstructionContinual Learning	CodeCode Available	2
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning	Apr 10, 2022	Continual Learning	CodeCode Available	2
Interactive Continual Learning: Fast and Slow Thinking	Mar 5, 2024	Continual LearningOutlier Detection	CodeCode Available	2
LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation	Apr 10, 2025	Code GenerationContinual Learning	CodeCode Available	2
CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation	Aug 26, 2024	Continual Learning	CodeCode Available	2
Class-Incremental Learning: A Survey	Feb 7, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	2
Continual Learning on Graphs: Challenges, Solutions, and Opportunities	Feb 18, 2024	Continual LearningGraph Learning	CodeCode Available	2
Continual Training of Language Models for Few-Shot Learning	Oct 11, 2022	Continual LearningContinual Pretraining	CodeCode Available	2
DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning	Apr 18, 2022	Active LearningContinual Learning	CodeCode Available	2
Adapting a Language Model While Preserving its General Knowledge	Jan 21, 2023	Continual LearningGeneral Knowledge	CodeCode Available	2
ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning	Mar 29, 2024	Continual LearningContinual Panoptic Segmentation	CodeCode Available	2
Generative replay with feedback connections as a general strategy for continual learning	Sep 27, 2018	Continual LearningLifelong learning	CodeCode Available	2
HiDe-PET: Continual Learning via Hierarchical Decomposition of Parameter-Efficient Tuning	Jul 7, 2024	Continual LearningRepresentation Learning	CodeCode Available	2
iSDF: Real-Time Neural Signed Distance Fields for Robot Perception	Apr 5, 2022	Continual LearningDenoising	CodeCode Available	2
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning	Jul 16, 2023	Continual LearningFederated Learning	CodeCode Available	2
COALA: A Practical and Vision-Centric Federated Learning Platform	Jul 23, 2024	BenchmarkingContinual Learning	CodeCode Available	2
A Comprehensive Survey on Continual Learning in Generative Models	Jun 16, 2025	Continual LearningSurvey	CodeCode Available	2
Continual Pre-training of Language Models	Feb 7, 2023	Continual LearningContinual Pretraining	CodeCode Available	2
Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Models	Sep 30, 2024	BenchmarkingContinual Learning	CodeCode Available	2
CloSe: A 3D Clothing Segmentation Dataset and Model	Jan 22, 2024	Continual Learningmodel	CodeCode Available	2
Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives	Feb 5, 2024	Continual LearningMulti-Task Learning	CodeCode Available	2

Show:10 25 50

← PrevPage 1 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified