Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 501–550 of 2644 papers

Title	Date	Tasks	Status	Hype	Score
A Unified Continual Learning Framework with General Parameter-Efficient Tuning	Mar 17, 2023	Continual Learning	CodeCode Available	1	5
Gradient Projection Memory for Continual Learning	Mar 17, 2021	AttributeContinual Learning	CodeCode Available	1	5
Continual Reinforcement Learning with Multi-Timescale Replay	Apr 16, 2020	Continual Learningcontinuous-control	CodeCode Available	1	5
Unsupervised Model Personalization while Preserving Privacy and Scalability: An Open Problem	Mar 30, 2020	Continual LearningDomain Adaptation	CodeCode Available	1	5
GUIDE: Guidance-based Incremental Learning with Diffusion Models	Mar 6, 2024	Continual LearningIncremental Learning	CodeCode Available	1	5
Continual Action Assessment via Task-Consistent Score-Discriminative Feature Distribution Modeling	Sep 29, 2023	Action AssessmentAction Quality Assessment	CodeCode Available	1	5
Hardware-agnostic Computation for Large-scale Knowledge Graph Embeddings	Jul 18, 2022	Continual LearningGraph Embedding	CodeCode Available	1	5
Vector Quantization Prompting for Continual Learning	Oct 27, 2024	Continual LearningQuantization	CodeCode Available	1	5
Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks	Feb 19, 2024	Continual Learning	CodeCode Available	1	5
HCV: Hierarchy-Consistency Verification for Incremental Implicitly-Refined Classification	Oct 21, 2021	ClassificationContinual Learning	CodeCode Available	1	5
Mitigating the Alignment Tax of RLHF	Sep 12, 2023	Common Sense ReasoningContinual Learning	CodeCode Available	1	5
HPCR: Holistic Proxy-based Contrastive Replay for Online Continual Learning	Sep 26, 2023	Continual Learning	CodeCode Available	1	5
Imbalanced Continual Learning with Partitioning Reservoir Sampling	Sep 8, 2020	Continual LearningMulti-Label Classification	CodeCode Available	1	5
Importance Driven Continual Learning for Segmentation Across Domains	Apr 30, 2020	Brain SegmentationContinual Learning	CodeCode Available	1	5
Automatic Recall Machines: Internal Replay, Continual Learning and the Brain	Jun 22, 2020	Continual Learning	CodeCode Available	1	5
Continual atlas-based segmentation of prostate MRI	Nov 1, 2023	Continual Learningimage-classification	CodeCode Available	1	5
Automating Continual Learning	Dec 1, 2023	Continual Learningimage-classification	CodeCode Available	1	5
Continual Backprop: Stochastic Gradient Descent with Persistent Randomness	Aug 13, 2021	Continual LearningReinforcement Learning (RL)	CodeCode Available	1	5
InfoCL: Alleviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective	Oct 10, 2023	ClassificationContinual Learning	CodeCode Available	1	5
Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature	Jul 7, 2020	Continual LearningExtractive Summarization	CodeCode Available	1	5
Avalanche: an End-to-End Library for Continual Learning	Apr 1, 2021	class-incremental learningClass Incremental Learning	CodeCode Available	1	5
Continual Learning at the Edge: Real-Time Training on Smartphone Devices	May 24, 2021	Continual Learning	CodeCode Available	1	5
Continual Learning for Abdominal Multi-Organ and Tumor Segmentation	Jun 1, 2023	Continual LearningOrgan Segmentation	CodeCode Available	1	5
Continual Learning, Fast and Slow	Sep 6, 2022	Continual LearningHippocampus	CodeCode Available	1	5
Avalanche RL: a Continual Reinforcement Learning Library	Feb 28, 2022	Continual LearningOpenAI Gym	CodeCode Available	1	5
Improving Task-free Continual Learning by Distributionally Robust Memory Evolution	Jul 15, 2022	Continual Learning	CodeCode Available	1	5
Learning Causal Models Online	Jun 12, 2020	Continual Learning	CodeCode Available	1	5
Continual Learning for Image Segmentation with Dynamic Query	Nov 29, 2023	Continual LearningDiversity	CodeCode Available	1	5
Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments	Dec 31, 2021	Continual LearningDeep Learning	CodeCode Available	1	5
Continual Learning for Image-Based Camera Localization	Aug 20, 2021	Autonomous DrivingCamera Localization	CodeCode Available	1	5
Lifelong Learning of Compositional Structures	Jul 15, 2020	Continual LearningLifelong learning	CodeCode Available	1	5
FLUID: A Unified Evaluation Framework for Flexible Sequential Data	Jul 6, 2020	Continual LearningRepresentation Learning	CodeCode Available	1	5
MRN: Multiplexed Routing Network for Incremental Multilingual Text Recognition	May 24, 2023	Continual LearningIncremental Learning	CodeCode Available	1	5
Is Parameter Collision Hindering Continual Learning in LLMs?	Oct 14, 2024	Continual Learning	CodeCode Available	1	5
Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases	Apr 18, 2023	Continual Learning	CodeCode Available	1	5
Continual Learning For On-Device Environmental Sound Classification	Jul 15, 2022	ClassificationComputational Efficiency	CodeCode Available	1	5
Continual Learning for Visual Search with Backward Consistent Feature Embedding	May 26, 2022	Continual Learning	CodeCode Available	1	5
Continual Domain Adaptation through Pruning-aided Domain-specific Weight Modulation	Apr 15, 2023	Continual LearningDomain Adaptation	CodeCode Available	1	5
Continual Learning for Unsupervised Anomaly Detection in Continuous Auditing of Financial Accounting Data	Dec 25, 2021	Anomaly Detectioncontinual anomaly detection	CodeCode Available	1	5
Language Guided Concept Bottleneck Models for Interpretable Continual Learning	Mar 30, 2025	Continual LearningDecision Making	CodeCode Available	1	5
Continual evaluation for lifelong learning: Identifying the stability gap	May 26, 2022	Continual LearningIncremental Learning	CodeCode Available	1	5
Continual Evidential Deep Learning for Out-of-Distribution Detection	Sep 6, 2023	Continual LearningDeep Learning	CodeCode Available	1	5
AIR: Analytic Imbalance Rectifier for Continual Learning	Aug 19, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	1	5
Continual Learning in Low-rank Orthogonal Subspaces	Oct 22, 2020	Continual Learning	CodeCode Available	1	5
Continual Learning of Control Primitives: Skill Discovery via Reset-Games	Nov 10, 2020	Continual Learning	CodeCode Available	1	5
Learning Equi-angular Representations for Online Continual Learning	Apr 2, 2024	Continual Learning	CodeCode Available	1	5
Continual learning with hypernetworks	Jun 3, 2019	Continual LearningTransfer Learning	CodeCode Available	1	5
Continual Learning in Recurrent Neural Networks	Jun 22, 2020	Continual Learning	CodeCode Available	1	5
Continual Hippocampus Segmentation with Transformers	Apr 17, 2022	Continual LearningHippocampus	CodeCode Available	1	5
Progressive Neural Networks	Jun 15, 2016	Continual Learningreinforcement-learning	CodeCode Available	1	5

Show:10 25 50

← PrevPage 11 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified