Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 301–350 of 2644 papers

Title	Date	Tasks	Status	Hype
Continual Learning with Echo State Networks	May 17, 2021	Continual Learning	CodeCode Available	1
Continual Action Assessment via Task-Consistent Score-Discriminative Feature Distribution Modeling	Sep 29, 2023	Action AssessmentAction Quality Assessment	CodeCode Available	1
A multifidelity approach to continual learning for physical systems	Apr 8, 2023	Continual Learning	CodeCode Available	1
Continual Learning with Scaled Gradient Projection	Feb 2, 2023	Continual Learningimage-classification	CodeCode Available	1
A Multi-Head Model for Continual Learning via Out-of-Distribution Replay	Aug 20, 2022	class-incremental learningClass Incremental Learning	CodeCode Available	1
On Tiny Episodic Memories in Continual Learning	Feb 27, 2019	Class Incremental LearningContinual Learning	CodeCode Available	1
Continual Multimodal Knowledge Graph Construction	May 15, 2023	Continual Learninggraph construction	CodeCode Available	1
BiasPruner: Debiased Continual Learning for Medical Image Classification	Jul 11, 2024	ClassificationContinual Learning	CodeCode Available	1
Continual Normalization: Rethinking Batch Normalization for Online Continual Learning	Mar 30, 2022	Continual Learning	CodeCode Available	1
Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism	May 6, 2022	Continual LearningDiversity	CodeCode Available	1
Bilevel Continual Learning	Jul 30, 2020	Bilevel OptimizationContinual Learning	CodeCode Available	1
Continual Prompt Tuning for Dialog State Tracking	Mar 13, 2022	Continual Learningdialog state tracking	CodeCode Available	1
Continual Reinforcement Learning with Multi-Timescale Replay	Apr 16, 2020	Continual Learningcontinuous-control	CodeCode Available	1
Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learning	Nov 29, 2023	Continual LearningContinual Self-Supervised Learning	CodeCode Available	1
Continual atlas-based segmentation of prostate MRI	Nov 1, 2023	Continual Learningimage-classification	CodeCode Available	1
Continual Variational Autoencoder Learning via Online Cooperative Memorization	Jul 20, 2022	Continual LearningDiversity	CodeCode Available	1
Continuous Coordination As a Realistic Scenario for Lifelong Learning	Mar 4, 2021	Continual LearningDeep Reinforcement Learning	CodeCode Available	1
Continuous Knowledge-Preserving Decomposition for Few-Shot Continual Learning	Jan 9, 2025	class-incremental learningClass Incremental Learning	CodeCode Available	1
Consistent Prompting for Rehearsal-Free Continual Learning	Mar 13, 2024	Continual Learning	CodeCode Available	1
Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases	Apr 18, 2023	Continual Learning	CodeCode Available	1
Cross-modal Learning for Domain Adaptation in 3D Semantic Segmentation	Jan 18, 2021	3D Semantic SegmentationAutonomous Driving	CodeCode Available	1
CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation	Aug 14, 2023	Continual LearningContinual Pretraining	CodeCode Available	1
DAM: Dynamic Adapter Merging for Continual Video QA Learning	Mar 13, 2024	Continual Learningimage-classification	CodeCode Available	1
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach	Sep 19, 2022	Bilevel OptimizationContinual Learning	CodeCode Available	1
A unifying framework for generalised Bayesian online learning in non-stationary environments	Nov 15, 2024	Continual LearningMulti-Armed Bandits	CodeCode Available	1
DataDAM: Efficient Dataset Distillation with Attention Matching	Sep 29, 2023	Continual LearningDataset Distillation	CodeCode Available	1
ConSlide: Asynchronous Hierarchical Interaction Transformer with Breakup-Reorganize Rehearsal for Continual Whole Slide Image Analysis	Aug 25, 2023	Continual LearningPrognosis	CodeCode Available	1
Continual Backprop: Stochastic Gradient Descent with Persistent Randomness	Aug 13, 2021	Continual LearningReinforcement Learning (RL)	CodeCode Available	1
Computationally Budgeted Continual Learning: What Does Matter?	Mar 20, 2023	Continual LearningGPU	CodeCode Available	1
Compositional Language Continual Learning	May 1, 2020	Continual LearningMachine Translation	CodeCode Available	1
Chameleon: A MatMul-Free Temporal Convolutional Network Accelerator for End-to-End Few-Shot and Continual Learning from Sequential Data	May 30, 2025	Continual LearningFew-Shot Learning	CodeCode Available	1
Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning	Sep 2, 2024	Continual LearningContrastive Learning	CodeCode Available	1
Distilled Replay: Overcoming Forgetting through Synthetic Samples	Mar 29, 2021	Continual Learning	CodeCode Available	1
Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition	Oct 8, 2022	Causal InferenceContinual Learning	CodeCode Available	1
Drinking from a Firehose: Continual Learning with Web-scale Natural Language	Jul 18, 2020	Continual Learning	CodeCode Available	1
Dropout as an Implicit Gating Mechanism For Continual Learning	Apr 24, 2020	Continual Learning	CodeCode Available	1
Conditional Channel Gated Networks for Task-Aware Continual Learning	Mar 31, 2020	Continual Learning	CodeCode Available	1
Brain-inspired replay for continual learning with artificial neural networks	Aug 13, 2020	class-incremental learningClass Incremental Learning	CodeCode Available	1
DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion	Nov 22, 2021	Class Incremental LearningContinual Learning	CodeCode Available	1
Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments	Dec 31, 2021	Continual LearningDeep Learning	CodeCode Available	1
An Empirical Investigation of the Role of Pre-training in Lifelong Learning	Dec 16, 2021	Continual Learningimage-classification	CodeCode Available	1
Efficient Continual Learning with Modular Networks and Task-Driven Priors	Dec 23, 2020	Continual Learning	CodeCode Available	1
Broken Neural Scaling Laws	Oct 26, 2022	Adversarial RobustnessContinual Learning	CodeCode Available	1
A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning	Jan 3, 2020	Continual Learningimage-classification	CodeCode Available	1
A Comprehensive Empirical Evaluation on Online Continual Learning	Aug 20, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	1
Building an Open-Vocabulary Video CLIP Model with Better Architectures, Optimization and Data	Oct 8, 2023	Action RecognitionContinual Learning	CodeCode Available	1
Building a Scalable and Interpretable Bayesian Deep Learning Framework for Quality Control of Free Form Surfaces	Apr 7, 1994	Active LearningBenchmarking	CodeCode Available	1
Engineering flexible machine learning systems by traversing functionally-invariant paths	Apr 30, 2022	Adversarial RobustnessContinual Learning	CodeCode Available	1
A Neural Span-Based Continual Named Entity Recognition Model	Feb 23, 2023	Continual LearningContinual Named Entity Recognition	CodeCode Available	1
CoLeCLIP: Open-Domain Continual Learning via Joint Task Prompt and Vocabulary Learning	Mar 15, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	1

Show:10 25 50

← PrevPage 7 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ProgressiveNet	Accuracy	76.16	—	Unverified
2	Piggyback	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	H$^{2}$	Accuracy	75.71	—	Unverified
6	PackNet	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified