Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1051–1100 of 2644 papers

Title	Date	Tasks	Status	Score
Generative vs. Discriminative: Rethinking The Meta-Continual Learning	Dec 1, 2021	Continual LearningMeta-Learning	CodeCode Available	5
Gradient-based Maximally Interfered Retrieval for Domain Incremental 3D Object Detection	Apr 27, 2023	3D Object DetectionAll	CodeCode Available	5
Encoding Binary Concepts in the Latent Space of Generative Models for Enhancing Data Representation	Mar 22, 2023	Continual LearningDisentanglement	CodeCode Available	5
Generative Models from the perspective of Continual Learning	Dec 21, 2018	Continual LearningImage Generation	CodeCode Available	5
Category Adaptation Meets Projected Distillation in Generalized Continual Category Discovery	Aug 23, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	5
Context selectivity with dynamic availability enables lifelong continual learning	Jun 2, 2023	Continual LearningLifelong learning	CodeCode Available	5
BACS: Background Aware Continual Semantic Segmentation	Apr 19, 2024	Autonomous DrivingContinual Learning	CodeCode Available	5
Continual Developmental Neurosimulation Using Embodied Computational Agents	Mar 7, 2021	Active LearningContinual Learning	CodeCode Available	5
From Static to Dynamic: A Continual Learning Framework for Large Language Models	Oct 22, 2023	Continual Learning	CodeCode Available	5
Functional Regularisation for Continual Learning with Gaussian Processes	Jan 31, 2019	Bayesian InferenceContinual Learning	CodeCode Available	5
Gradient Correlation Subspace Learning against Catastrophic Forgetting	Mar 4, 2024	Continual Learning	CodeCode Available	5
FPPL: An Efficient and Non-IID Robust Federated Continual Learning Framework	Nov 4, 2024	Continual LearningContrastive Learning	CodeCode Available	5
From Lab to Pocket: A Novel Continual Learning-based Mobile Application for Screening COVID-19	Oct 16, 2024	Continual Learning	CodeCode Available	5
Learning to Learn without Forgetting using Attention	Aug 6, 2024	Continual LearningMeta-Learning	CodeCode Available	5
Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning	Sep 28, 2024	Continual LearningMaximum Separation	CodeCode Available	5
From MNIST to ImageNet and Back: Benchmarking Continual Curriculum Learning	Mar 16, 2023	BenchmarkingContinual Learning	CodeCode Available	5
Continual Dialogue State Tracking via Reason-of-Select Distillation	Aug 19, 2024	Continual LearningDialogue State Tracking	CodeCode Available	5
AVQACL: A Novel Benchmark for Audio-Visual Question Answering Continual Learning	Jan 1, 2025	Audio-visual Question AnsweringContinual Learning	CodeCode Available	5
Continual Dialogue State Tracking via Example-Guided Question Answering	May 23, 2023	Continual LearningDialogue State Tracking	CodeCode Available	5
Flatness-aware Sequential Learning Generates Resilient Backdoors	Jul 20, 2024	Continual Learning	CodeCode Available	5
F-MALLOC: Feed-forward Memory Allocation for Continual Learning in Neural Machine Translation	Apr 7, 2024	Continual LearningMachine Translation	CodeCode Available	5
Continual Deep Reinforcement Learning with Task-Agnostic Policy Distillation	Nov 25, 2024	Continual LearningDeep Reinforcement Learning	CodeCode Available	5
Few-shot Tuning of Foundation Models for Class-incremental Learning	May 26, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	5
Function-space Parameterization of Neural Networks for Sequential Learning	Mar 16, 2024	Continual LearningGaussian Processes	CodeCode Available	5
AGALE: A Graph-Aware Continual Learning Evaluation Framework	Jun 3, 2024	Continual LearningGraph Learning	CodeCode Available	5
Fine-tuning can cripple your foundation model; preserving features may be the solution	Aug 25, 2023	Continual Learning	CodeCode Available	5
FOCIL: Finetune-and-Freeze for Online Class Incremental Learning by Training Randomly Pruned Sparse Experts	Mar 13, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	5
Gradient Episodic Memory for Continual Learning	Jun 26, 2017	Continual LearningIncremental Learning	CodeCode Available	5
Efficient Lifelong Learning with A-GEM	Dec 2, 2018	class-incremental learningClass Incremental Learning	CodeCode Available	5
Continual Deep Active Learning for Medical Imaging: Replay-Base Architecture for Context Adaptation	Jan 14, 2025	Active Learningclass-incremental learning	CodeCode Available	5
Federated Learning of Socially Appropriate Agent Behaviours in Simulated Home Environments	Mar 12, 2024	Continual LearningFederated Learning	CodeCode Available	5
Federated Continual Graph Learning	Nov 28, 2024	Continual LearningGraph Learning	CodeCode Available	5
Federated Continual Learning for Text Classification via Selective Inter-client Transfer	Oct 12, 2022	Continual LearningFederated Learning	CodeCode Available	5
Continual Contrastive Learning for Image Classification	Jul 5, 2021	ClassificationContinual Learning	CodeCode Available	5
Efficient Continual Learning in Neural Networks with Embedding Regularization	Sep 9, 2019	Continual LearningLifelong learning	CodeCode Available	5
Exploring the Stability Gap in Continual Learning: The Role of the Classification Head	Nov 6, 2024	AttributeContinual Learning	CodeCode Available	5
An Investigation of the Combination of Rehearsal and Knowledge Distillation in Continual Learning for Spoken Language Understanding	Nov 15, 2022	class-incremental learningClass Incremental Learning	CodeCode Available	5
Exploring Open-world Continual Learning with Knowns-Unknowns Knowledge Transfer	Feb 27, 2025	Continual LearningTransfer Learning	CodeCode Available	5
Exploring the Evolution of Hidden Activations with Live-Update Visualization	May 24, 2024	Continual Learning	CodeCode Available	5
Exploring Continual Learning of Compositional Generalization in NLI	Mar 7, 2024	Continual LearningNatural Language Inference	CodeCode Available	5
Exploring Data Redundancy in Real-world Image Classification through Data Selection	Jun 25, 2023	Active LearningContinual Learning	CodeCode Available	5
Adapt before Continual Learning	Jun 4, 2025	Continual Learning	CodeCode Available	5
Expanding continual few-shot learning benchmarks to include recognition of specific instances	Aug 26, 2022	continual few-shot learningContinual Learning	CodeCode Available	5
Autonomous Deep Learning: Continual Learning Approach for Dynamic Environments	Oct 17, 2018	Continual LearningDeep Learning	CodeCode Available	5
Exploring Continual Learning for Code Generation Models	Jul 5, 2023	Code GenerationContinual Learning	CodeCode Available	5
Helpful or Harmful: Inter-Task Association in Continual Learning	Oct 23, 2022	Class Incremental LearningContinual Learning	CodeCode Available	5
Continual Audio-Visual Sound Separation	Nov 5, 2024	Continual LearningSemantic Similarity	CodeCode Available	5
Exclusive Supermask Subnetwork Training for Continual Learning	Oct 18, 2022	Continual LearningText Classification	CodeCode Available	5
Ex-Model: Continual Learning from a Stream of Trained Models	Dec 13, 2021	Continual Learning	CodeCode Available	5
Effect of Optimizer, Initializer, and Architecture of Hypernetworks on Continual Learning from Demonstration	Dec 31, 2023	Continual Learning	CodeCode Available	5

Show:10 25 50

← PrevPage 22 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified