Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 2101–2150 of 2644 papers

Title	Date	Tasks	Status
Overcoming Mode Collapse with Adaptive Multi Adversarial Training	Dec 29, 2021	Continual Learning	CodeCode Available
Continual Learning with Invertible Generative Models	Feb 11, 2022	Continual Learning	CodeCode Available
HAT-CL: A Hard-Attention-to-the-Task PyTorch Library for Continual Learning	Jul 18, 2023	Continual LearningHard Attention	CodeCode Available
Overcoming Recency Bias of Normalization Statistics in Continual Learning: Balance and Adaptation	Oct 13, 2023	Continual LearningSplit-CIFAR-10	CodeCode Available
Improving Data-aware and Parameter-aware Robustness for Continual Learning	May 27, 2024	Continual Learning	CodeCode Available
Wearable Sensor-Based Few-Shot Continual Learning on Hand Gestures for Motor-Impaired Individuals via Latent Embedding Exploitation	May 14, 2024	Continual LearningGesture Recognition	CodeCode Available
Improving Forward Compatibility in Class Incremental Learning by Increasing Representation Rank and Feature Richness	Mar 22, 2024	class-incremental learningClass Incremental Learning	CodeCode Available
Continual Learning with Guarantees via Weight Interval Constraints	Jun 16, 2022	Continual Learning	CodeCode Available
Toward Universal Medical Image Registration via Sharpness-Aware Meta-Continual Learning	Jun 25, 2024	Continual LearningImage Registration	CodeCode Available
Continual Learning with Dynamic Sparse Training: Exploring Algorithms for Effective Model Updates	Aug 28, 2023	Continual Learning	CodeCode Available
Improving Meta-Continual Learning Representations with Representation Replay	Sep 29, 2021	Continual LearningMeta-Learning	CodeCode Available
A Strategy for an Uncompromising Incremental Learner	May 2, 2017	class-incremental learningClass Incremental Learning	CodeCode Available
Improving Online Continual Learning Performance and Stability with Temporal Ensembles	Jun 29, 2023	Continual Learning	CodeCode Available
The Impact of Cross-Lingual Adjustment of Contextual Word Representations on Zero-Shot Transfer	Apr 13, 2022	Continual LearningCross-Lingual Transfer	CodeCode Available
H2ST: Hierarchical Two-Sample Tests for Continual Out-of-Distribution Detection	Jan 1, 2025	Continual LearningIncremental Learning	CodeCode Available
Improving Replay-Based Continual Semantic Segmentation with Smart Data Selection	Sep 20, 2022	Continual LearningContinual Semantic Segmentation	CodeCode Available
Adapt & Align: Continual Learning with Generative Models Latent Space Alignment	Dec 21, 2023	Continual Learning	CodeCode Available
Improving Representational Continuity via Continued Pretraining	Feb 26, 2023	Continual LearningRepresentation Learning	CodeCode Available
Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate	Jul 8, 2025	Continual LearningMixture-of-Experts	CodeCode Available
In-context Continual Learning Assisted by an External Continual Learner	Dec 20, 2024	Continual LearningIn-Context Learning	CodeCode Available
Incorporating brain-inspired mechanisms for multimodal learning in artificial intelligence	May 15, 2025	Computational EfficiencyContinual Learning	CodeCode Available
Growing a Brain with Sparsity-Inducing Generation for Continual Learning	Jan 1, 2023	Action RecognitionContinual Learning	CodeCode Available
A Natural Gas Consumption Forecasting System for Continual Learning Scenarios based on Hoeffding Trees with Change Point Detection Mechanism	Sep 7, 2023	Change Point DetectionContinual Learning	CodeCode Available
CITB: A Benchmark for Continual Instruction Tuning	Oct 23, 2023	Continual Learning	CodeCode Available
Gradient Episodic Memory for Continual Learning	Jun 26, 2017	Continual LearningIncremental Learning	CodeCode Available
Overcoming Catastrophic Forgetting via Direction-Constrained Optimization	Nov 25, 2020	Continual Learning	CodeCode Available
An Attention-based Representation Distillation Baseline for Multi-Label Continual Learning	Jul 19, 2024	Continual LearningTransfer Learning	CodeCode Available
The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translation	May 16, 2023	Abstractive Text SummarizationContinual Learning	CodeCode Available
Continual Learning with Differential Privacy	Oct 11, 2021	Continual Learning	CodeCode Available
Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognition	Nov 9, 2021	Continual LearningKnowledge Distillation	CodeCode Available
Sequential Function-Space Variational Inference via Gaussian Mixture Approximation	Mar 10, 2025	Continual LearningVariational Inference	CodeCode Available
Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-off	Mar 30, 2025	Continual Learning	CodeCode Available
Incremental Task Learning with Incremental Rank Updates	Jul 19, 2022	Continual Learning	CodeCode Available
Exploiting the Semantic Knowledge of Pre-trained Text-Encoders for Continual Learning	Aug 2, 2024	Continual LearningKnowledge Distillation	CodeCode Available
Partial Hypernetworks for Continual Learning	Jun 19, 2023	Continual Learning	CodeCode Available
iNeMo: Incremental Neural Mesh Models for Robust Class-Incremental Learning	Jul 12, 2024	class-incremental learningClass Incremental Learning	CodeCode Available
Gradient Correlation Subspace Learning against Catastrophic Forgetting	Mar 4, 2024	Continual Learning	CodeCode Available
Gradient-based Maximally Interfered Retrieval for Domain Incremental 3D Object Detection	Apr 27, 2023	3D Object DetectionAll	CodeCode Available
Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning	Aug 29, 2024	Continual LearningMulti-Task Learning	CodeCode Available
Analyzing BERT Cross-lingual Transfer Capabilities in Continual Sequence Labeling	Oct 1, 2022	Continual LearningCross-Lingual Transfer	CodeCode Available
PaRT: Parallel Learning Towards Robust and Transparent AI	Jan 24, 2022	Continual LearningMulti-Task Learning	CodeCode Available
Patch-Based Contrastive Learning and Memory Consolidation for Online Unsupervised Continual Learning	Sep 24, 2024	Continual LearningContrastive Learning	CodeCode Available
Continual Learning with Deep Streaming Regularized Discriminant Analysis	Sep 15, 2023	Continual LearningImage Classification	CodeCode Available
PathNet: Evolution Channels Gradient Descent in Super Neural Networks	Jan 30, 2017	Continual Learningreinforcement-learning	CodeCode Available
Chunking: Continual Learning is not just about Distribution Shift	Oct 3, 2023	ChunkingContinual Learning	CodeCode Available
Graceful task adaptation with a bi-hemispheric RL agent	Jul 16, 2024	Continual Learning	CodeCode Available
AMP: Adaptive Masked Proxies for Few-Shot Segmentation	Oct 1, 2019	Continual LearningFew-Shot Semantic Segmentation	CodeCode Available
Instance exploitation for learning temporary concepts from sparsely labeled drifting data streams	Sep 20, 2020	Continual Learning	CodeCode Available
Shared and Private VAEs with Generative Replay for Continual Learning	May 17, 2021	Continual LearningPermuted-MNIST	CodeCode Available
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation	Jun 16, 2024	Continual LearningGSM8K	CodeCode Available

Show:10 25 50

← PrevPage 43 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified