Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 351–400 of 2644 papers

Title	Date	Tasks	Status	Hype
Continual Learning of Personalized Generative Face Models with Experience Replay	Dec 3, 2024	Continual LearningFace Model	—Unverified	0
FCL-ViT: Task-Aware Attention Tuning for Continual Learning	Dec 3, 2024	Continual Learning	—Unverified	0
Optimizing Latent Goal by Learning from Trajectory Preference	Dec 3, 2024	Continual LearningInstruction Following	—Unverified	0
Sustainable Self-evolution Adversarial Training	Dec 3, 2024	Adversarial DefenseContinual Learning	—Unverified	0
Learning Mamba as a Continual Learner	Dec 1, 2024	Continual LearningMamba	—Unverified	0
Adaptive Rank, Reduced Forgetting: Knowledge Retention in Continual Learning Vision-Language Models with Dynamic Rank-Selective LoRA	Dec 1, 2024	Continual Learning	—Unverified	0
Deep evolving semi-supervised anomaly detection	Dec 1, 2024	Anomaly DetectionContinual Learning	—Unverified	0
Incremental Multi-Scene Modeling via Continual Neural Graphics Primitives	Nov 29, 2024	Continual LearningNeRF	—Unverified	0
Federated Continual Graph Learning	Nov 28, 2024	Continual LearningGraph Learning	CodeCode Available	0
DESIRE: Dynamic Knowledge Consolidation for Rehearsal-Free Continual Learning	Nov 28, 2024	Continual Learningparameter-efficient fine-tuning	—Unverified	0
Investigating Plausibility of Biologically Inspired Bayesian Learning in ANNs	Nov 27, 2024	Bayesian InferenceContinual Learning	—Unverified	0
Continual Learning in Machine Speech Chain Using Gradient Episodic Memory	Nov 27, 2024	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified	0
DECODE: Domain-aware Continual Domain Expansion for Motion Prediction	Nov 26, 2024	Autonomous DrivingAutonomous Vehicles	CodeCode Available	0
Modality-Incremental Learning with Disjoint Relevance Mapping Networks for Image-based Semantic Segmentation	Nov 26, 2024	Autonomous DrivingContinual Learning	—Unverified	0
COBRA: A Continual Learning Approach to Vision-Brain Understanding	Nov 25, 2024	Continual Learning	—Unverified	0
Learning New Concepts, Remembering the Old: A Novel Continual Learning	Nov 25, 2024	Continual LearningIncremental Learning	—Unverified	0
Continual Deep Reinforcement Learning with Task-Agnostic Policy Distillation	Nov 25, 2024	Continual LearningDeep Reinforcement Learning	CodeCode Available	0
Soft-TransFormers for Continual Learning	Nov 25, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
Distribution-aware Online Continual Learning for Urban Spatio-Temporal Forecasting	Nov 24, 2024	Continual LearningScheduling	—Unverified	0
Mamba-CL: Optimizing Selective State Space Model in Null Space for Continual Learning	Nov 23, 2024	Continual LearningMamba	—Unverified	0
CODE-CL: Conceptor-Based Gradient Projection for Deep Continual Learning	Nov 21, 2024	Continual LearningImage Classification	CodeCode Available	1
Separable Mixture of Low-Rank Adaptation for Continual Visual Instruction Tuning	Nov 21, 2024	Continual LearningInstruction Following	—Unverified	0
Dealing with Synthetic Data Contamination in Online Continual Learning	Nov 21, 2024	Continual LearningImage Generation	CodeCode Available	0
CLFace: A Scalable and Resource-Efficient Continual Learning Framework for Lifelong Face Recognition	Nov 21, 2024	Continual LearningFace Recognition	—Unverified	0
Revised Regularization for Efficient Continual Learning through Correlation-Based Parameter Update in Bayesian Neural Networks	Nov 21, 2024	Continual LearningTransfer Learning	—Unverified	0
Leveraging Prior Experience: An Expandable Auxiliary Knowledge Base for Text-to-SQL	Nov 20, 2024	Continual LearningIn-Context Learning	CodeCode Available	0
Federated Continual Learning for Edge-AI: A Comprehensive Survey	Nov 20, 2024	Continual LearningEdge-computing	—Unverified	0
Class-Incremental Learning for Sound Event Localization and Detection	Nov 19, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
What Makes a Good Dataset for Knowledge Distillation?	Nov 19, 2024	Continual LearningKnowledge Distillation	—Unverified	0
C^2INet: Realizing Incremental Trajectory Prediction with Prior-Aware Continual Causal Intervention	Nov 19, 2024	Autonomous DrivingContinual Learning	—Unverified	0
Reviving Dormant Memories: Investigating Catastrophic Forgetting in Language Models through Rationale-Guidance Difficulty	Nov 18, 2024	Continual Learning	—Unverified	0
Continual Task Learning through Adaptive Policy Self-Composition	Nov 18, 2024	Continual LearningOffline RL	CodeCode Available	0
FedCL-Ensemble Learning: A Framework of Federated Continual Learning with Ensemble Transfer Learning Enhanced for Alzheimer's MRI Classifications while Preserving Privacy	Nov 15, 2024	ClassificationContinual Learning	—Unverified	0
Continual Adversarial Reinforcement Learning (CARL) of False Data Injection detection: forgetting and explainability	Nov 15, 2024	Continual LearningReinforcement Learning (RL)	—Unverified	0
A unifying framework for generalised Bayesian online learning in non-stationary environments	Nov 15, 2024	Continual LearningMulti-Armed Bandits	CodeCode Available	1
FedRewind: Rewinding Continual Model Exchange for Decentralized Federated Learning	Nov 14, 2024	Continual LearningFederated Learning	—Unverified	0
Joint Diffusion models in Continual Learning	Nov 12, 2024	Continual LearningKnowledge Distillation	—Unverified	0
ODEStream: A Buffer-Free Online Learning Framework with ODE-based Adaptor for Streaming Time Series Forecasting	Nov 11, 2024	Continual LearningTime Series	—Unverified	0
Slowing Down Forgetting in Continual Learning	Nov 11, 2024	Continual LearningIncremental Learning	—Unverified	0
Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning	Nov 11, 2024	Continual Learning	—Unverified	0
SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models	Nov 9, 2024	Continual Learning	CodeCode Available	0
Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptation	Nov 8, 2024	Continual Learning	CodeCode Available	2
Gradient Localization Improves Lifelong Pretraining of Language Models	Nov 7, 2024	Continual LearningWorld Knowledge	—Unverified	0
Differential Privacy in Continual Learning: Which Labels to Update?	Nov 7, 2024	Continual Learningvalid	—Unverified	0
Exploring the Stability Gap in Continual Learning: The Role of the Classification Head	Nov 6, 2024	AttributeContinual Learning	CodeCode Available	0
Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset	Nov 6, 2024	Continual LearningMulti-Armed Bandits	—Unverified	0
Continual Audio-Visual Sound Separation	Nov 5, 2024	Continual LearningSemantic Similarity	CodeCode Available	0
Out-of-Distribution Recovery with Object-Centric Keypoint Inverse Policy for Visuomotor Imitation Learning	Nov 5, 2024	Continual LearningImitation Learning	—Unverified	0
Sparse Orthogonal Parameters Tuning for Continual Learning	Nov 5, 2024	Continual Learning	—Unverified	0
Vocal Sandbox: Continual Learning and Adaptation for Situated Human-Robot Collaboration	Nov 4, 2024	Continual Learning	—Unverified	0

Show:10 25 50

← PrevPage 8 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified