Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 651–700 of 2644 papers

Title	Date	Tasks	Status	Hype
Continual Traffic Forecasting via Mixture of Experts	Jun 5, 2024	Continual LearningMixture-of-Experts	—Unverified	0
Unsupervised Few-Shot Continual Learning for Remote Sensing Image Scene Classification	Jun 4, 2024	Continual LearningScene Classification	CodeCode Available	0
Parameterizing Federated Continual Learning for Reproducible Research	Jun 4, 2024	Continual LearningFederated Learning	—Unverified	0
Federated Class-Incremental Learning with Hierarchical Generative Prototypes	Jun 4, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Confidence-Based Task Prediction in Continual Disease Classification Using Probability Distribution	Jun 3, 2024	Continual Learning	—Unverified	0
iKAN: Global Incremental Learning with KAN for Human Activity Recognition Across Heterogeneous Datasets	Jun 3, 2024	Activity RecognitionContinual Learning	—Unverified	0
An Analysis under a Unified Fomulation of Learning Algorithms with Output Constraints	Jun 3, 2024	Continual LearningNatural Language Inference	—Unverified	0
AGALE: A Graph-Aware Continual Learning Evaluation Framework	Jun 3, 2024	Continual LearningGraph Learning	CodeCode Available	0
Disentangling and Mitigating the Impact of Task Similarity for Continual Learning	May 30, 2024	Continual LearningPermuted-MNIST	CodeCode Available	0
Forward-Backward Knowledge Distillation for Continual Clustering	May 29, 2024	ClusteringContinual Learning	—Unverified	0
Federated Continual Learning Goes Online: Uncertainty-Aware Memory Management for Vision Tasks and Beyond	May 29, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Provable Contrastive Continual Learning	May 29, 2024	Continual Learning	—Unverified	0
Learning to Continually Learn with the Bayesian Principle	May 29, 2024	Continual LearningMeta-Learning	CodeCode Available	1
A Study of Plasticity Loss in On-Policy Deep Reinforcement Learning	May 29, 2024	Continual LearningDeep Reinforcement Learning	CodeCode Available	0
Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning	May 29, 2024	Continual LearningExemplar-Free	CodeCode Available	1
Recent Advances of Foundation Language Models-based Continual Learning: A Survey	May 28, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task	May 28, 2024	Autonomous DrivingContinual Learning	—Unverified	0
An Empirical Analysis of Forgetting in Pre-trained Models with Incremental Low-Rank Updates	May 28, 2024	Continual Learning	CodeCode Available	0
Understanding Forgetting in Continual Learning with Linear Regression	May 27, 2024	Continual Learningregression	—Unverified	0
Perturbation-Restrained Sequential Model Editing	May 27, 2024	Continual Learningmodel	CodeCode Available	1
Improving Data-aware and Parameter-aware Robustness for Continual Learning	May 27, 2024	Continual Learning	CodeCode Available	0
Theories of synaptic memory consolidation and intelligent plasticity for continual learning	May 27, 2024	Continual Learning	—Unverified	0
On Sequential Loss Approximation for Continual Learning	May 26, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
Few-shot Tuning of Foundation Models for Class-incremental Learning	May 26, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
Exploring the Evolution of Hidden Activations with Live-Update Visualization	May 24, 2024	Continual Learning	CodeCode Available	0
HINT: Hypernetwork Approach to Training Weight Interval Regions in Continual Learning	May 24, 2024	Continual Learning	CodeCode Available	0
Distributed Continual Learning	May 23, 2024	Continual LearningFederated Learning	—Unverified	0
ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks	May 23, 2024	Continual LearningMulti Label Text Classification	—Unverified	0
Mixture of Experts Meets Prompt-Based Continual Learning	May 23, 2024	Continual LearningMixture-of-Experts	CodeCode Available	1
Variational Bayes for Federated Continual Learning	May 23, 2024	Continual LearningFederated Learning	CodeCode Available	0
Adaptive Retention & Correction: Test-Time Training for Continual Learning	May 23, 2024	ARCContinual Learning	—Unverified	0
U-TELL: Unsupervised Task Expert Lifelong Learning	May 23, 2024	ClusteringContinual Learning	CodeCode Available	0
PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis	May 23, 2024	Continual LearningContrastive Learning	—Unverified	0
Task agnostic continual learning with Pairwise layer architecture	May 22, 2024	Continual Learningimage-classification	CodeCode Available	0
Gradient Projection For Continual Parameter-Efficient Tuning	May 22, 2024	Continual LearningHallucination	—Unverified	0
Continual Learning in Medical Imaging: A Survey and Practical Analysis	May 22, 2024	Continual LearningSurvey	CodeCode Available	1
Mitigating Interference in the Knowledge Continuum through Attention-Guided Incremental Learning	May 22, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
Continual Deep Reinforcement Learning for Decentralized Satellite Routing	May 20, 2024	Continual LearningDeep Reinforcement Learning	—Unverified	0
Adversarially Diversified Rehearsal Memory (ADRM): Mitigating Memory Overfitting Challenge in Continual Learning	May 20, 2024	Continual LearningDiversity	CodeCode Available	0
FeTT: Continual Class Incremental Learning via Feature Transformation Tuning	May 20, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Preparing for Black Swans: The Antifragility Imperative for Machine Learning	May 18, 2024	Continual LearningDecision Making	—Unverified	0
Bayesian Learning-driven Prototypical Contrastive Loss for Class-Incremental Learning	May 17, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System	May 16, 2024	Continual LearningTask-Oriented Dialogue Systems	—Unverified	0
Beyond Traditional Single Object Tracking: A Survey	May 16, 2024	Continual LearningDomain Adaptation	—Unverified	0
Densely Distilling Cumulative Knowledge for Continual Learning	May 16, 2024	AllContinual Learning	—Unverified	0
SPD-CFL: Stepwise Parameter Dropout for Efficient Continual Federated Learning	May 15, 2024	Continual LearningFederated Learning	—Unverified	0
MGSER-SAM: Memory-Guided Soft Experience Replay with Sharpness-Aware Optimization for Enhanced Continual Learning	May 15, 2024	Continual Learning	—Unverified	0
Overcoming Domain Drift in Online Continual Learning	May 15, 2024	Continual Learning	—Unverified	0
Wearable Sensor-Based Few-Shot Continual Learning on Hand Gestures for Motor-Impaired Individuals via Latent Embedding Exploitation	May 14, 2024	Continual LearningGesture Recognition	CodeCode Available	0
Liquid Ensemble Selection for Continual Learning	May 12, 2024	Continual Learning	—Unverified	0

Show:10 25 50

← PrevPage 14 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ProgressiveNet	Accuracy	76.16	—	Unverified
2	Piggyback	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	H$^{2}$	Accuracy	75.71	—	Unverified
6	PackNet	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified