Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 701–750 of 2644 papers

Title	Date	Tasks	Status	Hype
Cross-Domain Continual Learning via CLAMP	May 12, 2024	Continual LearningDomain Adaptation	CodeCode Available	0
Liquid Ensemble Selection for Continual Learning	May 12, 2024	Continual Learning	—Unverified	0
Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis	May 9, 2024	Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)	—Unverified	0
Compressed Latent Replays for Lightweight Continual Learning on Spiking Neural Networks	May 8, 2024	Continual Learning	CodeCode Available	0
Continual Learning in the Presence of Repetition	May 7, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Examining Changes in Internal Representations of Continual Learning Models Through Tensor Decomposition	May 6, 2024	Continual LearningTensor Decomposition	—Unverified	0
Exploring the Efficacy of Federated-Continual Learning Nodes with Attention-Based Classifier for Robust Web Phishing Detection: An Empirical Investigation	May 6, 2024	Continual LearningFederated Learning	—Unverified	0
Beyond Unimodal Learning: The Importance of Integrating Multiple Modalities for Lifelong Learning	May 4, 2024	Continual LearningLifelong learning	CodeCode Available	0
Continual Learning for Robust Gate Detection under Dynamic Lighting in Autonomous Drone Racing	May 2, 2024	Continual Learning	—Unverified	0
Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling	May 2, 2024	Continual LearningImitation Learning	—Unverified	0
Active Dendrites Enable Efficient Continual Learning in Time-To-First-Spike Neural Networks	Apr 30, 2024	Continual LearningSplit-MNIST	—Unverified	0
CLASSP: a Biologically-Inspired Approach to Continual Learning through Adjustment Suppression and Sparsity Promotion	Apr 29, 2024	Continual LearningSentiment Analysis	CodeCode Available	0
Integrating Present and Past in Unsupervised Continual Learning	Apr 29, 2024	Continual LearningDiversity	CodeCode Available	0
HFT: Half Fine-Tuning for Large Language Models	Apr 29, 2024	Continual Learning	—Unverified	0
IMEX-Reg: Implicit-Explicit Regularization in the Function Space for Continual Learning	Apr 28, 2024	Continual LearningRepresentation Learning	CodeCode Available	0
Towards Incremental Learning in Large Language Models: A Critical Review	Apr 28, 2024	Continual LearningIncremental Learning	—Unverified	0
Prompt Customization for Continual Learning	Apr 28, 2024	Continual LearningIncremental Learning	CodeCode Available	0
Parameter Efficient Fine-tuning of Self-supervised ViTs without Catastrophic Forgetting	Apr 26, 2024	Continual LearningDomain Adaptation	CodeCode Available	1
A Self-Organizing Clustering System for Unsupervised Distribution Shift Detection	Apr 25, 2024	ClusteringContinual Learning	—Unverified	0
Continual Learning of Large Language Models: A Comprehensive Survey	Apr 25, 2024	Continual LearningSurvey	CodeCode Available	4
Layer Ensemble Averaging for Improving Memristor-Based Artificial Neural Network Performance	Apr 24, 2024	Continual Learning	—Unverified	0
Revisiting Neural Networks for Continual Learning: An Architectural Perspective	Apr 23, 2024	Class Incremental LearningContinual Learning	CodeCode Available	1
Dynamically Anchored Prompting for Task-Imbalanced Continual Learning	Apr 23, 2024	Continual Learning	CodeCode Available	0
Brain-Inspired Continual Learning-Robust Feature Distillation and Re-Consolidation for Class Incremental Learning	Apr 22, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
QCore: Data-Efficient, On-Device Continual Calibration for Quantized Models -- Extended Version	Apr 22, 2024	Continual Learning	CodeCode Available	0
DynaMMo: Dynamic Model Merging for Efficient Class Incremental Learning for Medical Images	Apr 22, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
I2CANSAY:Inter-Class Analogical Augmentation and Intra-Class Significance Analysis for Non-Exemplar Online Task-Free Continual Learning	Apr 21, 2024	Continual Learningimage-classification	—Unverified	0
Double Mixture: Towards Continual Event Detection from Speech	Apr 20, 2024	Continual LearningDisentanglement	CodeCode Available	0
Unlocking Robust Segmentation Across All Age Groups via Continual Learning	Apr 19, 2024	AllAnatomy	—Unverified	0
BACS: Background Aware Continual Semantic Segmentation	Apr 19, 2024	Autonomous DrivingContinual Learning	CodeCode Available	0
Data-Incremental Continual Offline Reinforcement Learning	Apr 19, 2024	Continual LearningOffline RL	—Unverified	0
Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation	Apr 19, 2024	Continual Learning	CodeCode Available	0
FedMeS: Personalized Federated Continual Learning Leveraging Local Memory	Apr 19, 2024	AllContinual Learning	—Unverified	0
Adaptive Memory Replay for Continual Learning	Apr 18, 2024	Continual Learning	—Unverified	0
Graph Continual Learning with Debiased Lossless Memory Replay	Apr 17, 2024	Continual LearningIncremental Learning	—Unverified	0
LAECIPS: Large Vision Model Assisted Adaptive Edge-Cloud Collaboration for IoT-based Embodied Intelligence System	Apr 16, 2024	Autonomous DrivingContinual Learning	—Unverified	0
Watch Your Step: Optimal Retrieval for Continual Learning at Scale	Apr 16, 2024	Continual Learningobject-detection	—Unverified	0
Continual Offline Reinforcement Learning via Diffusion-based Dual Generative Replay	Apr 16, 2024	Continual Learningreinforcement-learning	CodeCode Available	1
Towards Practical Tool Usage for Continually Learning LLMs	Apr 14, 2024	Continual Learning	—Unverified	0
E3: Ensemble of Expert Embedders for Adapting Synthetic Image Detectors to New Generators Using Limited Data	Apr 12, 2024	Continual LearningSynthetic Image Detection	CodeCode Available	0
AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees	Apr 12, 2024	Continual Learning	—Unverified	0
CEM: A Data-Efficient Method for Large Language Models to Continue Evolving From Mistakes	Apr 11, 2024	Continual LearningContinual Pretraining	—Unverified	0
Learning to Classify New Foods Incrementally Via Compressed Exemplars	Apr 11, 2024	Continual LearningFood Recognition	—Unverified	0
Sketch-Plan-Generalize: Learning and Planning with Neuro-Symbolic Programmatic Representations for Inductive Spatial Concepts	Apr 11, 2024	Code Generationcontinual few-shot learning	—Unverified	0
Remembering Transformer for Continual Learning	Apr 11, 2024	Continual LearningKnowledge Distillation	—Unverified	0
Continual Learning of Range-Dependent Transmission Loss for Underwater Acoustic using Conditional Convolutional Neural Net	Apr 11, 2024	Continual Learning	—Unverified	0
Realistic Continual Learning Approach using Pre-trained Models	Apr 11, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Scalable Language Model with Generalized Continual Learning	Apr 11, 2024	Continual LearningLanguage Modeling	CodeCode Available	1
Calibration of Continual Learning Models	Apr 11, 2024	Continual Learning	CodeCode Available	0
Toward industrial use of continual learning : new metrics proposal for class incremental learning	Apr 10, 2024	class-incremental learningClass Incremental Learning	—Unverified	0

Show:10 25 50

← PrevPage 15 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ProgressiveNet	Accuracy	76.16	—	Unverified
2	Piggyback	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	H$^{2}$	Accuracy	75.71	—	Unverified
6	PackNet	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified