Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 551–600 of 2644 papers

Title	Date	Tasks	Status	Hype
PromptCCD: Learning Gaussian Mixture Prompt Pool for Continual Category Discovery	Jul 26, 2024	Continual LearningRepresentation Learning	CodeCode Available	0
Continual Learning with Hebbian Plasticity in Sparse and Predictive Coding Networks: A Survey and Perspective	Jul 24, 2024	Continual LearningSurvey	—Unverified	0
Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective	Jul 24, 2024	Continual Learningparameter-efficient fine-tuning	—Unverified	0
COALA: A Practical and Vision-Centric Federated Learning Platform	Jul 23, 2024	BenchmarkingContinual Learning	CodeCode Available	2
Local vs Global continual learning	Jul 23, 2024	Continual Learning	—Unverified	0
KAN or MLP: A Fairer Comparison	Jul 23, 2024	Continual Learning	CodeCode Available	2
CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learning	Jul 22, 2024	Continual LearningIncremental Learning	CodeCode Available	1
CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning	Jul 22, 2024	Class Incremental LearningContinual Learning	—Unverified	0
A Life-long Learning Intrusion Detection System for 6G-Enabled IoV	Jul 22, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Enhancing Hardware Fault Tolerance in Machines with Reinforcement Learning Policy Gradient Algorithms	Jul 21, 2024	Continual LearningOpenAI Gym	—Unverified	0
Flatness-aware Sequential Learning Generates Resilient Backdoors	Jul 20, 2024	Continual Learning	CodeCode Available	0
An Attention-based Representation Distillation Baseline for Multi-Label Continual Learning	Jul 19, 2024	Continual LearningTransfer Learning	CodeCode Available	0
Continual Learning for Remote Physiological Measurement: Minimize Forgetting and Simplify Inference	Jul 19, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
Continual Panoptic Perception: Towards Multi-modal Incremental Interpretation of Remote Sensing Images	Jul 19, 2024	Caption GenerationContinual Learning	CodeCode Available	0
Continual Distillation Learning: Knowledge Distillation in Prompt-based Continual Learning	Jul 18, 2024	Continual LearningKnowledge Distillation	—Unverified	0
Learn to Memorize and to Forget: A Continual Learning Perspective of Dynamic SLAM	Jul 18, 2024	Continual LearningSimultaneous Localization and Mapping	—Unverified	0
Continual Learning for Adaptable Car-Following in Dynamic Traffic Environments	Jul 17, 2024	Autonomous DrivingContinual Learning	—Unverified	0
FETCH: A Memory-Efficient Replay Approach for Continual Learning in Image Classification	Jul 17, 2024	BenchmarkingContinual Learning	—Unverified	0
ER-FSL: Experience Replay with Feature Subspace Learning for Online Continual Learning	Jul 17, 2024	Continual Learning	—Unverified	0
An Evaluation of Continual Learning for Advanced Node Semiconductor Defect Inspection	Jul 17, 2024	Continual LearningMeta-Learning	—Unverified	0
Continual Learning for Temporal-Sensitive Question Answering	Jul 17, 2024	Continual LearningContrastive Learning	—Unverified	0
CroMo-Mixup: Augmenting Cross-Model Representations for Continual Self-Supervised Learning	Jul 16, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
Mitigating Background Shift in Class-Incremental Semantic Segmentation	Jul 16, 2024	Class Incremental LearningClass-Incremental Semantic Segmentation	CodeCode Available	1
Graceful task adaptation with a bi-hemispheric RL agent	Jul 16, 2024	Continual Learning	CodeCode Available	0
DDFAD: Dataset Distillation Framework for Audio Data	Jul 15, 2024	Continual LearningDataset Distillation	—Unverified	0
Efficient Continual Learning with Low Memory Footprint For Edge Device	Jul 15, 2024	Continual LearningRecommendation Systems	—Unverified	0
Continual Deep Learning on the Edge via Stochastic Local Competition among Subnetworks	Jul 15, 2024	Continual Learning	—Unverified	0
Advancing Continual Learning for Robust Deepfake Audio Classification	Jul 14, 2024	Audio ClassificationClassification	—Unverified	0
Order parameters and phase transitions of continual learning in deep neural networks	Jul 14, 2024	Continual LearningTransfer Learning	—Unverified	0
Beyond Prompt Learning: Continual Adapter for Efficient Rehearsal-Free Continual Learning	Jul 14, 2024	Continual LearningIncremental Learning	—Unverified	0
Overcoming Catastrophic Forgetting in Tabular Data Classification: A Pseudorehearsal-based approach	Jul 12, 2024	Continual LearningLifelong learning	—Unverified	0
iNeMo: Incremental Neural Mesh Models for Robust Class-Incremental Learning	Jul 12, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
One Stone, Four Birds: A Comprehensive Solution for QA System Using Supervised Contrastive Learning	Jul 12, 2024	Continual LearningContrastive Learning	CodeCode Available	0
Continually Learn to Map Visual Concepts to Large Language Models in Resource-constrained Environments	Jul 11, 2024	Continual LearningLanguage Modeling	—Unverified	0
CLEO: Continual Learning of Evolving Ontologies	Jul 11, 2024	Autonomous DrivingContinual Learning	—Unverified	0
Towards stable training of parallel continual learning	Jul 11, 2024	Autonomous VehiclesContinual Learning	CodeCode Available	0
BiasPruner: Debiased Continual Learning for Medical Image Classification	Jul 11, 2024	ClassificationContinual Learning	CodeCode Available	1
Exemplar-free Continual Representation Learning via Learnable Drift Compensation	Jul 11, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	1
Rethinking Few-shot Class-incremental Learning: Learning from Yourself	Jul 10, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	1
How to Leverage Predictive Uncertainty Estimates for Reducing Catastrophic Forgetting in Online Continual Learning	Jul 10, 2024	Continual Learning	—Unverified	0
Neuromimetic metaplasticity for adaptive continual learning	Jul 9, 2024	Continual LearningData Poisoning	—Unverified	0
Certified Continual Learning for Neural Network Regression	Jul 9, 2024	Continual Learningregression	—Unverified	0
MagMax: Leveraging Model Merging for Seamless Continual Learning	Jul 8, 2024	Continual LearningIncremental Learning	CodeCode Available	1
HiDe-PET: Continual Learning via Hierarchical Decomposition of Parameter-Efficient Tuning	Jul 7, 2024	Continual LearningRepresentation Learning	CodeCode Available	2
Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Models	Jul 7, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	2
The Solution for the sequential task continual learning track of the 2nd Greater Bay Area International Algorithm Competition	Jul 6, 2024	Continual Learning	—Unverified	0
Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling	Jul 5, 2024	Continual LearningDiversity	—Unverified	0
Where's That Voice Coming? Continual Learning for Sound Source Localization	Jul 4, 2024	Continual LearningExemplar-Free	—Unverified	0
UCIL: An Unsupervised Class Incremental Learning Approach for Sound Event Detection	Jul 4, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Continual Learning Optimizations for Auto-regressive Decoder of Multilingual ASR systems	Jul 4, 2024	Continual LearningDecoder	—Unverified	0

Show:10 25 50

← PrevPage 12 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified