Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1101–1150 of 2644 papers

Title	Date	Tasks	Status	Hype
DFIL: Deepfake Incremental Learning by Exploiting Domain-invariant Forgery Clues	Sep 18, 2023	Continual LearningContrastive Learning	CodeCode Available	1
A Continual Learning Paradigm for Non-differentiable Visual Programming Frameworks on Visual Reasoning Tasks	Sep 18, 2023	Continual LearningVisual Reasoning	—Unverified	0
CaT: Balanced Continual Graph Learning with Graph Condensation	Sep 18, 2023	Continual LearningGraph Learning	CodeCode Available	1
Adaptive Reorganization of Neural Pathways for Continual Learning with Spiking Neural Networks	Sep 18, 2023	Continual Learning	—Unverified	0
SHAPNN: Shapley Value Regularized Tabular Neural Network	Sep 15, 2023	Continual Learningvalid	—Unverified	0
EgoObjects: A Large-Scale Egocentric Dataset for Fine-Grained Object Understanding	Sep 15, 2023	Continual LearningObject	CodeCode Available	1
Continual Learning with Deep Streaming Regularized Discriminant Analysis	Sep 15, 2023	Continual LearningImage Classification	CodeCode Available	0
VERSE: Virtual-Gradient Aware Streaming Lifelong Learning with Anytime Inference	Sep 15, 2023	AI AgentContinual Learning	—Unverified	0
FedJudge: Federated Legal Large Language Model	Sep 15, 2023	Continual LearningFederated Learning	CodeCode Available	1
Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization	Sep 15, 2023	Continual LearningSplit-MNIST	—Unverified	0
Enhancing Performance, Calibration Time and Efficiency in Brain-Machine Interfaces through Transfer Learning and Wearable EEG Technology	Sep 14, 2023	Continual LearningEEG	—Unverified	0
Semantic Parsing in Limited Resource Conditions	Sep 14, 2023	Active LearningContinual Learning	—Unverified	0
Domain-Aware Augmentations for Unsupervised Online General Continual Learning	Sep 13, 2023	Continual LearningContrastive Learning	—Unverified	0
Continual Learning with Dirichlet Generative-based Rehearsal	Sep 13, 2023	Continual LearningIncremental Learning	—Unverified	0
PILOT: A Pre-Trained Model-Based Continual Learning Toolbox	Sep 13, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	2
Plasticity-Optimized Complementary Networks for Unsupervised Continual Learning	Sep 12, 2023	Continual LearningExemplar-Free	CodeCode Available	0
Mitigating the Alignment Tax of RLHF	Sep 12, 2023	Common Sense ReasoningContinual Learning	CodeCode Available	1
Class-Incremental Grouping Network for Continual Audio-Visual Learning	Sep 11, 2023	audio-visual learningclass-incremental learning	CodeCode Available	1
Continual Robot Learning using Self-Supervised Task Inference	Sep 10, 2023	Continual LearningMulti-Task Learning	—Unverified	0
UER: A Heuristic Bias Addressing Approach for Online Continual Learning	Sep 8, 2023	Continual Learning	CodeCode Available	0
Navigating Out-of-Distribution Electricity Load Forecasting during COVID-19: Benchmarking energy load forecasting models without and with continual learning	Sep 8, 2023	BenchmarkingContinual Learning	CodeCode Available	0
A Natural Gas Consumption Forecasting System for Continual Learning Scenarios based on Hoeffding Trees with Change Point Detection Mechanism	Sep 7, 2023	Change Point DetectionContinual Learning	CodeCode Available	0
Privacy-preserving Continual Federated Clustering via Adaptive Resonance Theory	Sep 7, 2023	ClusteringContinual Learning	CodeCode Available	0
Continual Evidential Deep Learning for Out-of-Distribution Detection	Sep 6, 2023	Continual LearningDeep Learning	CodeCode Available	1
Rethinking Momentum Knowledge Distillation in Online Continual Learning	Sep 6, 2023	Continual LearningKnowledge Distillation	CodeCode Available	1
Continual Cross-Dataset Adaptation in Road Surface Classification	Sep 5, 2023	Autonomous VehiclesClassification	—Unverified	0
On the Query Strategies for Efficient Online Active Distillation	Sep 4, 2023	Active LearningContinual Learning	—Unverified	0
Instant Continual Learning of Neural Radiance Fields	Sep 4, 2023	3D Scene ReconstructionContinual Learning	—Unverified	0
Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning	Sep 3, 2023	Continual LearningFederated Learning	—Unverified	0
New metrics for analyzing continual learners	Sep 1, 2023	Continual Learning	—Unverified	0
Continual Learning From a Stream of APIs	Aug 31, 2023	Continual Learning	—Unverified	0
ScrollNet: Dynamic Weight Importance for Continual Learning	Aug 31, 2023	Continual Learning	CodeCode Available	1
Introducing Language Guidance in Prompt-based Continual Learning	Aug 30, 2023	Continual Learning	CodeCode Available	0
Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence	Aug 29, 2023	Continual Learning	CodeCode Available	1
Continual Learning for Generative Retrieval over Dynamic Corpora	Aug 29, 2023	Continual LearningQuantization	CodeCode Available	0
On-Device Learning with Binary Neural Networks	Aug 29, 2023	Continual LearningQuantization	—Unverified	0
CLNeRF: Continual Learning Meets NeRF	Aug 28, 2023	Continual LearningNeRF	CodeCode Available	0
Continual Learning with Dynamic Sparse Training: Exploring Algorithms for Effective Model Updates	Aug 28, 2023	Continual Learning	CodeCode Available	0
Online Continual Learning on Hierarchical Label Expansion	Aug 28, 2023	Continual Learning	—Unverified	0
Universal Graph Continual Learning	Aug 27, 2023	Continual LearningGraph Classification	—Unverified	0
Rethinking Exemplars for Continual Semantic Segmentation in Endoscopy Scenes: Entropy-based Mini-Batch Pseudo-Replay	Aug 27, 2023	Continual LearningContinual Semantic Segmentation	—Unverified	0
GRASP: A Rehearsal Policy for Efficient Online Continual Learning	Aug 25, 2023	class-incremental learningClass Incremental Learning	—Unverified	0
ConSlide: Asynchronous Hierarchical Interaction Transformer with Breakup-Reorganize Rehearsal for Continual Whole Slide Image Analysis	Aug 25, 2023	Continual LearningPrognosis	CodeCode Available	1
Fine-tuning can cripple your foundation model; preserving features may be the solution	Aug 25, 2023	Continual Learning	CodeCode Available	0
FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning	Aug 24, 2023	Continual LearningFederated Learning	CodeCode Available	1
A Continual Learning Approach for Cross-Domain White Blood Cell Classification	Aug 24, 2023	ClassificationContinual Learning	—Unverified	0
Overcoming Generic Knowledge Loss with Selective Parameter Update	Aug 23, 2023	Continual LearningGeneral Knowledge	CodeCode Available	1
Category Adaptation Meets Projected Distillation in Generalized Continual Category Discovery	Aug 23, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	0
An Intentional Forgetting-Driven Self-Healing Method For Deep Reinforcement Learning Systems	Aug 23, 2023	Continual LearningDeep Reinforcement Learning	CodeCode Available	0
Maintaining Plasticity in Continual Learning via Regenerative Regularization	Aug 23, 2023	Continual LearningL2 Regularization	—Unverified	0

Show:10 25 50

← PrevPage 23 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified