Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 501–550 of 2644 papers

Title	Date	Tasks	Status	Hype
Simplex-enabled Safe Continual Learning Machine	Sep 5, 2024	Continual LearningDeep Reinforcement Learning	—Unverified	0
Continual Skill and Task Learning via Dialogue	Sep 5, 2024	Continual LearningInstruction Following	—Unverified	0
Few-Shot Continual Learning for Activity Recognition in Classroom Surveillance Images	Sep 5, 2024	Activity RecognitionContinual Learning	—Unverified	0
Cog-GA: A Large Language Models-based Generative Agent for Vision-Language Navigation in Continuous Environments	Sep 4, 2024	Continual LearningNavigate	—Unverified	0
Vision-Language Navigation with Continual Learning	Sep 4, 2024	Continual LearningNavigate	—Unverified	0
Latent Distillation for Continual Object Detection at the Edge	Sep 3, 2024	Class-Incremental Object DetectionContinual Learning	CodeCode Available	1
Towards General Industrial Intelligence: A Survey of Continual Large Models in Industrial IoT	Sep 2, 2024	Cloud ComputingContinual Learning	—Unverified	0
Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning	Sep 2, 2024	Continual LearningContrastive Learning	CodeCode Available	1
Novel-WD: Exploring acquisition of Novel World Knowledge in LLMs Using Prefix-Tuning	Aug 30, 2024	Causal Language ModelingContinual Learning	—Unverified	0
Continual learning with the neural tangent ensemble	Aug 30, 2024	Continual Learningvalid	—Unverified	0
Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning	Aug 29, 2024	Continual LearningMulti-Task Learning	CodeCode Available	0
Maelstrom Networks	Aug 29, 2024	Continual LearningInductive Bias	—Unverified	0
Adaptive Variational Continual Learning via Task-Heuristic Modelling	Aug 29, 2024	Continual LearningModel Optimization	—Unverified	0
LM-PUB-QUIZ: A Comprehensive Framework for Zero-Shot Evaluation of Relational Knowledge in Language Models	Aug 28, 2024	Continual LearningKnowledge Probing	—Unverified	0
Continual-learning-based framework for structural damage recognition	Aug 28, 2024	Continual Learning	—Unverified	0
Prior-free Balanced Replay: Uncertainty-guided Reservoir Sampling for Long-Tailed Continual Learning	Aug 27, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation	Aug 26, 2024	Continual Learning	CodeCode Available	2
Dual-CBA: Improving Online Continual Learning via Dual Continual Bias Adaptors from a Bi-level Optimization Perspective	Aug 26, 2024	Continual Learning	—Unverified	0
May the Forgetting Be with You: Alternate Replay for Learning with Noisy Labels	Aug 26, 2024	Continual LearningLearning with noisy labels	—Unverified	0
A Practitioner's Guide to Continual Multimodal Pretraining	Aug 26, 2024	Continual LearningContinual Pretraining	CodeCode Available	2
PHEVA: A Privacy-preserving Human-centric Video Anomaly Detection Dataset	Aug 26, 2024	Anomaly DetectionContinual Learning	CodeCode Available	1
CF-KAN: Kolmogorov-Arnold Network-based Collaborative Filtering to Mitigate Catastrophic Forgetting in Recommender Systems	Aug 25, 2024	Collaborative FilteringContinual Learning	CodeCode Available	1
Online Continuous Generalized Category Discovery	Aug 24, 2024	Continual Learning	CodeCode Available	1
A Unified Framework for Continual Learning and Unlearning	Aug 21, 2024	Continual LearningKnowledge Distillation	—Unverified	0
CluMo: Cluster-based Modality Fusion Prompt for Continual Learning in Visual Question Answering	Aug 21, 2024	Continual LearningQuestion Answering	CodeCode Available	0
Overcoming Growth-Induced Forgetting in Task-Agnostic Continual Learning	Aug 20, 2024	Continual Learning	—Unverified	0
AIR: Analytic Imbalance Rectifier for Continual Learning	Aug 19, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	1
Continual Dialogue State Tracking via Reason-of-Select Distillation	Aug 19, 2024	Continual LearningDialogue State Tracking	CodeCode Available	0
Mitigating the Stability-Plasticity Dilemma in Adaptive Train Scheduling with Curriculum-Driven Continual DQN Expansion	Aug 19, 2024	Continual LearningScheduling	—Unverified	0
Boosting Open-Domain Continual Learning via Leveraging Intra-domain Category-aware Prototype	Aug 19, 2024	Continual Learning	—Unverified	0
E-CGL: An Efficient Continual Graph Learner	Aug 18, 2024	Continual LearningGraph Learning	CodeCode Available	0
TACOS: Task Agnostic Continual Learning in Spiking Neural Networks	Aug 16, 2024	Continual LearningIncremental Learning	—Unverified	0
Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models	Aug 16, 2024	Continual Learningparameter-efficient fine-tuning	—Unverified	0
SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training	Aug 15, 2024	Continual Learningimage-classification	CodeCode Available	2
Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities	Aug 14, 2024	Continual LearningFew-Shot Learning	CodeCode Available	4
Adjusting Model Size in Continual Gaussian Processes: How Big is Big Enough?	Aug 14, 2024	Continual LearningGaussian Processes	CodeCode Available	0
Towards Effective Authorship Attribution: Integrating Class-Incremental Learning	Aug 12, 2024	Authorship Attributionclass-incremental learning	CodeCode Available	0
Continual Learning of Nonlinear Independent Representations	Aug 11, 2024	Continual LearningLifelong learning	—Unverified	0
SRTFD: Scalable Real-Time Fault Diagnosis through Online Continual Learning	Aug 11, 2024	Continual LearningFault Diagnosis	—Unverified	0
KIF: Knowledge Identification and Fusion for Language Model Continual Learning	Aug 9, 2024	Continual LearningLanguage Modeling	CodeCode Available	1
The Need for a Big World Simulator: A Scientific Challenge for Continual Learning	Aug 6, 2024	Continual Learning	—Unverified	0
Learning to Learn without Forgetting using Attention	Aug 6, 2024	Continual LearningMeta-Learning	CodeCode Available	0
Distribution-Level Memory Recall for Continual Learning: Preserving Knowledge and Avoiding Confusion	Aug 4, 2024	Continual Learning	—Unverified	0
Exploiting the Semantic Knowledge of Pre-trained Text-Encoders for Continual Learning	Aug 2, 2024	Continual LearningKnowledge Distillation	CodeCode Available	0
PMoE: Progressive Mixture of Experts with Asymmetric Transformer for Continual Learning	Jul 31, 2024	Continual LearningGeneral Knowledge	—Unverified	0
VIPeR: Visual Incremental Place Recognition with Adaptive Mining and Continual Learning	Jul 31, 2024	Continual LearningKnowledge Distillation	—Unverified	0
Distribution-Aware Replay for Continual MRI Segmentation	Jul 30, 2024	Continual LearningHippocampus	—Unverified	0
PIP: Prototypes-Injected Prompt for Federated Class Incremental Learning	Jul 30, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
An Effective Dynamic Gradient Calibration Method for Continual Learning	Jul 30, 2024	Continual Learning	—Unverified	0
Constructing Enhanced Mutual Information for Online Class-Incremental Learning	Jul 26, 2024	class-incremental learningClass Incremental Learning	—Unverified	0

Show:10 25 50

← PrevPage 11 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified