Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 401–450 of 2644 papers

Title	Date	Tasks	Status	Hype
SAFE: Slow and Fast Parameter-Efficient Tuning for Continual Learning with Pre-Trained Models	Nov 4, 2024	Continual LearningGeneral Knowledge	CodeCode Available	1
FPPL: An Efficient and Non-IID Robust Federated Continual Learning Framework	Nov 4, 2024	Continual LearningContrastive Learning	CodeCode Available	0
One VLM to Keep it Learning: Generation and Balancing for Data-free Continual Visual Question Answering	Nov 4, 2024	Continual LearningQuestion Answering	—Unverified	0
Masked Autoencoders are Parameter-Efficient Federated Continual Learners	Nov 4, 2024	Continual LearningFederated Learning	CodeCode Available	0
Dual Low-Rank Adaptation for Continual Learning with Pre-Trained Models	Nov 1, 2024	Continual Learningparameter-efficient fine-tuning	—Unverified	0
EXACFS -- A CIL Method to mitigate Catastrophic Forgetting	Oct 31, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
EF-LLM: Energy Forecasting LLM with AI-assisted Automation, Enhanced Sparse Prediction, Hallucination Detection	Oct 30, 2024	Continual LearningHallucination	—Unverified	0
NCAdapt: Dynamic adaptation with domain-specific Neural Cellular Automata for continual hippocampus segmentation	Oct 30, 2024	BenchmarkingContinual Learning	CodeCode Available	0
Learning Successor Features the Simple Way	Oct 29, 2024	Continual LearningDeep Reinforcement Learning	CodeCode Available	1
Adapting Diffusion Models for Improved Prompt Compliance and Controllable Image Synthesis	Oct 29, 2024	Continual LearningImage Generation	CodeCode Available	0
Similarity-based context aware continual learning for spiking neural networks	Oct 28, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
ImageNet-RIB Benchmark: Large Pre-Training Datasets Don't Guarantee Robustness after Fine-Tuning	Oct 28, 2024	Continual Learning	—Unverified	0
Plastic Learning with Deep Fourier Features	Oct 27, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Vector Quantization Prompting for Continual Learning	Oct 27, 2024	Continual LearningQuantization	CodeCode Available	1
Self-Normalized Resets for Plasticity in Continual Learning	Oct 26, 2024	Continual Learning	—Unverified	0
LLMs Can Evolve Continually on Modality for X-Modal Reasoning	Oct 26, 2024	Continual Learningmultimodal interaction	CodeCode Available	1
Improving Multimodal Large Language Models Using Continual Learning	Oct 25, 2024	Continual LearningNatural Language Understanding	—Unverified	0
Prompting Continual Person Search	Oct 25, 2024	AttributeContinual Learning	CodeCode Available	0
Delta: A Cloud-assisted Data Enrichment Framework for On-Device Continual Learning	Oct 24, 2024	Continual Learning	—Unverified	0
TEAM: Topological Evolution-aware Framework for Traffic Forecasting--Extended Version	Oct 24, 2024	Continual LearningTime Series	CodeCode Available	1
Closed-form merging of parameter-efficient modules for Federated Continual Learning	Oct 23, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
UnCLe: Unsupervised Continual Learning of Depth Completion	Oct 23, 2024	Continual LearningDepth Completion	—Unverified	0
Continual Learning on a Data Diet	Oct 23, 2024	Continual Learning	CodeCode Available	0
Unsupervised Replay Strategies for Continual Learning with Limited Data	Oct 21, 2024	Continual Learning	—Unverified	0
Solving Continual Offline RL through Selective Weights Activation on Aligned Spaces	Oct 21, 2024	Continual LearningLifelong learning	—Unverified	0
TS-ACL: Closed-Form Solution for Time Series-oriented Continual Learning	Oct 21, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Budgeted Online Continual Learning by Adaptive Layer Freezing and Frequency-based Sampling	Oct 19, 2024	Continual LearningRetrieval	—Unverified	0
Controllable Discovery of Intents: Incremental Deep Clustering Using Semi-Supervised Contrastive Learning	Oct 18, 2024	ClusteringContinual Learning	—Unverified	0
Text-to-Image Representativity Fairness Evaluation Framework	Oct 18, 2024	Continual LearningFairness	—Unverified	0
CFTS-GAN: Continual Few-Shot Teacher Student for Generative Adversarial Networks	Oct 17, 2024	Continual LearningDiversity	—Unverified	0
CLIMB: Language-Guided Continual Learning for Task Planning with Iterative Model Building	Oct 17, 2024	Continual LearningDescriptive	—Unverified	0
SeQuiFi: Mitigating Catastrophic Forgetting in Speech Emotion Recognition with Sequential Class-Finetuning	Oct 16, 2024	Continual LearningEmotion Recognition	—Unverified	0
From Lab to Pocket: A Novel Continual Learning-based Mobile Application for Screening COVID-19	Oct 16, 2024	Continual Learning	CodeCode Available	0
ARIC: An Activity Recognition Dataset in Classroom Surveillance Images	Oct 16, 2024	Activity RecognitionContinual Learning	—Unverified	0
Towards Differentiable Multilevel Optimization: A Gradient-Based Approach	Oct 15, 2024	Continual Learning	—Unverified	0
ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy	Oct 14, 2024	Continual Learning	CodeCode Available	0
Is Parameter Collision Hindering Continual Learning in LLMs?	Oct 14, 2024	Continual Learning	CodeCode Available	1
Continual Learning Improves Zero-Shot Action Recognition	Oct 14, 2024	Action RecognitionContinual Learning	—Unverified	0
Continual Learning with Neuromorphic Computing: Theories, Methods, and Applications	Oct 11, 2024	Continual Learning	—Unverified	0
Temporal-Difference Variational Continual Learning	Oct 10, 2024	Continual Learning	CodeCode Available	0
Continual Learning: Less Forgetting, More OOD Generalization via Adaptive Contrastive Replay	Oct 9, 2024	Continual LearningContrastive Learning	CodeCode Available	0
Efficient Weight-Space Laplace-Gaussian Filtering and Smoothing for Sequential Deep Learning	Oct 9, 2024	Bayesian InferenceContinual Learning	—Unverified	0
Continual Learning in the Frequency Domain	Oct 9, 2024	Continual Learning	CodeCode Available	1
MIRACLE 3D: Memory-efficient Integrated Robust Approach for Continual Learning on Point Clouds via Shape Model construction	Oct 8, 2024	3D Object ClassificationContinual Learning	—Unverified	0
Does RoBERTa Perform Better than BERT in Continual Learning: An Attention Sink Perspective	Oct 8, 2024	Continual LearningDiversity	CodeCode Available	0
ModalPrompt:Dual-Modality Guided Prompt for Continual Learning of Large Multimodal Models	Oct 8, 2024	Continual LearningPrompt Learning	—Unverified	0
Brain-inspired continual pre-trained learner via silent synaptic consolidation	Oct 8, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Recent Advances of Multimodal Continual Learning: A Comprehensive Survey	Oct 7, 2024	Continual LearningSurvey	CodeCode Available	2
Low-Rank Continual Personalization of Diffusion Models	Oct 7, 2024	Continual Learning	CodeCode Available	0
Leveraging Hierarchical Taxonomies in Prompt-based Continual Learning	Oct 6, 2024	Continual Learning	CodeCode Available	0

Show:10 25 50

← PrevPage 9 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified