Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 801–850 of 2644 papers

Title	Date	Tasks	Status	Hype
Open Continual Feature Selection via Granular-Ball Knowledge Transfer	Mar 15, 2024	Continual Learningfeature selection	CodeCode Available	0
Hyperparameters in Continual Learning: A Reality Check	Mar 14, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Select and Distill: Selective Dual-Teacher Knowledge Transfer for Continual Learning on Vision-Language Models	Mar 14, 2024	Continual LearningKnowledge Distillation	—Unverified	0
FOCIL: Finetune-and-Freeze for Online Class Incremental Learning by Training Randomly Pruned Sparse Experts	Mar 13, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
Training Self-localization Models for Unseen Unfamiliar Places via Teacher-to-Student Data-Free Knowledge Transfer	Mar 13, 2024	Continual LearningImage Retrieval	—Unverified	0
Probabilistic Metaplasticity for Continual Learning with Memristors	Mar 13, 2024	Continual Learning	—Unverified	0
Consistent Prompting for Rehearsal-Free Continual Learning	Mar 13, 2024	Continual Learning	CodeCode Available	1
Simple and Scalable Strategies to Continually Pre-train Large Language Models	Mar 13, 2024	Continual LearningLanguage Modelling	—Unverified	0
MergeOcc: Bridge the Domain Gap between Different LiDARs for Robust Occupancy Prediction	Mar 13, 2024	Autonomous VehiclesContinual Learning	—Unverified	0
DAM: Dynamic Adapter Merging for Continual Video QA Learning	Mar 13, 2024	Continual Learningimage-classification	CodeCode Available	1
Continual Learning by Three-Phase Consolidation	Mar 12, 2024	Continual Learning	—Unverified	0
Towards a Dynamic Future with Adaptable Computing and Network Convergence (ACNC)	Mar 12, 2024	Continual LearningDimensionality Reduction	—Unverified	0
Online Continual Learning For Interactive Instruction Following Agents	Mar 12, 2024	Continual LearningIncremental Learning	CodeCode Available	1
Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neurons	Mar 12, 2024	Continual LearningModel Compression	—Unverified	0
Federated Learning of Socially Appropriate Agent Behaviours in Simulated Home Environments	Mar 12, 2024	Continual LearningFederated Learning	CodeCode Available	0
Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers	Mar 12, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Premonition: Using Generative Models to Preempt Future Data Changes in Continual Learning	Mar 12, 2024	Continual LearningFine-Grained Image Classification	CodeCode Available	0
12 mJ per Class On-Device Online Few-Shot Class-Incremental Learning	Mar 12, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure	Mar 12, 2024	AllContinual Learning	CodeCode Available	1
Semantic Residual Prompts for Continual Learning	Mar 11, 2024	Continual Learning	—Unverified	0
Last Iterate Convergence of Incremental Methods and Applications in Continual Learning	Mar 11, 2024	Continual Learning	—Unverified	0
On the Diminishing Returns of Width for Continual Learning	Mar 11, 2024	Continual LearningLearning Theory	CodeCode Available	0
Adaptive Hyperparameter Optimization for Continual Learning Scenarios	Mar 9, 2024	Continual LearningHyperparameter Optimization	—Unverified	0
Continual Learning and Catastrophic Forgetting	Mar 8, 2024	Continual Learning	—Unverified	0
TEGEE: Task dEfinition Guided Expert Ensembling for Generalizable and Few-shot Learning	Mar 7, 2024	Continual LearningDefinition Extraction	—Unverified	0
Contrastive Continual Learning with Importance Sampling and Prototype-Instance Relation Distillation	Mar 7, 2024	Continual LearningContrastive Learning	CodeCode Available	1
Contrastive Augmented Graph2Graph Memory Interaction for Few Shot Continual Learning	Mar 7, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Exploring Continual Learning of Compositional Generalization in NLI	Mar 7, 2024	Continual LearningNatural Language Inference	CodeCode Available	0
LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error	Mar 7, 2024	Continual LearningIn-Context Learning	CodeCode Available	2
GUIDE: Guidance-based Incremental Learning with Diffusion Models	Mar 6, 2024	Continual LearningIncremental Learning	CodeCode Available	1
Continual Segmentation with Disentangled Objectness Learning and Class Recognition	Mar 6, 2024	Continual LearningSegmentation	CodeCode Available	0
Interactive Continual Learning Architecture for Long-Term Personalization of Home Service Robots	Mar 6, 2024	Continual Learning	—Unverified	0
Interactive Continual Learning: Fast and Slow Thinking	Mar 5, 2024	Continual LearningOutlier Detection	CodeCode Available	2
Recall-Oriented Continual Learning with Generative Adversarial Meta-Model	Mar 5, 2024	Continual LearningIncremental Learning	CodeCode Available	0
Gradient Correlation Subspace Learning against Catastrophic Forgetting	Mar 4, 2024	Continual Learning	CodeCode Available	0
Transformers for Supervised Online Continual Learning	Mar 3, 2024	Continual LearningFew-Shot Learning	—Unverified	0
Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal	Mar 2, 2024	Continual LearningIn-Context Learning	CodeCode Available	1
Towards Backward-Compatible Continual Learning of Image Compression	Feb 29, 2024	Continual LearningImage Compression	CodeCode Available	0
Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning	Feb 29, 2024	Continual LearningLanguage Modelling	CodeCode Available	1
Generalizable Two-Branch Framework for Image Class-Incremental Learning	Feb 28, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
HOP to the Next Tasks and Domains for Continual Learning in NLP	Feb 28, 2024	Continual Learning	—Unverified	0
Investigating Continual Pretraining in Large Language Models: Insights and Implications	Feb 27, 2024	Continual LearningContinual Pretraining	—Unverified	0
A Vanilla Multi-Task Framework for Dense Visual Prediction Solution to 1st VCL Challenge -- Multi-Task Robustness Track	Feb 27, 2024	3D Object DetectionContinual Learning	—Unverified	0
LLM-Assisted Multi-Teacher Continual Learning for Visual Question Answering in Robotic Surgery	Feb 26, 2024	Continual LearningExemplar-Free	CodeCode Available	0
Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learning	Feb 26, 2024	Continual LearningDomain Adaptation	—Unverified	0
One-stage Prompt-based Continual Learning	Feb 25, 2024	Continual Learning	—Unverified	0
Fixed Random Classifier Rearrangement for Continual Learning	Feb 23, 2024	Binary ClassificationContinual Learning	—Unverified	0
COPR: Continual Human Preference Learning via Optimal Policy Regularization	Feb 22, 2024	Continual Learning	—Unverified	0
DSLR: Diversity Enhancement and Structure Learning for Rehearsal-based Graph Continual Learning	Feb 21, 2024	Continual LearningDiversity	CodeCode Available	1
Can Similarity-Based Domain-Ordering Reduce Catastrophic Forgetting for Intent Recognition?	Feb 21, 2024	Continual LearningIntent Recognition	—Unverified	0

Show:10 25 50

← PrevPage 17 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified