Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 851–900 of 2644 papers

Title	Date	Tasks	Status	Hype
Efficient Parameter Mining and Freezing for Continual Object Detection	Feb 20, 2024	Continual LearningIncremental Learning	—Unverified	0
Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervision	Feb 20, 2024	Continual Learning	—Unverified	0
Structural Knowledge Informed Continual Multivariate Time Series Forecasting	Feb 20, 2024	Continual LearningGraph structure learning	—Unverified	0
Neuro-mimetic Task-free Unsupervised Online Learning with Continual Self-Organizing Maps	Feb 19, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification	Feb 19, 2024	ClassificationContinual Learning	CodeCode Available	0
Towards Cross-Domain Continual Learning	Feb 19, 2024	Continual Learning	CodeCode Available	0
Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks	Feb 19, 2024	Continual Learning	CodeCode Available	1
Mitigating Catastrophic Forgetting in Multi-domain Chinese Spelling Correction by Multi-stage Knowledge Transfer Framework	Feb 18, 2024	Continual LearningSpelling Correction	—Unverified	0
Continual Learning on Graphs: Challenges, Solutions, and Opportunities	Feb 18, 2024	Continual LearningGraph Learning	CodeCode Available	2
Maintaining Adversarial Robustness in Continuous Learning	Feb 17, 2024	Adversarial RobustnessContinual Learning	—Unverified	0
Uncertainty-guided annotation enhances segmentation with the human-in-the-loop	Feb 16, 2024	Continual Learning	—Unverified	0
Evaluating and Improving Continual Learning in Spoken Language Understanding	Feb 16, 2024	Continual LearningSpoken Language Understanding	—Unverified	0
What to Do When Your Discrete Optimization Is the Size of a Neural Network?	Feb 15, 2024	Continual Learningimage-classification	CodeCode Available	0
Is Continual Learning Ready for Real-world Challenges?	Feb 15, 2024	3D Semantic SegmentationContinual Learning	—Unverified	0
TinyCL: An Efficient Hardware Architecture for Continual Learning on Autonomous Systems	Feb 15, 2024	Continual LearningGPU	—Unverified	0
Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning	Feb 14, 2024	Continual Learning	CodeCode Available	0
Random Representations Outperform Online Continually Learned Representations	Feb 13, 2024	Continual LearningExemplar-Free	CodeCode Available	0
Distal Interference: Exploring the Limits of Model-Based Continual Learning	Feb 13, 2024	Continual Learning	—Unverified	0
The Physics of Learning: From Autoencoders to Truly Autonomous Learning Machines	Feb 12, 2024	Continual Learning	—Unverified	0
A Benchmark Grocery Dataset of Realworld Point Clouds From Single View	Feb 12, 2024	3D Point Cloud ClassificationContinual Learning	—Unverified	0
Where is the Truth? The Risk of Getting Confounded in a Continual World	Feb 9, 2024	Continual Learningvalid	CodeCode Available	0
Continual Learning on Graphs: A Survey	Feb 9, 2024	Continual LearningGraph Learning	—Unverified	0
Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery	Feb 8, 2024	Continual LearningContinual Semantic Segmentation	CodeCode Available	0
Continual Domain Adversarial Adaptation via Double-Head Discriminators	Feb 5, 2024	Continual LearningDomain Adaptation	—Unverified	0
Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives	Feb 5, 2024	Continual LearningMulti-Task Learning	CodeCode Available	2
CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay	Feb 2, 2024	Continual Learning	CodeCode Available	0
From Words to Molecules: A Survey of Large Language Models in Chemistry	Feb 2, 2024	Continual Learning	—Unverified	0
Class incremental learning with probability dampening and cascaded gated classifier	Feb 2, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
Continual Learning for Large Language Models: A Survey	Feb 2, 2024	Continual LearningContinual Pretraining	—Unverified	0
Multiple Integration Model for Single-source Domain Generalizable Person Re-identification	Feb 1, 2024	Continual LearningGeneralizable Person Re-identification	—Unverified	0
Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI	Feb 1, 2024	Continual LearningDeep Learning	—Unverified	0
Continuous Unsupervised Domain Adaptation Using Stabilized Representations and Experience Replay	Jan 31, 2024	Continual LearningDomain Adaptation	CodeCode Available	0
Benchmarking Sensitivity of Continual Graph Learning for Skeleton-Based Action Recognition	Jan 31, 2024	Action RecognitionBenchmarking	—Unverified	0
Step-size Optimization for Continual Learning	Jan 30, 2024	Continual Learning	—Unverified	0
Augmenting Replay in World Models for Continual Reinforcement Learning	Jan 30, 2024	Continual LearningModel-based Reinforcement Learning	CodeCode Available	0
EdgeOL: Efficient in-situ Online Learning on Edge Devices	Jan 30, 2024	Continual LearningObject Recognition	—Unverified	0
Continual Learning with Pre-Trained Models: A Survey	Jan 29, 2024	Continual LearningFairness	CodeCode Available	4
Sample Weight Estimation Using Meta-Updates for Online Continual Learning	Jan 29, 2024	Continual LearningMeta-Learning	CodeCode Available	0
Dynamic Transformer Architecture for Continual Learning of Multimodal Tasks	Jan 27, 2024	Continual LearningEdge-computing	—Unverified	0
Emulating Complex Synapses Using Interlinked Proton Conductors	Jan 26, 2024	Continual Learning	—Unverified	0
Improving conversion rate prediction via self-supervised pre-training in online advertising	Jan 25, 2024	Continual Learning	—Unverified	0
Topology-aware Embedding Memory for Continual Learning on Expanding Networks	Jan 24, 2024	Continual Learning	CodeCode Available	0
Locality Sensitive Sparse Encoding for Learning World Models Online	Jan 23, 2024	Continual LearningModel-based Reinforcement Learning	—Unverified	0
The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting -- An Analytical Model	Jan 23, 2024	Continual Learningregression	—Unverified	0
CloSe: A 3D Clothing Segmentation Dataset and Model	Jan 22, 2024	Continual Learningmodel	CodeCode Available	2
INCPrompt: Task-Aware incremental Prompting for Rehearsal-Free Class-incremental Learning	Jan 22, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Detecting Out-of-Distribution Samples via Conditional Distribution Entropy with Optimal Transport	Jan 22, 2024	Continual Learning	—Unverified	0
Hierarchical Prompts for Rehearsal-free Continual Learning	Jan 21, 2024	Continual Learning	—Unverified	0
Towards Open-World Gesture Recognition	Jan 20, 2024	Continual LearningGesture Recognition	—Unverified	0
Self-training from Self-memory in Data-to-text Generation	Jan 19, 2024	Continual LearningData-to-Text Generation	CodeCode Available	0

Show:10 25 50

← PrevPage 18 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified