Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1401–1450 of 2644 papers

Title	Date	Tasks	Status	Hype
Better Generative Replay for Continual Federated Learning	Feb 25, 2023	Continual LearningFederated Learning	—Unverified	0
Detachedly Learn a Classifier for Class-Incremental Learning	Feb 23, 2023	class-incremental learningClass Incremental Learning	—Unverified	0
A Neural Span-Based Continual Named Entity Recognition Model	Feb 23, 2023	Continual LearningContinual Named Entity Recognition	CodeCode Available	1
Preventing Catastrophic Forgetting in Continual Learning of New Natural Language Tasks	Feb 22, 2023	Continual LearningMulti-Task Learning	—Unverified	0
Effects of Architectures on Continual Semantic Segmentation	Feb 21, 2023	Continual LearningContinual Semantic Segmentation	—Unverified	0
Emphasizing Unseen Words: New Vocabulary Acquisition for End-to-End Speech Recognition	Feb 20, 2023	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified	0
New Insights for the Stability-Plasticity Dilemma in Online Continual Learning	Feb 17, 2023	Continual Learning	CodeCode Available	1
Efficiency 360: Efficient Vision Transformers	Feb 16, 2023	Continual LearningFairness	CodeCode Available	1
New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning	Feb 16, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	0
À-la-carte Prompt Tuning (APT): Combining Distinct Data Via Composable Prompting	Feb 15, 2023	Continual Learning	—Unverified	0
Error Sensitivity Modulation based Experience Replay: Mitigating Abrupt Representation Drift in Continual Learning	Feb 14, 2023	Continual LearningLifelong learning	CodeCode Available	0
Task-Aware Information Routing from Common Representation Space in Lifelong Learning	Feb 14, 2023	Continual LearningLifelong learning	CodeCode Available	0
Task-Specific Skill Localization in Fine-tuned Language Models	Feb 13, 2023	Continual Learningparameter-efficient fine-tuning	CodeCode Available	1
Theory on Forgetting and Generalization of Continual Learning	Feb 12, 2023	Continual Learning	—Unverified	0
Generalized Few-Shot Continual Learning with Contrastive Mixture of Adapters	Feb 12, 2023	Continual LearningContrastive Learning	CodeCode Available	1
Lightweight Transformers for Clinical Natural Language Processing	Feb 9, 2023	Continual LearningKnowledge Distillation	CodeCode Available	1
Class-Incremental Learning: A Survey	Feb 7, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	2
Continual Pre-training of Language Models	Feb 7, 2023	Continual LearningContinual Pretraining	CodeCode Available	2
Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learning	Feb 7, 2023	Continual LearningRepresentation Learning	—Unverified	0
On the Limitation and Experience Replay for GNNs in Continual Learning	Feb 7, 2023	Continual LearningIncremental Learning	—Unverified	0
Efficient Parametric Approximations of Neural Network Function Space Distance	Feb 7, 2023	Continual Learning	—Unverified	0
Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Network	Feb 6, 2023	Continual Learning	—Unverified	0
Avalanche: A PyTorch Library for Deep Continual Learning	Feb 2, 2023	class-incremental learningClass Incremental Learning	—Unverified	0
Continual Learning with Scaled Gradient Projection	Feb 2, 2023	Continual Learningimage-classification	CodeCode Available	1
Real-Time Evaluation in Online Continual Learning: A New Hope	Feb 2, 2023	Continual Learning	CodeCode Available	1
Multi-scale Feature Alignment for Continual Learning of Unlabeled Domains	Feb 2, 2023	Continual LearningDomain Adaptation	—Unverified	0
Neuro-Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsal	Feb 2, 2023	Continual Learning	CodeCode Available	1
Online Continual Learning via the Knowledge Invariant and Spread-out Properties	Feb 2, 2023	Continual Learning	—Unverified	0
Open-VCLIP: Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimization	Feb 1, 2023	Action RecognitionContinual Learning	CodeCode Available	1
A Comprehensive Survey of Continual Learning: Theory, Method and Application	Jan 31, 2023	Continual LearningLearning Theory	CodeCode Available	1
IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing	Jan 31, 2023	Anomaly DetectionContinual Learning	CodeCode Available	1
Continual Learning for Predictive Maintenance: Overview and Challenges	Jan 29, 2023	Continual Learning	—Unverified	0
Progressive Prompts: Continual Learning for Language Models	Jan 29, 2023	Continual Learning	CodeCode Available	1
Adversarial Learning Networks: Source-free Unsupervised Domain Incremental Learning	Jan 28, 2023	Continual LearningDisease Prediction	—Unverified	0
Restricted Orthogonal Gradient Projection for Continual Learning	Jan 28, 2023	Continual LearningTransfer Learning	—Unverified	0
CyclicFL: A Cyclic Model Pre-Training Approach to Efficient Federated Learning	Jan 28, 2023	Continual LearningFederated Learning	—Unverified	0
Continual Graph Learning: A Survey	Jan 28, 2023	Continual LearningGraph Learning	—Unverified	0
Streaming LifeLong Learning With Any-Time Inference	Jan 27, 2023	AI AgentContinual Learning	—Unverified	0
Class-Incremental Learning with Repetition	Jan 26, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	0
Towards Continual Egocentric Activity Recognition: A Multi-modal Egocentric Activity Dataset for Continual Learning	Jan 26, 2023	Activity RecognitionContinual Learning	CodeCode Available	1
Probabilistic Bilevel Coreset Selection	Jan 24, 2023	Bilevel OptimizationContinual Learning	—Unverified	0
Adapting a Language Model While Preserving its General Knowledge	Jan 21, 2023	Continual LearningGeneral Knowledge	CodeCode Available	2
Adaptively Integrated Knowledge Distillation and Prediction Uncertainty for Continual Learning	Jan 18, 2023	Continual LearningKnowledge Distillation	—Unverified	0
Hierarchical Memory Pool Based Edge Semi-Supervised Continual Learning Method	Jan 17, 2023	Continual Learning	—Unverified	0
Artificial Neuronal Ensembles with Learned Context Dependent Gating	Jan 17, 2023	Continual LearningPermuted-MNIST	CodeCode Available	0
Knowledge Acquisition and Completion for Long-Term Human-Robot Interactions using Knowledge Graph Embedding	Jan 17, 2023	Continual LearningGraph Embedding	—Unverified	0
Self-recovery of memory via generative replay	Jan 15, 2023	Continual Learning	—Unverified	0
Predictive World Models from Real-World Partial Observations	Jan 12, 2023	Continual LearningOpen-Ended Question Answering	CodeCode Available	0
A Distinct Unsupervised Reference Model From The Environment Helps Continual Learning	Jan 11, 2023	Continual Learning	—Unverified	0
Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learning	Jan 11, 2023	class-incremental learningClass Incremental Learning	—Unverified	0

Show:10 25 50

← PrevPage 29 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified