Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 201–250 of 2644 papers

Title	Date	Tasks	Status	Hype
CBA: Improving Online Continual Learning via Continual Bias Adaptor	Aug 14, 2023	Continual Learning	CodeCode Available	1
CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation	Aug 14, 2023	Continual LearningContinual Pretraining	CodeCode Available	1
OpenGCD: Assisting Open World Recognition with Generalized Category Discovery	Aug 14, 2023	Continual LearningDiversity	CodeCode Available	1
Cost-effective On-device Continual Learning over Memory Hierarchy with Miro	Aug 11, 2023	Continual LearningEdge-computing	CodeCode Available	1
Do You Remember? Overcoming Catastrophic Forgetting for Fake Audio Detection	Aug 7, 2023	Continual LearningEmotion Recognition	CodeCode Available	1
Online Prototype Learning for Online Continual Learning	Aug 1, 2023	Continual LearningKnowledge Distillation	CodeCode Available	1
CLR: Channel-wise Lightweight Reprogramming for Continual Learning	Jul 21, 2023	Continual Learningimage-classification	CodeCode Available	1
EgoAdapt: A multi-stream evaluation study of adaptation to real-world egocentric user video	Jul 11, 2023	Action RecognitionContinual Learning	CodeCode Available	1
RanPAC: Random Projections and Pre-trained Models for Continual Learning	Jul 5, 2023	Continual Learning	CodeCode Available	1
Parameter-Level Soft-Masking for Continual Learning	Jun 26, 2023	Continual LearningIncremental Learning	CodeCode Available	1
Learnability and Algorithm for Continual Learning	Jun 22, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	1
Progressive Fourier Neural Representation for Sequential Video Compilation	Jun 20, 2023	Continual Learning	CodeCode Available	1
Memory-Based Dual Gaussian Processes for Sequential Learning	Jun 6, 2023	Active LearningBayesian Optimization	CodeCode Available	1
Task Relation-aware Continual User Representation Learning	Jun 1, 2023	Continual LearningRelation	CodeCode Available	1
Continual Learning for Abdominal Multi-Organ and Tumor Segmentation	Jun 1, 2023	Continual LearningOrgan Segmentation	CodeCode Available	1
Prediction Error-based Classification for Class-Incremental Learning	May 30, 2023	Classificationclass-incremental learning	CodeCode Available	1
Reduced Precision Floating-Point Optimization for Deep Neural Network On-Device Learning on MicroControllers	May 30, 2023	Continual Learningimage-classification	CodeCode Available	1
Learning without Forgetting for Vision-Language Models	May 30, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	1
Summarizing Stream Data for Memory-Constrained Online Continual Learning	May 26, 2023	Continual LearningInformativeness	CodeCode Available	1
MRN: Multiplexed Routing Network for Incremental Multilingual Text Recognition	May 24, 2023	Continual LearningIncremental Learning	CodeCode Available	1
Continual Learning on Dynamic Graphs via Parameter Isolation	May 23, 2023	Continual LearningGraph Learning	CodeCode Available	1
Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation	May 23, 2023	Continual Learning	CodeCode Available	1
Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models	May 17, 2023	Continual Learning	CodeCode Available	1
Online Continual Learning Without the Storage Constraint	May 16, 2023	Continual Learning	CodeCode Available	1
Continual Multimodal Knowledge Graph Construction	May 15, 2023	Continual Learninggraph construction	CodeCode Available	1
BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning	May 8, 2023	Continual LearningDiversity	CodeCode Available	1
Learning to Predict Navigational Patterns from Partial Observations	Apr 26, 2023	Continual LearningData Augmentation	CodeCode Available	1
SequeL: A Continual Learning Library in PyTorch and JAX	Apr 21, 2023	Continual Learning	CodeCode Available	1
Regularizing Second-Order Influences for Continual Learning	Apr 20, 2023	Continual Learning	CodeCode Available	1
Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases	Apr 18, 2023	Continual Learning	CodeCode Available	1
Continual Domain Adaptation through Pruning-aided Domain-specific Weight Modulation	Apr 15, 2023	Continual LearningDomain Adaptation	CodeCode Available	1
PCR: Proxy-based Contrastive Replay for Online Class-Incremental Continual Learning	Apr 10, 2023	Continual Learning	CodeCode Available	1
A multifidelity approach to continual learning for physical systems	Apr 8, 2023	Continual Learning	CodeCode Available	1
Continual Learning for LiDAR Semantic Segmentation: Class-Incremental and Coarse-to-Fine strategies on Sparse Data	Apr 8, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	1
Asynchronous Federated Continual Learning	Apr 7, 2023	Continual LearningFederated Learning	CodeCode Available	1
I2I: Initializing Adapters with Improvised Knowledge	Apr 4, 2023	Continual LearningQuestion Answering	CodeCode Available	1
Kaizen: Practical Self-supervised Continual Learning with Continual Fine-tuning	Mar 30, 2023	Continual LearningKnowledge Distillation	CodeCode Available	1
CoDeC: Communication-Efficient Decentralized Continual Learning	Mar 27, 2023	Continual LearningFederated Learning	CodeCode Available	1
Forget-free Continual Learning with Soft-Winning SubNetworks	Mar 27, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	1
Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learning	Mar 26, 2023	Continual LearningRelation	CodeCode Available	1
Preserving Linear Separability in Continual Learning by Backward Feature Projection	Mar 26, 2023	Continual LearningKnowledge Distillation	CodeCode Available	1
CCL: Continual Contrastive Learning for LiDAR Place Recognition	Mar 24, 2023	Autonomous DrivingContinual Learning	CodeCode Available	1
Leveraging Old Knowledge to Continually Learn New Classes in Medical Images	Mar 24, 2023	Continual Learning	CodeCode Available	1
SeiT: Storage-Efficient Vision Training with Tokens Using 1% of Pixel Storage	Mar 20, 2023	Continual Learning	CodeCode Available	1
Computationally Budgeted Continual Learning: What Does Matter?	Mar 20, 2023	Continual LearningGPU	CodeCode Available	1
Sparse Distributed Memory is a Continual Learner	Mar 20, 2023	Continual Learning	CodeCode Available	1
SIESTA: Efficient Online Continual Learning with Sleep	Mar 19, 2023	Computational EfficiencyContinual Learning	CodeCode Available	1
A Unified Continual Learning Framework with General Parameter-Efficient Tuning	Mar 17, 2023	Continual Learning	CodeCode Available	1
CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation	Mar 17, 2023	Continual LearningDepth Estimation	CodeCode Available	1
Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning	Mar 16, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	1

Show:10 25 50

← PrevPage 5 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified