Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1001–1050 of 2644 papers

Title	Date	Tasks	Status	Hype
Prompt Pool based Class-Incremental Continual Learning for Dialog State Tracking	Nov 17, 2023	Continual Learningdialog state tracking	CodeCode Available	1
Assurance for Deployed Continual Learning Systems	Nov 16, 2023	Continual Learningimage-classification	—Unverified	0
Online Continual Knowledge Learning for Language Models	Nov 16, 2023	Continual LearningFact Checking	—Unverified	0
Investigating the Impact of Weight Sharing Decisions on Knowledge Transfer in Continual Learning	Nov 16, 2023	Continual LearningDecision Making	—Unverified	0
Carpe Diem: On the Evaluation of World Knowledge in Lifelong Language Models	Nov 14, 2023	Continual LearningQuestion Answering	CodeCode Available	1
GlanceSeg: Real-time microaneurysm lesion segmentation with gaze-map-guided foundation model for early detection of diabetic retinopathy	Nov 14, 2023	Continual LearningLesion Segmentation	—Unverified	0
Leveraging Hamilton-Jacobi PDEs with time-dependent Hamiltonians for continual scientific machine learning	Nov 13, 2023	Computational EfficiencyContinual Learning	CodeCode Available	0
A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Tasks	Nov 13, 2023	class-incremental learningClass Incremental Learning	—Unverified	0
Concept Matching: Clustering-based Federated Continual Learning	Nov 12, 2023	ClusteringContinual Learning	—Unverified	0
Towards Continual Reinforcement Learning for Quadruped Robots	Nov 12, 2023	Continual Learningreinforcement-learning	—Unverified	0
Online Continual Learning via Logit Adjusted Softmax	Nov 11, 2023	Continual Learning	CodeCode Available	0
When Meta-Learning Meets Online and Continual Learning: A Survey	Nov 9, 2023	Continual LearningMeta-Learning	—Unverified	0
Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How	Nov 8, 2023	Continual Learning	CodeCode Available	0
Class-Incremental Continual Learning for General Purpose Healthcare Models	Nov 7, 2023	Continual Learning	—Unverified	0
Learning to Learn for Few-shot Continual Active Learning	Nov 7, 2023	Active LearningContinual Learning	—Unverified	0
Task Arithmetic with LoRA for Continual Learning	Nov 4, 2023	Continual LearningTask Arithmetic	—Unverified	0
Continual Learning of Unsupervised Monocular Depth from Videos	Nov 4, 2023	Autonomous DrivingContinual Learning	CodeCode Available	0
Continual Learning Under Language Shift	Nov 2, 2023	Continual LearningLanguage Modeling	—Unverified	0
Look-Ahead Selective Plasticity for Continual Learning of Visual Tasks	Nov 2, 2023	Continual LearningRepresentation Learning	CodeCode Available	0
A Definition of Open-Ended Learning Problems for Goal-Conditioned Agents	Nov 1, 2023	Continual LearningLifelong learning	—Unverified	0
Implicit biases in multitask and continual learning from a backward error analysis perspective	Nov 1, 2023	Continual Learning	—Unverified	0
Continual atlas-based segmentation of prostate MRI	Nov 1, 2023	Continual Learningimage-classification	CodeCode Available	1
Class Incremental Learning with Pre-trained Vision-Language Models	Oct 31, 2023	class-incremental learningClass Incremental Learning	—Unverified	0
Look At Me, No Replay! SurpriseNet: Anomaly Detection Inspired Class Incremental Learning	Oct 30, 2023	Anomaly Detectionclass-incremental learning	CodeCode Available	0
Meta-Learning Strategies through Value Maximization in Neural Networks	Oct 30, 2023	Continual LearningMeta-Learning	—Unverified	0
Fast Trainable Projection for Robust Fine-Tuning	Oct 29, 2023	Continual Learning	CodeCode Available	1
Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations	Oct 27, 2023	Continual LearningNeRF	—Unverified	0
CL-MASR: A Continual Learning Benchmark for Multilingual ASR	Oct 25, 2023	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	1
Minimax Forward and Backward Learning of Evolving Tasks with Performance Guarantees	Oct 24, 2023	Continual LearningIncremental Learning	CodeCode Available	0
TiC-CLIP: Continual Training of CLIP Models	Oct 24, 2023	Continual LearningRetrieval	CodeCode Available	0
COPR: Continual Learning Human Preference through Optimal Policy Regularization	Oct 24, 2023	Continual Learningreinforcement-learning	—Unverified	0
Visually Grounded Continual Language Learning with Selective Specialization	Oct 24, 2023	Continual LearningDiagnostic	CodeCode Available	0
Synthetic Data as Validation	Oct 24, 2023	Computed Tomography (CT)Continual Learning	—Unverified	0
SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding	Oct 23, 2023	Continual LearningMulti-Task Learning	—Unverified	0
Coordinated Replay Sample Selection for Continual Federated Learning	Oct 23, 2023	Continual LearningFederated Learning	—Unverified	0
CITB: A Benchmark for Continual Instruction Tuning	Oct 23, 2023	Continual Learning	CodeCode Available	0
Continual Named Entity Recognition without Catastrophic Forgetting	Oct 23, 2023	Continual LearningContinual Named Entity Recognition	CodeCode Available	1
A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application	Oct 22, 2023	Continual LearningContinual Semantic Segmentation	CodeCode Available	1
Orthogonal Subspace Learning for Language Model Continual Learning	Oct 22, 2023	Continual LearningLanguage Modeling	CodeCode Available	1
From Static to Dynamic: A Continual Learning Framework for Large Language Models	Oct 22, 2023	Continual Learning	CodeCode Available	0
Towards a General Framework for Continual Learning with Pre-training	Oct 21, 2023	Continual Learningparameter-efficient fine-tuning	CodeCode Available	1
Continual Invariant Risk Minimization	Oct 21, 2023	Continual Learning	—Unverified	0
Technical Report for ICCV 2023 Visual Continual Learning Challenge: Continuous Test-time Adaptation for Semantic Segmentation	Oct 20, 2023	Continual LearningSemantic Segmentation	—Unverified	0
Towards a Deep Learning-based Online Quality Prediction System for Welding Processes	Oct 19, 2023	ARCContinual Learning	—Unverified	0
Recasting Continual Learning as Sequence Modeling	Oct 18, 2023	Continual Learning	CodeCode Available	0
A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm	Oct 18, 2023	Continual LearningIncremental Learning	CodeCode Available	1
Bayesian Flow Networks in Continual Learning	Oct 18, 2023	Bayesian InferenceContinual Learning	—Unverified	0
Prior-Free Continual Learning with Unlabeled Data in the Wild	Oct 16, 2023	Continual Learningimage-classification	CodeCode Available	0
Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation	Oct 15, 2023	Continual LearningTransfer Learning	—Unverified	0
Neuronal Auditory Machine Intelligence (NEURO-AMI) In Perspective	Oct 14, 2023	Continual Learning	—Unverified	0

Show:10 25 50

← PrevPage 21 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified