Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 951–1000 of 2644 papers

Title	Date	Tasks	Status	Hype
LAMM: Label Alignment for Multi-Modal Prompt Learning	Dec 13, 2023	Continual LearningPrompt Learning	CodeCode Available	1
Continual Learning through Networks Splitting and Merging with Dreaming-Meta-Weighted Model Fusion	Dec 12, 2023	Continual Learning	—Unverified	0
Spatiotemporal Event Graphs for Dynamic Scene Understanding	Dec 11, 2023	Action DetectionActivity Detection	—Unverified	0
Class-Prototype Conditional Diffusion Model with Gradient Projection for Continual Learning	Dec 10, 2023	Continual LearningDenoising	—Unverified	0
NeVRF: Neural Video-based Radiance Fields for Long-duration Sequences	Dec 10, 2023	Continual LearningNeRF	—Unverified	0
Continual learning for surface defect segmentation by subnetwork creation and selection	Dec 8, 2023	Continual LearningLifelong learning	CodeCode Available	0
Cross-Modal Alternating Learning with Task-Aware Representations for Continual Learning	Dec 7, 2023	Continual Learningimage-classification	CodeCode Available	0
Wake-Sleep Consolidated Learning	Dec 6, 2023	Continual LearningHippocampus	—Unverified	0
Benchmarking Continual Learning from Cognitive Perspectives	Dec 6, 2023	BenchmarkingContinual Learning	—Unverified	0
Pseudo Replay-based Class Continual Learning for Online New Category Anomaly Detection in Advanced Manufacturing	Dec 5, 2023	Anomaly Detectionclass-incremental learning	—Unverified	0
Distributed Continual Learning with CoCoA in High-dimensional Linear Regression	Dec 4, 2023	Continual Learningregression	—Unverified	0
Efficient Expansion and Gradient Based Task Inference for Replay Free Incremental Learning	Dec 2, 2023	class-incremental learningClass Incremental Learning	—Unverified	0
Towards Redundancy-Free Sub-networks in Continual Learning	Dec 1, 2023	Continual Learning	CodeCode Available	0
Label Delay in Online Continual Learning	Dec 1, 2023	Continual LearningGPU	—Unverified	0
Improving Plasticity in Online Continual Learning via Collaborative Learning	Dec 1, 2023	Continual Learning	CodeCode Available	1
Automating Continual Learning	Dec 1, 2023	Continual Learningimage-classification	CodeCode Available	1
Directions of Curvature as an Explanation for Loss of Plasticity	Nov 30, 2023	Continual Learning	—Unverified	0
Negotiated Representations to Prevent Forgetting in Machine Learning Applications	Nov 30, 2023	Continual LearningImage Classification	CodeCode Available	0
Continual Diffusion with STAMINA: STack-And-Mask INcremental Adapters	Nov 30, 2023	Continual LearningHard Attention	—Unverified	0
Adversarial Robust Memory-Based Continual Learner	Nov 29, 2023	Adversarial RobustnessContinual Learning	CodeCode Available	0
Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learning	Nov 29, 2023	Continual LearningContinual Self-Supervised Learning	CodeCode Available	1
Continual Learning with Low Rank Adaptation	Nov 29, 2023	Continual LearningIncremental Learning	—Unverified	0
Continual Learning for Image Segmentation with Dynamic Query	Nov 29, 2023	Continual LearningDiversity	CodeCode Available	1
SCStory: Self-supervised and Continual Online Story Discovery	Nov 27, 2023	ArticlesContinual Learning	CodeCode Available	0
Class-Adaptive Sampling Policy for Efficient Continual Learning	Nov 27, 2023	Continual Learning	CodeCode Available	0
Continual Instruction Tuning for Large Multimodal Models	Nov 27, 2023	Continual Learning	—Unverified	0
FALCON: Fairness Learning via Contrastive Attention Approach to Continual Semantic Scene Understanding	Nov 27, 2023	Continual LearningContinual Semantic Segmentation	—Unverified	0
Efficient Rehearsal Free Zero Forgetting Continual Learning using Adaptive Weight Modulation	Nov 26, 2023	Continual Learning	—Unverified	0
Hessian Aware Low-Rank Perturbation for Order-Robust Continual Learning	Nov 26, 2023	Continual Learning	CodeCode Available	0
KOPPA: Improving Prompt-based Continual Learning with Key-Query Orthogonal Projection and Prototype-based One-Versus-All	Nov 26, 2023	AllContinual Learning	—Unverified	0
CUCL: Codebook for Unsupervised Continual Learning	Nov 25, 2023	Continual LearningDiversity	CodeCode Available	0
Class Gradient Projection For Continual Learning	Nov 25, 2023	Continual LearningContrastive Learning	CodeCode Available	0
Learning in Deep Factor Graphs with Gaussian Belief Propagation	Nov 24, 2023	Continual LearningDenoising	CodeCode Available	0
Continual Learning of Diffusion Models with Generative Distillation	Nov 23, 2023	Continual LearningDenoising	CodeCode Available	1
Density Distribution-based Learning Framework for Addressing Online Continual Learning Challenges	Nov 22, 2023	class-incremental learningClass Incremental Learning	CodeCode Available	0
Evaluating Pretrained models for Deployable Lifelong Learning	Nov 22, 2023	Atari Gamesclass-incremental learning	—Unverified	0
CMFDFormer: Transformer-based Copy-Move Forgery Detection with Continual Learning	Nov 22, 2023	Continual Learning	—Unverified	0
Revisiting Supervision for Continual Representation Learning	Nov 22, 2023	Continual LearningRepresentation Learning	CodeCode Available	0
CovarNav: Machine Unlearning via Model Inversion and Covariance Navigation	Nov 21, 2023	Continual LearningMachine Unlearning	—Unverified	0
Digital Twin-Based User-Centric Edge Continual Learning in Integrated Sensing and Communication	Nov 20, 2023	Continual LearningEdge-computing	—Unverified	0
What Can AutoML Do For Continual Learning?	Nov 20, 2023	AutoMLContinual Learning	—Unverified	0
Replay-enhanced Continual Reinforcement Learning	Nov 20, 2023	Continual Learningreinforcement-learning	CodeCode Available	0
An NMF-Based Building Block for Interpretable Neural Networks With Continual Learning	Nov 20, 2023	Continual Learning	CodeCode Available	0
BrainWash: A Poisoning Attack to Forget in Continual Learning	Nov 20, 2023	Continual LearningData Poisoning	—Unverified	0
Continual Learning: Applications and the Road Forward	Nov 20, 2023	Continual LearningModel Editing	—Unverified	0
Negotiated Representations for Machine Mearning Application	Nov 19, 2023	Continual LearningMemorization	CodeCode Available	0
LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms	Nov 19, 2023	Continual LearningMeta-Learning	CodeCode Available	0
From Categories to Classifiers: Name-Only Continual Learning by Exploring the Web	Nov 19, 2023	Continual Learningimage-classification	—Unverified	0
One Size Fits All for Semantic Shifts: Adaptive Prompt Tuning for Continual Learning	Nov 18, 2023	AllContinual Learning	CodeCode Available	0
PACOL: Poisoning Attacks Against Continual Learners	Nov 18, 2023	Continual LearningData Poisoning	—Unverified	0

Show:10 25 50

← PrevPage 20 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified