Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 601–650 of 2644 papers

Title	Date	Tasks	Status	Hype
PCAPVision: PCAP-Based High-Velocity and Large-Volume Network Failure Detection	Jul 1, 2024	Continual LearningTransfer Learning	—Unverified	0
Mask and Compress: Efficient Skeleton-based Action Recognition in Continual Learning	Jul 1, 2024	Action RecognitionContinual Learning	CodeCode Available	0
Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning	Jul 1, 2024	Continual LearningMixture-of-Experts	—Unverified	0
Normalization and effective learning rates in reinforcement learning	Jul 1, 2024	Atari GamesContinual Learning	—Unverified	0
Less Forgetting for Better Generalization: Exploring Continual-learning Fine-tuning Methods for Speech Self-supervised Representations	Jun 30, 2024	Continual LearningDomain Generalization	—Unverified	0
TEAL: New Selection Strategy for Small Buffers in Experience Replay Class Incremental Learning	Jun 30, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	1
Learning System Dynamics without Forgetting	Jun 30, 2024	Continual LearningTrajectory Prediction	CodeCode Available	0
BAPO: Base-Anchored Preference Optimization for Overcoming Forgetting in Large Language Models Personalization	Jun 30, 2024	Continual LearningGeneral Knowledge	—Unverified	0
Characterizing Continual Learning Scenarios and Strategies for Audio Analysis	Jun 29, 2024	Continual Learning	—Unverified	0
The impact of model size on catastrophic forgetting in Online Continual Learning	Jun 28, 2024	class-incremental learningClass Incremental Learning	—Unverified	0
Attack On Prompt: Backdoor Attack in Prompt-Based Continual Learning	Jun 28, 2024	Backdoor AttackContinual Learning	—Unverified	0
Personalized Federated Continual Learning via Multi-granularity Prompt	Jun 27, 2024	Continual LearningFederated Learning	CodeCode Available	1
Multi-Epoch learning with Data Augmentation for Deep Click-Through Rate Prediction	Jun 27, 2024	Click-Through Rate PredictionContinual Learning	—Unverified	0
Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models	Jun 27, 2024	Continual LearningIncremental Learning	CodeCode Available	1
Enhancing Continual Learning in Visual Question Answering with Modality-Aware Feature Distillation	Jun 27, 2024	Continual LearningQuestion Answering	CodeCode Available	0
Towards Synchronous Memorizability and Generalizability with Site-Modulated Diffusion Replay for Cross-Site Continual Segmentation	Jun 26, 2024	Continual LearningDomain Generalization	CodeCode Available	0
Learn it or Leave it: Module Composition and Pruning for Continual Learning	Jun 26, 2024	Continual LearningTransfer Learning	—Unverified	0
Toward Universal Medical Image Registration via Sharpness-Aware Meta-Continual Learning	Jun 25, 2024	Continual LearningImage Registration	CodeCode Available	0
Forget but Recall: Incremental Latent Rectification in Continual Learning	Jun 25, 2024	Continual LearningIncremental Learning	—Unverified	0
Unlocking Continual Learning Abilities in Language Models	Jun 25, 2024	Continual LearningInductive Bias	CodeCode Available	1
Theory on Mixture-of-Experts in Continual Learning	Jun 24, 2024	Continual LearningMixture-of-Experts	—Unverified	0
EVCL: Elastic Variational Continual Learning with Weight Consolidation	Jun 23, 2024	Continual LearningIncremental Learning	CodeCode Available	0
Continual Learning with Diffusion-based Generative Replay for Industrial Streaming Data	Jun 22, 2024	Continual LearningKnowledge Distillation	—Unverified	0
Behaviour Distillation	Jun 21, 2024	Continual Learningcontinuous-control	CodeCode Available	0
VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought	Jun 20, 2024	Action AnticipationContinual Learning	—Unverified	0
Controlling Forgetting with Test-Time Data in Continual Learning	Jun 19, 2024	Continual Learning	—Unverified	0
Low-Rank Mixture-of-Experts for Continual Medical Image Segmentation	Jun 19, 2024	Continual LearningImage Segmentation	—Unverified	0
ViLCo-Bench: VIdeo Language COntinual learning Benchmark	Jun 19, 2024	Continual LearningSelf-Supervised Learning	CodeCode Available	1
Unsupervised Online Continual Learning for Automatic Speech Recognition	Jun 18, 2024	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	0
PromptDSI: Prompt-based Rehearsal-free Instance-wise Incremental Learning for Document Retrieval	Jun 18, 2024	Continual LearningIncremental Learning	—Unverified	0
CM2-Net: Continual Cross-Modal Mapping Network for Driver Action Recognition	Jun 17, 2024	Action RecognitionContinual Learning	—Unverified	0
Zero-Shot Generalization during Instruction Tuning: Insights from Similarity and Granularity	Jun 17, 2024	Continual LearningZero-shot Generalization	CodeCode Available	0
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation	Jun 16, 2024	Continual LearningGSM8K	CodeCode Available	0
Forgetting Order of Continual Learning: Examples That are Learned First are Forgotten Last	Jun 14, 2024	Continual Learningimage-classification	—Unverified	0
Data-dependent and Oracle Bounds on Forgetting in Continual Learning	Jun 13, 2024	Continual Learning	—Unverified	0
Reflecting on the State of Rehearsal-free Continual Learning with Pretrained Models	Jun 13, 2024	Continual Learning	—Unverified	0
Diffusion Soup: Model Merging for Text-to-Image Diffusion Models	Jun 12, 2024	Continual LearningImage Generation	—Unverified	0
Continuous fake media detection: adapting deepfake detectors to new generative techniques	Jun 12, 2024	Continual LearningDeepFake Detection	—Unverified	0
StreamFP: Learnable Fingerprint-guided Data Selection for Efficient Stream Learning	Jun 11, 2024	Computational EfficiencyContinual Learning	CodeCode Available	0
A Parameter-efficient Language Extension Framework for Multilingual ASR	Jun 10, 2024	Continual Learningparameter-efficient fine-tuning	—Unverified	0
Learning Continually by Spectral Regularization	Jun 10, 2024	Continual LearningDiversity	—Unverified	0
A Statistical Theory of Regularization-Based Continual Learning	Jun 10, 2024	Continual Learningregression	—Unverified	0
Visual Prompt Tuning in Null Space for Continual Learning	Jun 9, 2024	Continual LearningVisual Prompt Tuning	CodeCode Available	1
Online DPO: Online Direct Preference Optimization with Fast-Slow Chasing	Jun 8, 2024	Continual Learning	—Unverified	0
Data-Free Generative Replay for Class-Incremental Learning on Imbalanced Data	Jun 7, 2024	class-incremental learningClass Incremental Learning	CodeCode Available	0
Lifelong Learning of Video Diffusion Models From a Single Video Stream	Jun 7, 2024	Continual LearningLifelong learning	—Unverified	0
CLoG: Benchmarking Continual Learning of Image Generation Models	Jun 7, 2024	BenchmarkingContinual Learning	CodeCode Available	1
REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learning	Jun 7, 2024	Continual Learningimage-classification	—Unverified	0
The Expanding Scope of the Stability Gap: Unveiling its Presence in Joint Incremental Learning of Homogeneous Tasks	Jun 7, 2024	Continual LearningIncremental Learning	—Unverified	0
Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need	Jun 5, 2024	AllContinual Learning	—Unverified	0

Show:10 25 50

← PrevPage 13 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified