Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1551–1600 of 2644 papers

Title	Date	Tasks	Status	Hype
Review Learning: Alleviating Catastrophic Forgetting with Generative Replay without Generator	Oct 17, 2022	Binary ClassificationContinual Learning	—Unverified	0
Navigating Memory Construction by Global Pseudo-Task Simulation for Continual Learning	Oct 16, 2022	Combinatorial OptimizationContinual Learning	CodeCode Available	0
Learning with Style: Continual Semantic Segmentation Across Tasks and Domains	Oct 13, 2022	Autonomous Drivingclass-incremental learning	—Unverified	0
Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation	Oct 13, 2022	Class-Incremental Semantic SegmentationContinual Learning	CodeCode Available	1
Task-Free Continual Learning via Online Discrepancy Distance Learning	Oct 12, 2022	Continual LearningGeneralization Bounds	—Unverified	0
Improving information retention in large scale online continual learning	Oct 12, 2022	Continual Learning	—Unverified	0
Federated Continual Learning for Text Classification via Selective Inter-client Transfer	Oct 12, 2022	Continual LearningFederated Learning	CodeCode Available	0
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning	Oct 12, 2022	Continual Learning	CodeCode Available	1
Toward Sustainable Continual Learning: Detection and Knowledge Repurposing of Similar Tasks	Oct 11, 2022	Continual Learning	—Unverified	0
Schedule-Robust Online Continual Learning	Oct 11, 2022	Continual Learningimage-classification	—Unverified	0
Continual Training of Language Models for Few-Shot Learning	Oct 11, 2022	Continual LearningContinual Pretraining	CodeCode Available	2
Continual Learning by Modeling Intra-Class Variation	Oct 11, 2022	Continual Learning	CodeCode Available	1
Towards Continual Adaptation in Industrial Anomaly Detection	Oct 10, 2022	Anomaly Detectioncontinual anomaly detection	CodeCode Available	1
Patch-based Knowledge Distillation for Lifelong Person Re-Identification	Oct 10, 2022	Continual LearningKnowledge Distillation	CodeCode Available	1
Continual Learning with Evolving Class Ontologies	Oct 10, 2022	class-incremental learningClass Incremental Learning	—Unverified	0
A Simple Baseline that Questions the Use of Pretrained-Models in Continual Learning	Oct 10, 2022	Continual LearningExemplar-Free	CodeCode Available	0
Tracking changes using Kullback-Leibler divergence for the continual learning	Oct 10, 2022	Continual Learning	—Unverified	0
Few-Shot Continual Active Learning by a Robot	Oct 9, 2022	Active LearningContinual Learning	—Unverified	0
TAME: Task Agnostic Continual Learning using Multiple Experts	Oct 8, 2022	Continual LearningLifelong learning	—Unverified	0
Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition	Oct 8, 2022	Causal InferenceContinual Learning	CodeCode Available	1
CLAD: A realistic Continual Learning benchmark for Autonomous Driving	Oct 7, 2022	Autonomous DrivingContinual Learning	CodeCode Available	1
Learnware: Small Models Do Big	Oct 7, 2022	Continual Learning	—Unverified	0
CLIP model is an Efficient Continual Learner	Oct 6, 2022	Continual LearningIncremental Learning	CodeCode Available	1
Topological Continual Learning with Wasserstein Distance and Barycenter	Oct 6, 2022	Continual Learningimage-classification	—Unverified	0
ImpressLearn: Continual Learning via Combined Task Impressions	Oct 5, 2022	Continual Learningimage-classification	—Unverified	0
SHINE-Mapping: Large-Scale 3D Mapping Using Sparse Hierarchical Implicit Neural Representations	Oct 5, 2022	3D ReconstructionContinual Learning	CodeCode Available	2
Efficient Meta-Learning for Continual Learning with Taylor Expansion Approximation	Oct 3, 2022	Computational EfficiencyContinual Learning	—Unverified	0
Analyzing BERT Cross-lingual Transfer Capabilities in Continual Sequence Labeling	Oct 1, 2022	Continual LearningCross-Lingual Transfer	CodeCode Available	0
HyperHawkes: Hypernetwork based Neural Temporal Point Process	Oct 1, 2022	Continual LearningZero-Shot Learning	—Unverified	0
Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answering	Sep 30, 2022	Continual LearningQuestion Answering	CodeCode Available	0
Multiple Modes for Continual Learning	Sep 29, 2022	Continual Learning	—Unverified	0
Understanding Collapse in Non-Contrastive Siamese Representation Learning	Sep 29, 2022	Continual LearningContrastive Learning	CodeCode Available	1
RECALL: Rehearsal-free Continual Learning for Object Classification	Sep 29, 2022	ClassificationContinual Learning	CodeCode Available	0
Disentangling Transfer in Continual Reinforcement Learning	Sep 28, 2022	Continual Learningcontinuous-control	—Unverified	0
A simple but strong baseline for online continual learning: Repeated Augmented Rehearsal	Sep 28, 2022	Continual LearningReinforcement Learning (RL)	CodeCode Available	1
Exploring Example Influence in Continual Learning	Sep 25, 2022	Continual Learning	CodeCode Available	1
Learning from Students: Online Contrastive Distillation Network for General Continual Learning	Sep 24, 2022	Continual Learning	CodeCode Available	1
Optimizing Class Distribution in Memory for Multi-Label Online Continual Learning	Sep 23, 2022	Continual Learning	—Unverified	0
SparCL: Sparse Continual Learning on the Edge	Sep 20, 2022	Continual Learning	CodeCode Available	1
Improving Replay-Based Continual Semantic Segmentation with Smart Data Selection	Sep 20, 2022	Continual LearningContinual Semantic Segmentation	CodeCode Available	0
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach	Sep 19, 2022	Bilevel OptimizationContinual Learning	CodeCode Available	1
Learn the Time to Learn: Replay Scheduling in Continual Learning	Sep 18, 2022	Continual LearningScheduling	CodeCode Available	0
Continual Learning for Class- and Domain-Incremental Semantic Segmentation	Sep 16, 2022	class-incremental learningClass Incremental Learning	—Unverified	0
Continual Learning with Dependency Preserving Hypernetworks	Sep 16, 2022	Continual Learningimage-classification	—Unverified	0
Online Continual Learning via the Meta-learning Update with Multi-scale Knowledge Distillation and Data Augmentation	Sep 12, 2022	Continual LearningData Augmentation	—Unverified	0
Continual Learning for Pose-Agnostic Object Recognition in 3D Point Clouds	Sep 11, 2022	Continual LearningObject Recognition	—Unverified	0
OmDet: Large-scale vision-language multi-dataset pre-training with multimodal detection network	Sep 10, 2022	Continual LearningObject	CodeCode Available	3
Continual learning benefits from multiple sleep mechanisms: NREM, REM, and Synaptic Downscaling	Sep 9, 2022	Continual Learningimage-classification	—Unverified	0
Selecting Related Knowledge via Efficient Channel Attention for Online Continual Learning	Sep 9, 2022	Continual LearningKnowledge Distillation	—Unverified	0
nVFNet-RDC: Replay and Non-Local Distillation Collaboration for Continual Object Detection	Sep 8, 2022	Continual Learningobject-detection	—Unverified	0

Show:10 25 50

← PrevPage 32 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified