Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1651–1700 of 2644 papers

Title	Date	Tasks	Status	Hype
Continual Learning with Deep Learning Methods in an Application-Oriented Context	Jul 12, 2022	BIG-bench Machine LearningContinual Learning	—Unverified	0
A developmental approach for training deep belief networks	Jul 12, 2022	Continual Learning	—Unverified	0
Susceptibility of Continual Learning Against Adversarial Attacks	Jul 11, 2022	class-incremental learningClass Incremental Learning	—Unverified	0
Learning an evolved mixture model for task-free continual learning	Jul 11, 2022	Continual LearningDiversity	—Unverified	0
Repairing Neural Networks by Leaving the Right Past Behind	Jul 11, 2022	Continual Learning	—Unverified	0
Online Continual Learning of End-to-End Speech Recognition Models	Jul 11, 2022	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified	0
Consistency is the key to further mitigating catastrophic forgetting in continual learning	Jul 11, 2022	Continual LearningSelf-Supervised Learning	CodeCode Available	0
Challenging Common Assumptions about Catastrophic Forgetting	Jul 10, 2022	Continual LearningMemorization	—Unverified	0
Progressive Latent Replay for efficient Generative Rehearsal	Jul 4, 2022	Continual Learning	—Unverified	0
Memory Population in Continual Learning via Outlier Elimination	Jul 4, 2022	Continual Learning	CodeCode Available	0
Forget-free Continual Learning with Winning Subnetworks	Jul 1, 2022	Continual Learning	CodeCode Available	1
Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System	Jul 1, 2022	Continual Learningtext-classification	—Unverified	0
Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection	Jul 1, 2022	Continual LearningContrastive Learning	—Unverified	0
Continual Learning for Human State Monitoring	Jun 29, 2022	Continual LearningTime Series	CodeCode Available	0
Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision	Jun 29, 2022	Continual LearningGPU	CodeCode Available	1
Hebbian Continual Representation Learning	Jun 28, 2022	BIG-bench Machine Learningclass-incremental learning	—Unverified	0
NERDA-Con: Extending NER models for Continual Learning -- Integrating Distinct Tasks and Updating Distribution Shifts	Jun 28, 2022	Continual Learningnamed-entity-recognition	CodeCode Available	1
Continual Learning with Transformers for Image Classification	Jun 28, 2022	ClassificationContinual Learning	—Unverified	0
SHELS: Exclusive Feature Sets for Novelty Detection and Continual Learning Without Class Boundaries	Jun 28, 2022	class-incremental learningClass Incremental Learning	CodeCode Available	0
On-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling	Jun 27, 2022	Continual Learning	—Unverified	0
Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computing	Jun 27, 2022	Continual Learning	CodeCode Available	0
Sample Condensation in Online Continual Learning	Jun 23, 2022	Continual Learning	CodeCode Available	0
NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks	Jun 18, 2022	Continual Learning	CodeCode Available	1
CLiMB: A Continual Learning Benchmark for Vision-and-Language Tasks	Jun 18, 2022	Continual LearningTransfer Learning	CodeCode Available	1
Debugging using Orthogonal Gradient Descent	Jun 17, 2022	Continual Learning	—Unverified	0
Towards Diverse Evaluation of Class Incremental Learning: A Representation Learning Perspective	Jun 16, 2022	class-incremental learningClass Incremental Learning	—Unverified	0
Continual Learning with Guarantees via Weight Interval Constraints	Jun 16, 2022	Continual Learning	CodeCode Available	0
Lifelong Wandering: A realistic few-shot online continual learning setting	Jun 16, 2022	Continual LearningFew-Shot Learning	—Unverified	0
Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning	Jun 15, 2022	Adversarial Robustnessclass-incremental learning	CodeCode Available	0
Learning towards Synchronous Network Memorizability and Generalizability for Continual Segmentation across Multiple Sites	Jun 14, 2022	Continual Learning	CodeCode Available	0
Continual-Learning-as-a-Service (CLaaS): On-Demand Efficient Adaptation of Predictive Models	Jun 14, 2022	AttributeBIG-bench Machine Learning	—Unverified	0
What Should I Know? Using Meta-gradient Descent for Predictive Feature Discovery in a Single Stream of Experience	Jun 13, 2022	Continual LearningDecision Making	—Unverified	0
Matching options to tasks using Option-Indexed Hierarchical Reinforcement Learning	Jun 12, 2022	Continual LearningHierarchical Reinforcement Learning	—Unverified	0
Exploring the Intersection between Neural Architecture Search and Continual Learning	Jun 11, 2022	AutoMLContinual Learning	—Unverified	0
A Study of Continual Learning Methods for Q-Learning	Jun 8, 2022	Continual LearningQ-Learning	—Unverified	0
SYNERgy between SYNaptic consolidation and Experience Replay for general continual learning	Jun 8, 2022	Continual Learning	CodeCode Available	0
Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks	Jun 6, 2022	Continual LearningDataset Distillation	CodeCode Available	1
Effects of Auxiliary Knowledge on Continual Learning	Jun 3, 2022	Continual Learningimage-classification	—Unverified	0
Analysis of Catastrophic Forgetting for Random Orthogonal Transformation Tasks in the Overparameterized Regime	Jun 1, 2022	Continual Learningimage-classification	—Unverified	0
Dataset Distillation using Neural Feature Regression	Jun 1, 2022	Continual LearningDataset Distillation	CodeCode Available	0
Transfer without Forgetting	Jun 1, 2022	Continual LearningTransfer Learning	CodeCode Available	1
Label-Efficient Online Continual Object Detection in Streaming Video	Jun 1, 2022	Continual LearningHippocampus	CodeCode Available	1
Continual Object Detection: A review of definitions, strategies, and challenges	May 30, 2022	Autonomous VehiclesClass-Incremental Object Detection	—Unverified	0
ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection	May 30, 2022	class-incremental learningClass Incremental Learning	—Unverified	0
Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challenges	May 28, 2022	Continual LearningContinuous Control	CodeCode Available	1
The Effect of Task Ordering in Continual Learning	May 26, 2022	Continual LearningExperimental Design	—Unverified	0
Feature Forgetting in Continual Representation Learning	May 26, 2022	Continual LearningLifelong learning	—Unverified	0
Continual Learning for Visual Search with Backward Consistent Feature Embedding	May 26, 2022	Continual Learning	CodeCode Available	1
Continual evaluation for lifelong learning: Identifying the stability gap	May 26, 2022	Continual LearningIncremental Learning	CodeCode Available	1
An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems	May 25, 2022	Continual LearningFine-Grained Image Classification	CodeCode Available	0

Show:10 25 50

← PrevPage 34 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified