Continual Learning

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 2301–2350 of 2644 papers

Title	Date	Tasks	Status	Hype
Continual Class Incremental Learning for CT Thoracic Segmentation	Aug 12, 2020	class-incremental learningClass Incremental Learning	—Unverified	0
ARCADe: A Rapid Continual Anomaly Detector	Aug 10, 2020	Anomaly Detectioncontinual anomaly detection	CodeCode Available	1
Meta Continual Learning via Dynamic Programming	Aug 5, 2020	Continual Learning	CodeCode Available	0
Online Continual Learning under Extreme Memory Constraints	Aug 4, 2020	Continual Learning	CodeCode Available	1
GDumb: A Simple Approach that Questions Our Progress in Continual Learning	Aug 1, 2020	class-incremental learningClass Incremental Learning	CodeCode Available	1
Bilevel Continual Learning	Jul 30, 2020	Bilevel OptimizationContinual Learning	CodeCode Available	1
La-MAML: Look-ahead Meta Learning for Continual Learning	Jul 27, 2020	Continual LearningMeta-Learning	CodeCode Available	1
ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation	Jul 23, 2020	Continual LearningSession-Based Recommendations	CodeCode Available	1
Drinking from a Firehose: Continual Learning with Web-scale Natural Language	Jul 18, 2020	Continual Learning	CodeCode Available	1
A Biologically Plausible Audio-Visual Integration Model for Continual Learning	Jul 17, 2020	Continual LearningLifelong learning	—Unverified	0
Neuromodulated Neural Architectures with Local Error Signals for Memory-Constrained Online Continual Learning	Jul 16, 2020	Bayesian OptimizationClass Incremental Learning	—Unverified	0
SpaceNet: Make Free Space For Continual Learning	Jul 15, 2020	class-incremental learningClass Incremental Learning	CodeCode Available	0
Lifelong Learning of Compositional Structures	Jul 15, 2020	Continual LearningLifelong learning	CodeCode Available	1
Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition, and Selective Transfer	Jul 14, 2020	Continual LearningLifelong learning	—Unverified	0
RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning	Jul 13, 2020	Continual LearningImage Captioning	CodeCode Available	1
Disentanglement of Color and Shape Representations for Continual Learning	Jul 13, 2020	Continual LearningDisentanglement	—Unverified	0
Batch-level Experience Replay with Review for Continual Learning	Jul 11, 2020	Continual Learning	CodeCode Available	1
Graph-Based Continual Learning	Jul 9, 2020	Continual Learning	—Unverified	0
Graph Neural Networks with Continual Learning for Fake News Detection from Social Media	Jul 7, 2020	Continual LearningFact Checking	CodeCode Available	1
Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature	Jul 7, 2020	Continual LearningExtractive Summarization	CodeCode Available	1
Continual Learning in Human Activity Recognition: an Empirical Analysis of Regularization	Jul 6, 2020	Activity RecognitionContinual Learning	—Unverified	0
FLUID: A Unified Evaluation Framework for Flexible Sequential Data	Jul 6, 2020	Continual LearningRepresentation Learning	CodeCode Available	1
Pseudo-Rehearsal for Continual Learning with Normalizing Flows	Jul 5, 2020	Continual Learning	CodeCode Available	0
Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes	Jul 1, 2020	Continual Learning	—Unverified	0
Continual Relation Learning via Episodic Memory Activation and Reconsolidation	Jul 1, 2020	Continual LearningRelation	—Unverified	0
Enabling Continual Learning with Differentiable Hebbian Plasticity	Jun 30, 2020	Continual LearningPermuted-MNIST	—Unverified	0
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning	Jun 27, 2020	Continual Learning	CodeCode Available	1
Storing Encoded Episodes as Concepts for Continual Learning	Jun 26, 2020	Continual LearningStyle Transfer	—Unverified	0
Bookworm continual learning: beyond zero-shot learning and continual learning	Jun 26, 2020	AttributeContinual Learning	—Unverified	0
Continual Learning from the Perspective of Compression	Jun 26, 2020	Continual Learning	—Unverified	0
Insights from the Future for Continual Learning	Jun 24, 2020	Class Incremental LearningContinual Learning	CodeCode Available	1
OvA-INN: Continual Learning with Invertible Neural Networks	Jun 24, 2020	Continual Learning	—Unverified	0
Automatic Recall Machines: Internal Replay, Continual Learning and the Brain	Jun 22, 2020	Continual Learning	CodeCode Available	1
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions	Jun 22, 2020	Continual Learning	—Unverified	0
Continual Learning in Recurrent Neural Networks	Jun 22, 2020	Continual Learning	CodeCode Available	1
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent	Jun 21, 2020	Continual LearningTransfer Learning	CodeCode Available	1
Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes	Jun 19, 2020	Continual LearningDecision Making	CodeCode Available	1
Semi-Discriminative Representation Loss for Online Continual Learning	Jun 19, 2020	Continual LearningDiversity	CodeCode Available	0
Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint	Jun 19, 2020	Continual Learning	—Unverified	0
Learning to Learn with Feedback and Local Plasticity	Jun 16, 2020	Continual LearningMeta-Learning	—Unverified	0
Chaotic Continual Learning	Jun 14, 2020	Continual Learning	—Unverified	0
Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition and Selective Transfer	Jun 12, 2020	Continual LearningLifelong learning	—Unverified	0
Combining Variational Continual Learning with FiLM Layers	Jun 12, 2020	Continual Learning	—Unverified	0
Efficient Adaptation for End-to-End Vision-Based Robotic Manipulation	Jun 12, 2020	Continual LearningRobotic Grasping	—Unverified	0
Multilayer Neuromodulated Architectures for Memory-Constrained Online Continual Learning	Jun 12, 2020	Bayesian Optimizationclass-incremental learning	—Unverified	0
Learning Causal Models Online	Jun 12, 2020	Continual Learning	CodeCode Available	1
CPR: Classifier-Projection Regularization for Continual Learning	Jun 12, 2020	Continual Learning	CodeCode Available	1
Move-to-Data: A new Continual Learning approach with Deep CNNs, Application for image-class recognition	Jun 12, 2020	Continual Learningimage-classification	—Unverified	0
Deep Reinforcement and InfoMax Learning	Jun 12, 2020	Continual Learning	CodeCode Available	0
Understanding the Role of Training Regimes in Continual Learning	Jun 12, 2020	Continual Learning	CodeCode Available	1

Show:10 25 50

← PrevPage 47 of 53Next →

All datasets ASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Multi-task Learning (MTL; Upper Bound)	F1 - macro	0.88	—	Unverified
2	CTR	F1 - macro	0.84	—	Unverified
3	B-CL	F1 - macro	0.81	—	Unverified
4	LAMOL	F1 - macro	0.81	—	Unverified
5	OWM	F1 - macro	0.79	—	Unverified
6	A-GEM	F1 - macro	0.78	—	Unverified
7	HAT	F1 - macro	0.78	—	Unverified
8	Independent Learning (ONE)	F1 - macro	0.78	—	Unverified
9	KAN	F1 - macro	0.77	—	Unverified
10	Naive Continual Learning (NCL)	F1 - macro	0.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NetTailor	decathlon discipline (Score)	3,744	—	Unverified
2	Depthwise Soft Sharing	decathlon discipline (Score)	3,507	—	Unverified
3	Parallel Res. adapt.	decathlon discipline (Score)	3,412	—	Unverified
4	Depthwise Sharing	decathlon discipline (Score)	3,234	—	Unverified
5	Series Res. adapt.	decathlon discipline (Score)	3,159	—	Unverified
6	Res. adapt. (large)	decathlon discipline (Score)	3,131	—	Unverified
7	DAN	decathlon discipline (Score)	2,851	—	Unverified
8	Piggyback	decathlon discipline (Score)	2,838	—	Unverified
9	Res. adapt. finetune all	decathlon discipline (Score)	2,643	—	Unverified
10	Res. adapt. decay	decathlon discipline (Score)	2,621	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	94.99	—	Unverified
2	ALTA	Average Accuracy	92.98	—	Unverified
3	kNN-CLIP	Average Accuracy	90.8	—	Unverified
4	RMN	Average Accuracy	81	—	Unverified
5	CPG	Average Accuracy	80.9	—	Unverified
6	CondConvContinual	Average Accuracy	77.4	—	Unverified
7	PAENet	Average Accuracy	77.1	—	Unverified
8	CPG-light	Average Accuracy	77	—	Unverified
9	PackNet	Average Accuracy	67.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	89.8	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.73	—	Unverified
3	ALTA-RN101	Average Accuracy	83.35	—	Unverified
4	ALTA-RN50	Average Accuracy	81.07	—	Unverified
5	SNCL	Average Accuracy	52.85	—	Unverified
6	DER [buzzega2020dark]	Average Accuracy	51.78	—	Unverified
7	ER[riemer2018learning]	Average Accuracy	48.64	—	Unverified
8	iCaRL [rebuffi2017icarl]	Average Accuracy	31.55	—	Unverified
9	A-GEM [chaudhry2018efficient]	Average Accuracy	25.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAT (CNN backbone)	Acc	0.76	—	Unverified
2	CAT (MLP backbone)	Acc	0.69	—	Unverified
3	EWC	Acc	0.65	—	Unverified
4	HyperNet	Acc	0.6	—	Unverified
5	PathNet	Acc	0.58	—	Unverified
6	HAT	Acc	0.57	—	Unverified
7	RPSNet	Acc	0.55	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.95	—	Unverified
2	HAT	F1 - macro	0.95	—	Unverified
3	CAT	F1 - macro	0.95	—	Unverified
4	B-CL	F1 - macro	0.95	—	Unverified
5	EWC	F1 - macro	0.92	—	Unverified
6	LAMOL	F1 - macro	0.46	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	84.26	—	Unverified
2	H$^{2}$	Accuracy	84.1	—	Unverified
3	CPG	Accuracy	83.59	—	Unverified
4	Piggyback	Accuracy	80.5	—	Unverified
5	PackNet	Accuracy	80.41	—	Unverified
6	ProgressiveNet	Accuracy	78.94	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CTR	F1 - macro	0.89	—	Unverified
2	CAT	F1 - macro	0.87	—	Unverified
3	HAT	F1 - macro	0.86	—	Unverified
4	KAN	F1 - macro	0.81	—	Unverified
5	B-CL	F1 - macro	0.77	—	Unverified
6	EWC	F1 - macro	0.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	97.16	—	Unverified
2	CPG	Accuracy	96.62	—	Unverified
3	H$^{2}$	Accuracy	94.9	—	Unverified
4	Piggyback	Accuracy	94.77	—	Unverified
5	ProgressiveNet	Accuracy	93.41	—	Unverified
6	PackNet	Accuracy	93.04	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Piggyback	Accuracy	76.16	—	Unverified
2	ProgressiveNet	Accuracy	76.16	—	Unverified
3	CondConvContinual	Accuracy	76.16	—	Unverified
4	CPG	Accuracy	75.81	—	Unverified
5	PackNet	Accuracy	75.71	—	Unverified
6	H$^{2}$	Accuracy	75.71	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	80.77	—	Unverified
2	CPG	Accuracy	80.33	—	Unverified
3	Piggyback	Accuracy	79.91	—	Unverified
4	ProgressiveNet	Accuracy	76.35	—	Unverified
5	H$^{2}$	Accuracy	76.2	—	Unverified
6	PackNet	Accuracy	76.17	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CPG	Accuracy	92.8	—	Unverified
2	CondConvContinual	Accuracy	92.61	—	Unverified
3	H$^{2}$	Accuracy	90.6	—	Unverified
4	Piggyback	Accuracy	89.62	—	Unverified
5	ProgressiveNet	Accuracy	89.21	—	Unverified
6	PackNet	Accuracy	86.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CondConvContinual	Accuracy	78.32	—	Unverified
2	CPG	Accuracy	77.15	—	Unverified
3	H$^{2}$	Accuracy	75.1	—	Unverified
4	ProgressiveNet	Accuracy	74.94	—	Unverified
5	Piggyback	Accuracy	71.33	—	Unverified
6	PackNet	Accuracy	69.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ALTA-ViTB/16	Average Accuracy	92.85	—	Unverified
2	ALTA-RN50x4	Average Accuracy	84.91	—	Unverified
3	RMN (Resnet)	Average Accuracy	84.9	—	Unverified
4	ALTA-RN101	Average Accuracy	84.77	—	Unverified
5	ALTA-RN50	Average Accuracy	83.87	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Accuracy	68.1	—	Unverified
2	CondConvContinual	Accuracy	61.32	—	Unverified
3	CCGN	Accuracy	35.24	—	Unverified
4	DGMw	Accuracy	17.82	—	Unverified
5	DGMa	Accuracy	15.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMN	Average Accuracy	97.99	—	Unverified
2	Model Zoo-Continual	Average Accuracy	97.71	—	Unverified
3	CODE-CL	Average Accuracy	96.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	62.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	93.32	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TEST	1:3 Accuracy	2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Average Accuracy	62.79	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	82.69	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	88.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	84.27	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	61.58	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	68.83	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAG-RMSProp	Accuracy	57.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	53.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MRM	Acc	78.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Model Zoo-Continual	Average Accuracy	99.66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CODE-CL	Average Accuracy	77.21	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	97.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	H$^{2}$	Top 1 Accuracy %	99.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	IBM	Accuracy	52.38	—	Unverified