Continual Learning
Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.
Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review
Papers
Showing 101–125 of 2644 papers
All datasetsASC (19 tasks)visual domain decathlon (10 tasks)Cifar100 (20 tasks)Tiny-ImageNet (10tasks)F-CelebA (10 tasks)20Newsgroup (10 tasks)CUBS (Fine-grained 6 Tasks)DSC (10 tasks)Flowers (Fine-grained 6 Tasks)ImageNet (Fine-grained 6 Tasks)Sketch (Fine-grained 6 Tasks)Stanford Cars (Fine-grained 6 Tasks)
Benchmark Results
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | Multi-task Learning (MTL; Upper Bound) | F1 - macro | 0.88 | — | Unverified |
| 2 | CTR | F1 - macro | 0.84 | — | Unverified |
| 3 | B-CL | F1 - macro | 0.81 | — | Unverified |
| 4 | LAMOL | F1 - macro | 0.81 | — | Unverified |
| 5 | OWM | F1 - macro | 0.79 | — | Unverified |
| 6 | A-GEM | F1 - macro | 0.78 | — | Unverified |
| 7 | HAT | F1 - macro | 0.78 | — | Unverified |
| 8 | Independent Learning (ONE) | F1 - macro | 0.78 | — | Unverified |
| 9 | KAN | F1 - macro | 0.77 | — | Unverified |
| 10 | Naive Continual Learning (NCL) | F1 - macro | 0.77 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | NetTailor | decathlon discipline (Score) | 3,744 | — | Unverified |
| 2 | Depthwise Soft Sharing | decathlon discipline (Score) | 3,507 | — | Unverified |
| 3 | Parallel Res. adapt. | decathlon discipline (Score) | 3,412 | — | Unverified |
| 4 | Depthwise Sharing | decathlon discipline (Score) | 3,234 | — | Unverified |
| 5 | Series Res. adapt. | decathlon discipline (Score) | 3,159 | — | Unverified |
| 6 | Res. adapt. (large) | decathlon discipline (Score) | 3,131 | — | Unverified |
| 7 | DAN | decathlon discipline (Score) | 2,851 | — | Unverified |
| 8 | Piggyback | decathlon discipline (Score) | 2,838 | — | Unverified |
| 9 | Res. adapt. finetune all | decathlon discipline (Score) | 2,643 | — | Unverified |
| 10 | Res. adapt. decay | decathlon discipline (Score) | 2,621 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | Model Zoo-Continual | Average Accuracy | 94.99 | — | Unverified |
| 2 | ALTA | Average Accuracy | 92.98 | — | Unverified |
| 3 | kNN-CLIP | Average Accuracy | 90.8 | — | Unverified |
| 4 | RMN | Average Accuracy | 81 | — | Unverified |
| 5 | CPG | Average Accuracy | 80.9 | — | Unverified |
| 6 | CondConvContinual | Average Accuracy | 77.4 | — | Unverified |
| 7 | PAENet | Average Accuracy | 77.1 | — | Unverified |
| 8 | CPG-light | Average Accuracy | 77 | — | Unverified |
| 9 | PackNet | Average Accuracy | 67.5 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | ALTA-ViTB/16 | Average Accuracy | 89.8 | — | Unverified |
| 2 | ALTA-RN50x4 | Average Accuracy | 84.73 | — | Unverified |
| 3 | ALTA-RN101 | Average Accuracy | 83.35 | — | Unverified |
| 4 | ALTA-RN50 | Average Accuracy | 81.07 | — | Unverified |
| 5 | SNCL | Average Accuracy | 52.85 | — | Unverified |
| 6 | DER [buzzega2020dark] | Average Accuracy | 51.78 | — | Unverified |
| 7 | ER[riemer2018learning] | Average Accuracy | 48.64 | — | Unverified |
| 8 | iCaRL [rebuffi2017icarl] | Average Accuracy | 31.55 | — | Unverified |
| 9 | A-GEM [chaudhry2018efficient] | Average Accuracy | 25.33 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CAT (CNN backbone) | Acc | 0.76 | — | Unverified |
| 2 | CAT (MLP backbone) | Acc | 0.69 | — | Unverified |
| 3 | EWC | Acc | 0.65 | — | Unverified |
| 4 | HyperNet | Acc | 0.6 | — | Unverified |
| 5 | PathNet | Acc | 0.58 | — | Unverified |
| 6 | HAT | Acc | 0.57 | — | Unverified |
| 7 | RPSNet | Acc | 0.55 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CondConvContinual | Accuracy | 84.26 | — | Unverified |
| 2 | H$^{2}$ | Accuracy | 84.1 | — | Unverified |
| 3 | CPG | Accuracy | 83.59 | — | Unverified |
| 4 | Piggyback | Accuracy | 80.5 | — | Unverified |
| 5 | PackNet | Accuracy | 80.41 | — | Unverified |
| 6 | ProgressiveNet | Accuracy | 78.94 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CondConvContinual | Accuracy | 97.16 | — | Unverified |
| 2 | CPG | Accuracy | 96.62 | — | Unverified |
| 3 | H$^{2}$ | Accuracy | 94.9 | — | Unverified |
| 4 | Piggyback | Accuracy | 94.77 | — | Unverified |
| 5 | ProgressiveNet | Accuracy | 93.41 | — | Unverified |
| 6 | PackNet | Accuracy | 93.04 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | Piggyback | Accuracy | 76.16 | — | Unverified |
| 2 | ProgressiveNet | Accuracy | 76.16 | — | Unverified |
| 3 | CondConvContinual | Accuracy | 76.16 | — | Unverified |
| 4 | CPG | Accuracy | 75.81 | — | Unverified |
| 5 | PackNet | Accuracy | 75.71 | — | Unverified |
| 6 | H$^{2}$ | Accuracy | 75.71 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CondConvContinual | Accuracy | 80.77 | — | Unverified |
| 2 | CPG | Accuracy | 80.33 | — | Unverified |
| 3 | Piggyback | Accuracy | 79.91 | — | Unverified |
| 4 | ProgressiveNet | Accuracy | 76.35 | — | Unverified |
| 5 | H$^{2}$ | Accuracy | 76.2 | — | Unverified |
| 6 | PackNet | Accuracy | 76.17 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CPG | Accuracy | 92.8 | — | Unverified |
| 2 | CondConvContinual | Accuracy | 92.61 | — | Unverified |
| 3 | H$^{2}$ | Accuracy | 90.6 | — | Unverified |
| 4 | Piggyback | Accuracy | 89.62 | — | Unverified |
| 5 | ProgressiveNet | Accuracy | 89.21 | — | Unverified |
| 6 | PackNet | Accuracy | 86.11 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CondConvContinual | Accuracy | 78.32 | — | Unverified |
| 2 | CPG | Accuracy | 77.15 | — | Unverified |
| 3 | H$^{2}$ | Accuracy | 75.1 | — | Unverified |
| 4 | ProgressiveNet | Accuracy | 74.94 | — | Unverified |
| 5 | Piggyback | Accuracy | 71.33 | — | Unverified |
| 6 | PackNet | Accuracy | 69.4 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | ALTA-ViTB/16 | Average Accuracy | 92.85 | — | Unverified |
| 2 | ALTA-RN50x4 | Average Accuracy | 84.91 | — | Unverified |
| 3 | RMN (Resnet) | Average Accuracy | 84.9 | — | Unverified |
| 4 | ALTA-RN101 | Average Accuracy | 84.77 | — | Unverified |
| 5 | ALTA-RN50 | Average Accuracy | 83.87 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | RMN | Average Accuracy | 97.99 | — | Unverified |
| 2 | Model Zoo-Continual | Average Accuracy | 97.71 | — | Unverified |
| 3 | CODE-CL | Average Accuracy | 96.56 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | TAG-RMSProp | Accuracy | 62.59 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CODE-CL | Average Accuracy | 93.32 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | TEST | 1:3 Accuracy | 2 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | TAG-RMSProp | Average Accuracy | 62.79 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | IBM | Accuracy | 82.69 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | IBM | Accuracy | 88.15 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | Model Zoo-Continual | Average Accuracy | 84.27 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | TAG-RMSProp | Accuracy | 61.58 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CODE-CL | Average Accuracy | 68.83 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | TAG-RMSProp | Accuracy | 57.2 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | IBM | Accuracy | 53.9 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | MRM | Acc | 78.4 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | Model Zoo-Continual | Average Accuracy | 99.66 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CODE-CL | Average Accuracy | 77.21 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | H$^{2}$ | Top 1 Accuracy % | 97.3 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | H$^{2}$ | Top 1 Accuracy % | 99.9 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | IBM | Accuracy | 52.38 | — | Unverified |