This codebase was built using [PyCIL](https://github.com/LAMDA-CL/PyCIL) using PyTorch. We thank the authors of PyCIL for their amazing continual learning codebase.


## Introduction

This codebase will reproduce the results of the paper, Namely its class-IL experiments.


## Methods Tested
-  `Replay`: Baseline method with exemplar replay.
-  `iCaRL`: Incremental Classifier and Representation Learning. CVPR2017 [[paper](https://arxiv.org/abs/1611.07725)]
-  `BiC`: Large Scale Incremental Learning. CVPR2019 [[paper](https://arxiv.org/abs/1905.13260)]
-  `WA`: Maintaining Discrimination and Fairness in Class Incremental Learning. CVPR2020 [[paper](https://arxiv.org/abs/1911.07053)]
### Expansion-based Methods
-  `DER`: DER: Dynamically Expandable Representation for Class Incremental Learning. CVPR2021 [[paper](https://arxiv.org/abs/2103.16788)]
-  `FOSTER`: Feature Boosting and Compression for Class-incremental Learning. ECCV 2022 [[paper](https://arxiv.org/abs/2204.04662)]
-  `MEMO`: A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning. ICLR 2023 Spotlight [[paper](https://openreview.net/forum?id=S07feAlQHgM)]
- `Weight Space Consolidation (Ours)`: The proposed method.

## How To Use

### Data
1. CIFAR-100: The CIFAR-100 dataset will be automatically downloaded when the CIFAR-100 experiments are run.
2. ImageNet-100: ImageNet-100 can be easily obtained in the web. In our case, we used [Kaggle](https://www.kaggle.com/datasets/ambityga/imagenet100)
- If you face an error in loading the data into the loader, please check the folder locations. Also have a look at the data pre-processing code for imagenet, which will automatically split the data for you (./notebooks/imagenet_data_preparation.ipynb)

A. Please use the ./notebooks/cifar100_visualization.ipynb for visualization of the results.


### Dependencies (same as PyCIL)

1. [torch 1.81](https://github.com/pytorch/pytorch)
2. [torchvision 0.6.0](https://github.com/pytorch/vision)
3. [tqdm](https://github.com/tqdm/tqdm)
4. [numpy](https://github.com/numpy/numpy)
5. [scipy](https://github.com/scipy/scipy)
6. [quadprog](https://github.com/quadprog/quadprog)
7. [POT](https://github.com/PythonOT/POT)

### Run experiment

1. To reproduce the CIFAR-100 experiments:
```bash
sh run_cifar.sh
``` 

2. To reproduce the ImageNet-100 experiments:
```bash
sh run_imagenet.sh
``` 


## Acknowledgments

We follow the practice of the original repository [PyCIL](https://github.com/LAMDA-CL/PyCIL) and thank the following repos for providing helpful components in our work.

- [Continual-Learning-Reproduce](https://github.com/zhchuu/continual-learning-reproduce)
- [GEM](https://github.com/hursung1/GradientEpisodicMemory)
- [FACIL](https://github.com/mmasana/FACIL)
- [PyCIL](https://github.com/LAMDA-CL/PyCIL)
- [TRACE](https://github.com/BeyonderXX/TRACE)




