# Optimizing Canaries for Privacy Auditing with Metagradient Descent


## Canary Optimization 

```bash
cd metagradient
```

In order to setup the canary optimization experiments we need first to setup the environment:

```bash
conda env create -f environment.yml
conda activate metagrad
```

Once that's ready you can easily setup the output and data directory for the canaries in the 
```jaxlm/applications/optimize_canaries.py``` at lines ```508``` and ```509```, then start the optimization by running

```bash
sh scripts/run_opt.sh
```

## Auditing


```bash
cd dp-audit
```

Create conda environment

```bash
conda create -n dp-audit python=3.12
conda activate dp-audit
```

Install requirements

```bash
cd third_party/jax_privacy
pip install -e .

cd ../..
pip install -e .
```

To run and audit DP-SGD (from scratch), run the following (set `DEVICE=GPU_ID`)

```bash
./run/scripts/train.sh ${DEVICE}
./run/scripts/audit.sh
```

To run DP finetuning experiments, first download the zip file (found [here](https://drive.google.com/file/d/16QoncIC3ZRaf_9OYrdYjkzEN7X7G5Zsa/view?usp=sharing)) and unzip to `dp_audit/data/`. Then pretrain on CINIC-10 (without CIFAR-10 images).

```bash
./run/scripts/pretrain_cinic10_nondp.sh ${DEVICE}
```

Then run DP-SGD and audit

```bash
./run/scripts/train_ft_cinic10.sh ${DEVICE}
./run/scripts/audit_ft_cinic10.sh
```

# Acknowledgements
- One-run auditing code is adapted from code provided in https://arxiv.org/abs/2305.08846 and https://arxiv.org/abs/2410.22235
- DP-SGD code is adapted from the JAX Privacy library: https://github.com/google-deepmind/jax_privacy
- The CINIC-10 dataset was preprocessed using files provided in the original codebase: https://github.com/BayesWatch/cinic-10
