## Environment Setup
```bash
conda create -n hydra python=3.10
conda activate hydra
pip install -r requirements.txt
```
## CLIP

### Navigate to CLIP Directory

```bash
cd CLIP
```

### Download Data

```bash
python utils/download_data.py
```

**Note**: ImageNet training data must be downloaded manually. Get the ILSVRC2012 train split from https://www.image-net.org/download/

### Get Checkpoints

#### Gradient Pruning (24 Heads)

```bash
python get_pruned_ckpt.py \
  --heads_to_prune 24 \
  --strategy gradient \
  --out_dir checkpoints
```

#### Circuit-Based Pruning

```bash
python get_pruned_ckpt.py \
  --heads_to_prune <number_heads_to_prune> \
  --strategy predefined \
  --out_dir checkpoints \
  --circuit_path circuit/.../circuit.json \
  --prune_text_encoder  # Optional: include if circuit has pruned text encoder
```

**Note**: You can extract circuits yourself using `pruning/circuit_CLIP.py`

### Evaluation

#### Classification and OOD Detection

```bash
python testing/test.py --eval_ood
```

#### Temperature Scaling

Comment out `BayesVLM` in `testing/test.yaml` and run:

```bash
python testing/test.py --eval_temp_scaling
```

#### ViLU Evaluation

```bash
# For classification datasets
python testing/test_vilu.py --dataset_cls [dataset1 dataset2 ...]

# For OOD datasets
python testing/test_vilu.py --dataset_ood dataset_name
```

### Vit-B-16
- Code for pruning, merging (Hydra/GFC), and circuit extraction is available. Finetuned checkpoints will be released when the review process is over.
- ImageNet-1K zero-shot evaluation:
  - `python Vit-B-16/Imagenet1k/hydra_taylor_zero_shot.py`
  - `python Vit-B-16/Imagenet1k/hydra_circuit_zero_shot.py`

  Both scripts prune, merge, and evaluate Hydra on ID/OOD splits. The `circuits/` folder lists heads to prune (by score from the extraction script found in same folder).
- CIFAR100 mirrors the same structure, but torchvision ships no CIFAR100 ViT-B/16 weights; the code runs with randomly initialized ViTs unless you provide your own checkpoint. Its `circuits/` folder also contains heads (we extracted) to prune according to the task.

### BERT-base
- Pruning and evaluation code for BERT-base trained on SST-2 (ID and OOD) is available to test plus extracted circuits are made available too.
- Checkpoints are withheld to preserve anonymity. 
