# AutoQD: Automatic Behavioral Diversity for Quality-Diversity Optimization

This repository contains code for the AutoQD algorithm, which automatically discovers behavioral descriptors to use for Quality-Diversity optimization in continuous control tasks.

## Requirements

- Python 3.10
- Dependencies: Install using `pip install -r requirements.txt`

## Setup
```bash
# Clone the repo
git clone https://github.com/anonymous/autoqd.git
cd autoqd

# Create a virtual environment
python -m venv env
source env/bin/activate  # On Windows: env\Scripts\activate

# Install dependencies
pip install -r requirements.txt
```

## Running Experiments
### Training
You can run the core algorithms using the scripts in the `scripts/` directory:
```bash
# Run AutoQD on all environments
./scripts/auto_qd.sh

# Run baselines (AURORA, LSTM-AURORA, Regular QD, SMERL)
./scripts/aurora.sh
./scripts/lstm_aurora.sh
./scripts/regular_qd.sh
./scripts/smerl.sh

# Run ablation studies
./scripts/ablations.sh
```
The scripts use seed 42 by default, but you can specify a different seed:
```bash
./scripts/auto_qd.sh 123
```
You can also run individual experiments with specific configurations:
```bash
# Run AutoQD on the BipedalWalker environment with seed 42 and 2d measures (descriptors)
python -m src.main algorithm=auto_qd env=bipedal_walker seed=42 algorithm.measures_dim=2

# Disable wandb logging
python -m src.main algorithm=auto_qd env=bipedal_walker logging.wandb=false
```

### Evaluation
To evaluate a single experiment:
```bash
# Evaluate a specific experiment
python -m src.evaluation_suite.eval_single outputs/auto_qd_bipedal_walker_0411_1658
```
To evaluate all experiments in the outputs directory:
```bash
# Evaluate all experiments in the outputs directory
./eval_script.sh
```
For multi-seed evaluation and aggregation:
```bash
# Assuming you have results in 1_outputs/, 2_outputs/, and 3_outputs/
python -m src.evaluation_suite.multi_seed_eval
```
**Note:** When evaluating all experiments, the code expects results from DvD-ES to be provided in a JSON file (dvd_logs.json) in the outputs directory. DvD-ES needs to be trained separately using the authors' implementation (https://github.com/jparkerholder/DvD_ES). A similar evaluation script to eval_single.py can be used to evaluate DvD-ES policies.
**Note 2:**
Before running certain evaluations, you may need to compute gamma values:
```bash
python -m src.evaluation_suite.compute_gammas
```
### Visualization
To visualize policies from a trained archive:
```bash
# Visualize policies from a checkpoint
python -m src.viz checkpoint_path=outputs/auto_qd_bipedal_walker_0411_1658/checkpoints/final.pkl env_id=BipedalWalker-v3
```
### Analyzing Adaptation Performance
To evaluate how policies adapt to modified environments:
```bash
python -m src.evaluation_suite.adaptation
```
Important Note: The adaptation.py script has hardcoded directory references. If you want to run this script with your own trained models, you'll need to modify the following lines to point to your output directories:
```python
# In src/evaluation_suite/adaptation.py, modify:
for dir, algo_name in [
    ("auto_qd_bipedal_walker_0411_1658", "auto_qd"),
    ("aurora_bipedal_walker_0411_1233", "aurora"),
    ("lstm_aurora_bipedal_walker_0412_2233", "lstm_aurora"),
    ("regular_qd_bipedal_walker_0413_0554", "regular_qd"),
    ("smerl_bipedal_walker_0415_1044", "smerl"),
]
```

## Project Structure

- `src/`: Source code for the project
  - `algorithms/`: Implementation of AutoQD and baseline algorithms
  - `embeddings/`: Embedding methods for state/action encoding
  - `measure_maps/`: Methods for mapping embeddings to measure space
  - `qd/`: Quality-Diversity optimization components
  - `evaluation_suite/`: Scripts for evaluating algorithm performance
- `conf/`: Configuration files using Hydra
- `scripts/`: Shell scripts for running experiments


## Citation
If you use this code in your research, please cite our paper:
[Citation information will be added after the paper is published]
