# Dtrpl

A project on Discrete Trust Region Policy Layers.

This project is a lightweight implementation of a discrete trust region layer for downstream optimization. 
It includes a cartpole environment and a simple PPO agent as examples.

# Getting Started

## Installation

### Core Package Only

Clone the repository and install the core package in editable mode:

```bash
pip install -e .
```

### With Cartpole Example Dependencies

To run the cartpole examples, install with optional dependencies:

```bash
pip install -e ".[cartpole]"
```

Or install the core package first and then add dependencies manually:

```bash
pip install -e .
pip install gymnasium torch numpy tyro tensorboard
```

## Running Examples

### Cartpole Example Scripts

Run the DTRPL cartpole example:
```bash
python experiments/cartpole_example/dtrpl.py
```

Run the PPO cartpole example:
```bash
python experiments/cartpole_example/ppo.py
```

Both scripts support command line arguments. Use `--help` to see available options:
```bash
python experiments/cartpole_example/dtrpl.py --help
```

# Setting up the project

## Mamba
This project uses [mamba](https://github.com/conda-forge/miniforge)/[conda](https://docs.conda.io/en/latest/) and pip for handling packages and dependencies.
To install mamba on Unix, use either of:

```
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh
```

For Windows please see the documentation in the link above or use (not recommended).
```
conda install -c conda-forge mamba
```

### Installs

