# Multimodal Contractive Diffusion Policies (CDPs) in Real World

Real world deployment code for CDPs.

The repository trains CDPs on the following real world tasks.

![Real World CDP Tasks](media/cdp_tasks.png)
---

## Setup

### 1. Environment Setup

Create and activate a Conda or Mamba environment:

```bash
# Create Python 3.9 environment
conda create -n cdp-rw python=3.9

# Activate the environment
conda activate cdp-rw
```

### 2. Install Clean Diffuser

This project relies on the [Clean Diffuser](https://github.com/CleanDiffuserTeam/CleanDiffuser) implementation for diffusion policies.

```bash
# (Optional) Create a directory for libraries
mkdir libs && cd libs

# Clone the Clean Diffuser repository
git clone https://github.com/CleanDiffuserTeam/CleanDiffuser.git
cd CleanDiffuser

# Install in editable mode
pip install -e .
```

---

## Pipeline

The trained policy denoises noisy action samples into executable control commands. These actions are sent to the Polymetis middleware, which uses Torch JIT for high-frequency communication with the Franka control interface. The interface executes gripper and end-effector commands while streaming back observations—agent-view images, depth images, and proprioception—that are fed into the policy for closed-loop continuous control.

![CDP Pipeline in Real World](media/cdp_deploy.png)

---

## Train

### 1. Download the data, or collect your own

To train, make sure the datasets are downloaded and placed under the data folder according to the config file paths.

To download the datasets, check: [contractive-diffusion.github.io](contractive-diffusion.github.io).

### 2. Run the training scripts

Run the main training scripts, [train_cdp_realworld](train_cdp_realworld.py) or [train_dp_realworld](train_dp_realworld.py) to train CDP, DBC, or DP on the real worl imitation learning tasks.

### 3. Monitor the validation loss

Since direct evaluation in real world is time-consuming, make sure you monitor the validation loss and see a meaningful decrease before deployment.

---
## Credits

This repository is built on [CleanDiffuser](https://github.com/CleanDiffuserTeam/CleanDiffuser). We use the tuned hyperparameters and diffusion and conditioning backbones provided by the repository.

The datasets comes from D4RL and Robomimic projects addressed earlier.

---

## Hydra Configurations

All experiments are managed via [Hydra](https://hydra.cc/). Configuration files can be found in the [configs](configs) directory.

---

## Authors

Anonymous

---

## Citation

Anonymous

---