# DR-SAC

This repository contains a Distributionally Robust Soft Actor-Critic (DR-SAC) implementation.

## Setup

Install required dependencies:
```
conda config --set channel_priority flexible 
conda env create -f requirements.yaml -n DR-SAC
conda activate DR-SAC
```

## Running the Code

### Basic Training

Train with default configuration (Pendulum environment):

```
python train_sac.py data_path=... # please replace with absolute path of dataset
```

### Changing Environments

Train on a different environment. Due to the size limit of supplementary materials, we only provide dataset in Pendulum environment. All datasets used in our experiments will be released publicly later. You are also welcome to use your own datasets.

```
python train_sac.py env/sac=pendulum data_path=...
python train_sac.py env/sac=cartpole data_path=...
python train_sac.py env/sac=lunarlander data_path=...
python train_sac.py env/sac=halfcheetah data_path=...
python train_sac.py env/sac=reacher data_path=...
```

### Using Robust Policy

Train with robust policy:

```
python train_sac.py robust=true delta=0.1 data_path=...
```

### Changing Hyperparameters

Change any hyperparameter directly from command line, for other configuration please check the config files:

```
python train_sac.py batch_size=512 a_lr=0.001 hid_dim=128
```

### Evaluation Mode

Run in evaluation mode indicating your trained model:

```
python train_sac.py eval_model=true load_model=true load_path=... # please replace with absolute path of trained model
```
