# DexNDM - Generalist Training via BC, Neural Dynamics Model Training, Residual Policy Training




[Website](https://projectwebsitex.github.io/neudyn-reorientation/) | [Video](https://projectwebsitex.github.io/neudyn-reorientation/static/videos_lowres/demo_video_8(1).mp4)




## Environment Setup



The project is developed using Python 3.8. We use PyTorch 2.4.0 with cuda 12.1. 

Please refer to `environment.yaml` for remaining environment setup. 




## BC


After rolling out policies, save each tested environment's result in an independent numpy file, and generate a dictionary in the format `{fn_idx: fn_path}`. 


```bash
cd isaacgymenvs/ddim
python -m torch.distributed.run --nnodes=1 --nproc_per_node=8 main.py --config controlseq.yml --exp {EXP_NAME}  --doc cs --ni --model_type=invdyn --data_type=hora --data_path={DATA_DICT_PATH}  --future_ref_dim=3 --future_type='obj_motion' --obj_motion_format='rot_axis' --history_obs_dim=32 --invdyn_obs_type='hand_qpos_qtars' --use_stochastic_dataset --model_arch=resmlp --res_blocks=5 --history_length=10 --batch_size=64  --train_ddp 
```




## Dynamics Model Training

Dynamics model training via real world data. 


```bash
cd isaacgymenvs/ddim
python -m torch.distributed.run --nnodes=1 --nproc_per_node=8  --master_port=3811   main.py --config controlseq.yml --exp {EXP_NAME}  --doc cs --ni --model_type=invdyn --data_type=hora --data_path={DATA_DICT_PATH} --future_ref_dim=3 --future_type='obj_motion' --history_obs_dim=32 --invdyn_obs_type='hand_qpos_qtars' --use_stochastic_dataset --model_arch=resmlp --res_blocks=5 --history_length=10 --batch_size=128 --mask_out_obj_motion --world_model --finger_idx=-1 --joint_idx=-1  --wm_history_length=10 --n_epochs=100  --multi_joint_single_wm  --train_ddp  --load_pretrained_wm_ckpt={PRETRAINED_WM_CKPT}   --logging_step_interval=1000  
```


## Residual Policy Training

Same data used for BC training. 

```bash
cd isaacgymenvs/ddim
python -m torch.distributed.run --nnodes=1 --nproc_per_node=8  --master_port=12388  main.py --config controlseq.yml --exp {EXP_NAME} --doc cs --ni --model_type=invdyn --data_type=hora --data_path={DATA_DICT_PATH} --future_ref_dim=3 --future_type='obj_motion' --history_obs_dim=32 --invdyn_obs_type='hand_qpos_qtars' --use_stochastic_dataset --model_arch=resmlp --res_blocks=5 --history_length=10 --batch_size=128 --mask_out_obj_motion --train_delta_action_model  --joint_idx=-1 --finger_idx=-1 --wm_history_length=10 --hist_context_length=0  --train_ddp --stack_wm_history  --multi_joint_single_wm --delta_action_scale=0.041667
```


## Ack

The repo is built upon [IsaacGymEnvs](https://github.com/isaac-sim/IsaacGymEnvs) and [ddim](https://github.com/ermongroup/ddim). We thank authors for their great projects. 

