# DexNDM: CLOSING THE REALITY GAP FOR Dexterous IN-HAND ROTATION VIA Joint-wise NEURAL DYNAMICS Model

[Website](https://projectwebsitex.github.io/neudyn-reorientation/) | [Video](https://projectwebsitex.github.io/neudyn-reorientation/static/videos_lowres/demo_video_8(1).mp4)



## Overview

Code for DexNDM. 

This is composed of two codebases. 

- `RL`: Oracle polciy training and deployment. 
- `IsaacGymEnvs2`: BC for real-world deployable generalist, neural dynamics model training, residual policy training. 


## Environment Setup

Please refer to each codebase's `README.md` for environment setup. 


## Key Implementations

### Category-Specific Specialist Training via RL

Please check `RL/hora/tasks/leap_hand_hora.py` for details. 

For the observation space design, please refer the `compute_observations` function of the task class. 

For reward computation, please refer to `compute_reward` functon for detailed implementations. 


### Generalist Training via BC

Please check `IsaacGymEnvs2/isaacgymenvs/ddim/runners/diffusion_controlseq.py` for details. The function that we call to launch the training is `train_invdyn_ddp`. 



Please check `IsaacGymEnvs2/isaacgymenvs/ddim/models/diffusion_controlseq.py` for the BC model implementation. The model we use is `ModelInvDyn`. 


### Joint-Wise Neural Dynamics Model

Please check `IsaacGymEnvs2/isaacgymenvs/ddim/runners/diffusion_controlseq.py` for details. The function that we call to launch the training is `train_world_model_ddp`. 



Please check `IsaacGymEnvs2/isaacgymenvs/ddim/models/diffusion_controlseq.py` for the BC model implementation. The model we use is `WorldModel`. 



### Residual Policy Training

Please check `IsaacGymEnvs2/isaacgymenvs/ddim/runners/diffusion_controlseq.py` for details. The function that we call to launch the training is `train_world_model_delta_actions_ddp`. 



Please check `IsaacGymEnvs2/isaacgymenvs/ddim/models/diffusion_controlseq.py` for the BC model implementation. The model we use is `WorldModelDeltaActions`. 



## Usage

Please refer to `README.md` for each codebase for details. 


## Ack

The overall codebase is built upon [hora](https://github.com/HaozhiQi/hora), [IsaacGymEnvs](https://github.com/isaac-sim/IsaacGymEnvs), and [ddim](https://github.com/ermongroup/ddim). We thank authors for their great projects. 


