# Pretrain

## 版本

```shell
mkdir xpuyu_pretrain & cd xpuyu_pretrain
git clone https://github.com/pppppM/xtuner.git -b ppo+cache
git clone git@github.com:pjlab-sys4nlp/InternTrain.git -b official/official_kepler_moe_2024111
git clone ssh://git@gitlab.pjlab.org.cn:1122/openmmlab/bigmodel/xpuyu.git -b gjf/ppo

cd xpuyu
# 如果相对路径软链接失败，使用绝对路径代替
ln -s ../xtuner/xtuner xtuner
ln -s ../InternTrain/internlm internlm
ln -s ../InternTrain/configs configs
```

```shell
cd tools
bash shell/pretrain_aliyun_dsw.sh
```
