- Clone this repository and navigate to CLMoE folder
git clone XXXX
cd CL-MoE
- Install Package
conda create -n cpmoe python=3.10 -y
conda activate cpmoe
pip install --upgrade pip
pip install -e .
- Install additional packages for training cases
pip install -e ".[train]"
pip install flash-attn --no-build-isolation
We have prepared the scripts to train and evaluate the trained model in scripts/CLMoE/.
Run scripts/CLMoE/Train_NI/Train.sh for quick training for superNI task
Run scripts/CLMoE/Eval_NI/Eval_all_CL.sh for get continual learning result.