NVIDIA/Megatron-LM

When can we have a the MOE checkpoint convert script.

Opened this issue · 2 comments

shamanez commented

As mentioned here, having a proper MOE/Mixtral checkpoint converter script will help us to fine-tune Mixtral.

yqli2420 commented

+1

hwdef commented

I also strongly need this tool

#756 (comment)