Illyasville/ExpertTokenRouting

Expert Model Backbone

Opened this issue · 0 comments

Great work!
Just to make sure that all the 'Expert Models' are finetuned (SFT) from the QWen-7b (same as the meta LLM), right?
Thank you.