This repo is created for my purpose to understand how to merge and create mixture of expert (MoE) with mergkit.
More info can be found at: https://goddard.blog/posts/clown-moe/#moe-gates-without-training
This repo is created for my purpose to understand how to merge and create mixture of expert (MoE) with mergkit.
More info can be found at: https://goddard.blog/posts/clown-moe/#moe-gates-without-training