It would be good if pseudo code provided

[jimmieliu]

Hi I am interested of the MoV idea proposed in your paper, and I have implemented a variant myself. It still would be nice if you provide pseudo code about the "Router" in your paper Fig 2 right.

[ahmetustun]

Hi @jimmieliu,

Thanks for checking out our work. We used a simple linear layer for the router. You can find the implementation here:

https://github.com/for-ai/parameter-efficient-moe/blob/main/src/routing.py