Can we change self.blocking in dmoe.py from 128 to 64?

[seanM29]

I use megablocks to implement a fine-granded moe, the ffn_hidden_size is divisible by 64, but is not divisible by 128, can we change it to 64? Thanks a lot

[mvpatel2000]

@tgale96 what are the performance implications for block size selection?

For now, to unblock I'd recommend forking or overriding the variable... but I'm not as sure here.

[tgale96]

I recommend using the grouped code path rather than changing the block size. That is untested and likely to result in poor performance.