Liuhong99/Sophia

Sophia with multitensor apply / FusedSophia

Closed this issue · 1 comments

skyshine102 commented

Wondering if the authors plan to augment existing sophia algorithm with multitensor/ fused operations. Thanks!

Liuhong99 commented

I think the multi-tensor version can be easily implemented. I'll try to do that soon. The fused version is a bit more complicated though. I'll try to do that in several weeks,

  • ❤️1