Possible to remove matrix multiplication in SANet?
ebroglio opened this issue · 1 comments
ebroglio commented
Is it possible somehow to get rid of the two matrix multiplications in SANet.cs?
This is one reason it is really slow.
GlebSBrykin commented
Hello, ebroglio!
Matrix multiplication is part of any attention module, whether it is self-attention or style-attention. Regarding the speed of work, the problem is most likely directly in the implementation of matrix multiplication. I will check this code and make edits if necessary.
Sincerely,
Gleb Brykin.