Orthogonal initilizer does not support MLP?
rezunli96 opened this issue · 1 comments
rezunli96 commented
Hi, one question about the orthogonal initilizer, so it turns out if we initialize an MLP net with it, it will have the following error:
But this contradicts to my understanding: since a linear layer can be represented as some kind of matrix, why orthogonal initilizer cannot work in this case?
rezunli96 commented
Sorry, turns out to be a bug on my side. Thanks.