TACJu/TransFG

[minor error]The linear layer self.out of Attention in file modeling

Serayah1376 opened this issue · 0 comments

May be there is something wrong with the first argument of the linear layer self.out (line 78: self.out = Linear(config.hidden_size, config.hidden_size)) of function Attention in the modeling file and should be changed to self.out = Linear(self.all_head_size, config.hidden_size). Because in some cases, config.hidden_size and self.all_head_size might not be equal.