Enable different initializations in LearnedElementwiseScaling-Layer
niels-leif-bracher opened this issue · 1 comments
niels-leif-bracher commented
Hey Lynton,
I think the current definition of the parameters in self.s disable different initialization and self.s will always be initialized with zeros everywhere:
This would allow different initialization:
self.s = nn.Parameter(np.log(init_scale) * torch.ones(1, *dims_in[0]))
psorrenson commented
Thanks for pointing out this bug! It's fixed now, see #118