why not try LeCun normal initializer
Closed this issue · 0 comments
bliu6 commented
as I know, selu activation may require LeCun normal initializer whose stddev = sqrt(1 / n). however, I found netG_init_weight = 1e-3 in code, which means stddev = sqrt(1e-3 / n) . Maybe you have tried LeCun normal initializer and preferred current init weight for better performance ?