Have the authors tried fine-tuning the parameters of the LayerNorm layer with it turned on? If so, what were the results?
Changwei-Ouyang opened this issue · 1 comments
Changwei-Ouyang commented
Have the authors tried fine-tuning the parameters of the LayerNorm layer with it turned on? If so, what were the results?
taoyang1122 commented
Thanks for your interest. I believe we tried that, but I don't remember the exact results. You could easily try it by simply modifying the codes here .