About the understanding of geometric affine module

[Shi-Qi-Li]

Hi, @ma-xu
Thank you for sharing the code. I have a small question about the geometric affine module, based on my understanding, the geometric affine module can be considered equivalent to a LayerNorm operation when using the center mode normalize, I'm not sure if my understanding is correct.

[ma-xu]

@Shi-Qi-Li Probably not, you can double-check the mean operation over which dimensions. If interested, feel free to test with a layernorm and report the results, that would be appreciated. 😊

[Shi-Qi-Li]

@Shi-Qi-Li Probably not, you can double-check the mean operation over which dimensions. If interested, feel free to test with a layernorm and report the results, that would be appreciated. 😊

Thanks for your reply! I checked the code again and the two operations are indeed different, the mean operation in affine module is on k dim.