princeton-nlp/LLM-Shearing

duplicate mean values during mask initialization

Closed this issue · 2 comments

czhang99 commented

Observed duplicated initializations when initializing the mean value for masking variables. Is one of them preferred over the other?

https://github.com/princeton-nlp/LLM-Shearing/blob/3560a877e2833c3da393923be0bd6753b6ef1c6d/llmshearing/models/l0_module.py#L45-L46C17

xiamengzhou commented

Hii! Check out here issue #3 for an details answer!

  • 🚀1
czhang99 commented

thanks for the reference issue ticket. well-explained in #3. closing the issue.

  • ❤️1