berlino/gated_linear_attention

A Full LM class

Closed this issue · 2 comments

Cranial-XIX commented

Hi,

Thanks for this great work! I wonder if you could provide a wrapper for a full language model class, like in Mamba and RetNet they have MambaLMHeadModel and RetNetDecoder. Thanks a lot!

berlino commented

Thanks for your interests!

Yes, we plan to add the model file soon.

berlino commented

I've added a model class in gla_model.py, let me if there is any further questions

  • 👍1