[Codefuse开源轻训营] Support for Encoder-only models
Opened this issue · 0 comments
elvis-t9 commented
Add support for Encoder-only models (BERT-style architectures) in addition to the current Decoder-only LLM support. This enhancement will enable the conversion and fine-tuning of popular encoder-based models for embedding tasks, providing users with more architectural choices and leveraging the strengths of bidirectional attention for representation learning.