codefuse-ai/CodeFuse-Embeddings

[Codefuse开源轻训营] Support for Encoder-only models

Opened this issue · 0 comments

Add support for Encoder-only models (BERT-style architectures) in addition to the current Decoder-only LLM support. This enhancement will enable the conversion and fine-tuning of popular encoder-based models for embedding tasks, providing users with more architectural choices and leveraging the strengths of bidirectional attention for representation learning.