Anthropic Claude support
timonv opened this issue · 2 comments
Sonnet 3.5 has stellar reviews. If we can get it to SimplePrompt and potentially Embed, it should just work (tm). The Rust ecosystem lacks a mature client, which makes it a bit more work to implement.
One could use an existing crate/code for generic LLM interfacing, see for example:
https://crates.io/crates/genai
https://crates.io/crates/llmclient
https://crates.io/crates/allms
According to Anthropic, they don't provide embeddings and refer to VoyageAI
Checking the MTEB leaderboard, Stella-EN-1.5B has an unbelievable score compared to VoyageAI, Cohere or ElasticSearch elser-v2:
https://huggingface.co/spaces/mteb/leaderboard
https://huggingface.co/dunzhang/stella_en_1.5B_v5
My concern with existing libraries for Anthropic is that they feel a bit immature, lacking api coverage and important features like automatic backoff, error handling, etc (all of this is doable with reqwest). In comparison, async_openai has all of that and more, and is generic over the configuration, so adding groq support was trivial.
bge-small is the default currently when using fastembed, and has large available as well. It's generally not that hard to add new models to the library unless they require very specific math / operations that need to be translated to Rust (ndarrray is great though).
That said if you feel a library is mature enough, or want to implement it, feel free to suggest it and we can take a look together.