[Feature][Module Name] Feature title Maybe combine with Ollama is the best way to run it as business level
Opened this issue · 0 comments
zencorn commented
Search before asking
- I had searched in the issues and found no similar feature requirement.
Description
Currently, in real-world scenarios, the Ollama-hosted modeling scheme is more robust and convenient, as the Ollama-hosted model responds better without GPUs. However, the current modeling scheme is very complex, prone to exceptions, and after startup, the response is much slower than the response of the running model in Ollama.
Use case
No response
Related issues
No response
Feature Priority
None
Are you willing to submit PR?
- Yes I am willing to submit a PR!