Will there be gguf support?
babybirdprd opened this issue · 4 comments
Ollama, Litellm, lmstudio, etc
Yes, I will consider adding support for more inference engines. However, the current priority is to add support for multimodal models(this will lead to truly amazing features), and I need to prioritize getting this done first.
#10 #6 Hey guys, I have just completed support for the inference servers compatible with OpenAI API. Interested friends please get the latest version of the dev branch code and try it. Please note that config.json has been modified this time. It is best to delete config.json before running ailice.
For details, you need to read the "Models(or Services) compatible with OpenAI Interface" section in README.md.
Awesome! Will check it shortly.
Various inference services have been well supported, let's close the issue.