Add support for Ollama multimodal
suavelizard opened this issue · 6 comments
Draft PR for multimodal Ollama: ollama/ollama#1216
This would add Ollama as a supported provider alongside structure generation: https://github.com/lgrammel/modelfusion#generate-structure
@suavelizard thanks for letting me know. I plan to add Ollama multi-modal, once it is available in the Ollama API (similar to Llama.cpp)
Can you add more examples of using ollama with tools that make it be able to do more like for example aa ollama ts file with a search tool or to get more creative a ollama model with a stable diffusion tool to generate imags from query etc. If u get what im trying to say. Best i can explain it lol
Can you add more examples of using ollama with tools that make it be able to do more like for example aa ollama ts file with a search tool or to get more creative a ollama model with a stable diffusion tool to generate imags from query etc. If u get what im trying to say. Best i can explain it lol
Yeah an image generation tool has been on my list. There are existing tools and you can also define your own. Feel free to contribute working examples!
Multi-modal models are now available on the ollama main branch, there is also a pre-release https://github.com/jmorganca/ollama/releases/tag/v0.1.15. Would be great to have Modelfusion support it.
Basic multi-modal support was added w/ ModelFusion v0.97: https://github.com/lgrammel/modelfusion/releases/tag/v0.97.0
Thank you 😄