quarkiverse/quarkus-langchain4j

(question) How to use image generation/vision ?

Closed this issue · 6 comments

sebastienblanc commented

Hi there !

I want to start playing with image generation/vision but I couldn't find any documentation. I found this which helps a bit : https://github.com/quarkiverse/quarkus-langchain4j/blob/main/integration-tests/openai/src/main/java/org/acme/example/openai/images/ImageResource.java

I tried to find also how it is implemented for the devUI but got a bit lost in the codebase 😅

Can I use @RegisterAIService for an image generation endpoint ?

jmartisk commented

AI services aren't supported for image content right now AFAIK.

There are two types of models that can work with images:

In the Dev UI, we have this 'Image' page that appears if you have the OpenAI extension and uses an ImageModel underneath.
The Chat page in the Dev UI doesn't yet support images, but I'm working on it.

sebastienblanc commented

@jmartisk Thanks a lot for the detailed (and super fast) reply !

sebastienblanc commented

@jmartisk And could you point me where the devUI image page is implemented, I'm interested how you pass the different options like resolution, model etc ?

jmartisk commented

You need an application that uses OpenAI (for example, the samples/chatbot app from this repo will do) and make sure that you look under the 'OpenAI' card, not the general one, because right now the support is specific to OpenAI. The URL will be like http://localhost:8080/q/dev-ui/io.quarkiverse.langchain4j.quarkus-langchain4j-openai/images