UnsupportedOperationException on build() call of HuggingFaceChatModel

Question

UnsupportedOperationException on build() call of HuggingFaceChatModel

Opened this issue 2 months ago · 13 comments

Hi there,

I am encountering the following exception when trying to build a dev.langchain4j.model.huggingface.HuggingFaceChatModel

dev.langchain4j.model.huggingfacejava.lang.UnsupportedOperationException: Should not be called
	at io.quarkiverse.langchain4j.huggingface.QuarkusHuggingFaceClientFactory.create(QuarkusHuggingFaceClientFactory.java:32)
	at dev.langchain4j.model.huggingface.HuggingFaceChatModel.<init>(HuggingFaceChatModel.java:45)
	at dev.langchain4j.model.huggingface.HuggingFaceChatModel$Builder.build(HuggingFaceChatModel.java:153)

Code to reproduce:

var builder = HuggingFaceChatModel.builder().accessToken("<token>");
HuggingFaceChatModel chatModel = builder.build(); // Exception on calling build()

On another note:
Missing logRequests and logResponses methods on the HuggingFace builder (OpenAI, Anthropic, Gemini and Ollama builders have them defined)

geoand commented 2 months ago

I see

Answer 1 · 2024-05-20T13:58:37.000Z

Thanks for reporting.

This is definitely a bug, I'll have a look soon.

Answer 2 · 2024-05-20T14:07:18.000Z

You can however access use CDI to access ChatLanguageModel or use QuarkusHuggingFaceChatModel programmatically.

Answer 3 · 2024-05-20T14:22:46.000Z

I have tried it with QuarkusHuggingFaceModel, but it seems to require a url. Shouldn't there be any default for this, so that only the accessToken is needed?

java.lang.IllegalStateException: No URL specified. Cannot build a rest client without URL
	at io.quarkus.rest.client.reactive.runtime.RestClientBuilderImpl.build(RestClientBuilderImpl.java:338)
	at io.quarkus.rest.client.reactive.runtime.QuarkusRestClientBuilderImpl.build(QuarkusRestClientBuilderImpl.java:260)
	at io.quarkiverse.langchain4j.huggingface.QuarkusHuggingFaceClientFactory.create(QuarkusHuggingFaceClientFactory.java:48)
	at io.quarkiverse.langchain4j.huggingface.QuarkusHuggingFaceChatModel.<init>(QuarkusHuggingFaceChatModel.java:46)
	at io.quarkiverse.langchain4j.huggingface.QuarkusHuggingFaceChatModel$Builder.build(QuarkusHuggingFaceChatModel.java:194)

Answer 4 · 2024-05-20T14:32:59.000Z

The builder of QuarkusHuggingFaceChatModel seems to be missing the modelId setter which is available on the HuggingFaceChatModel builder

Answer 5 · 2024-05-20T14:36:01.000Z

#604 will provide a default URL

Answer 6 · 2024-05-20T14:36:16.000Z

The builder of QuarkusHuggingFaceChatModel seems to be missing the modelId setter which is available on the HuggingFaceChatModel builder

Right, it's part of the URL really.

Answer 7 · 2024-05-20T14:37:25.000Z

Which I should not is not ideal, but we'll have to take a closer look at what we do with HuggingFace at some point in the future.

Answer 8 · 2024-05-20T14:43:18.000Z

As I can see, the model is part of the url, however, from a library usage point of view it would be great if I could just hand over the modelId, and the url will be constructed for me.
I could of course also do it in our implementation code, but solving this within the library would be better as it would be more consistent with HuggingFaceChatModel which has modelId setter implemented

Answer 9 · 2024-05-20T14:45:43.000Z

I agree, we need to redo this as it is not optimal

Answer 10 · 2024-05-20T14:56:43.000Z

That said, if you use the CDI bean, you need to deal with creating the LangChain4j models at all 😀

Answer 11 · 2024-05-20T15:16:56.000Z

I need it for being able to configure it at runtime, so CDI bean will not be ideal as I would like to be able to create multiple ones with different parameters (potentially for chaining them).

@ApplicationScoped
public class HuggingFaceLanguageModelBuilder implements ILanguageModelBuilder {
    private static final String KEY_ACCESS_TOKEN = "accessToken";
    private static final String KEY_MODEL_ID = "modelId";
    private static final String KEY_TEMPERATURE = "temperature";
    private static final String KEY_TIMEOUT = "timeout";

    @Override
    public ChatLanguageModel build(Map<String, String> parameters) {
        var builder = HuggingFaceChatModel.builder();
        if (!isNullOrEmpty(parameters.get(KEY_ACCESS_TOKEN))) {
            builder.accessToken(parameters.get(KEY_ACCESS_TOKEN));
        }
        
        if (!isNullOrEmpty(parameters.get(KEY_MODEL_ID))) {
            builder.modelId(parameters.get(KEY_MODEL_ID));
        }

        if (!isNullOrEmpty(parameters.get(KEY_TIMEOUT))) {
            builder.timeout(Duration.ofMillis(Long.parseLong(parameters.get(KEY_TIMEOUT))));
        }

        if (!isNullOrEmpty(parameters.get(KEY_TEMPERATURE))) {
            builder.temperature(Double.parseDouble(parameters.get(KEY_TEMPERATURE)));
        }

        return builder.build();
    }
}

Answer 12 · 2024-05-20T15:32:14.000Z

I'll reopen since we need to improve here