quarkiverse/quarkus-langchain4j

Azure endpoint pattern domain name

Closed this issue · 7 comments

Azure provides Open AI not only in the *.openai.azure.com domain but also in *.azure-api.net.
Is it possible to change the pattern?
From

static final String AZURE_ENDPOINT_URL_PATTERN = "https://%s.openai.azure.com/openai/deployments/%s";

to e.g.

static final String AZURE_ENDPOINT_URL_PATTERN = "https://%s/openai/deployments/%s";

Thanks for reporting.

Is there any benefit from using one or the other?

The domain is used to separate data processed by public AI. This is used by commercial companies and people who do not want their data processed by public artificial intelligence (possible leak). The issue of security of corporate or private data.

Thanks.

cc @csotiriou

I find no reason why this would not change, I don't have access to models of that domain currently, but it seems that @duchamk has access to something like this.

I would propose https://%s.openai.azure.com/openai/deployments/%s to change to https://%s.%s/openai/deployments/%s, where the second argument would be the domain name, which would be configurable, and would be default be openai.azure.com, what do you think, @geoand ?

Sounds reasonable to me.

@duchamk would you like to contribute this change?

@geoand @duchamk I just found that quarkus-langchain4j has the endpoint configuration, which allows the entire model endpoint to be set, and be overriden for every different model.

Perhaps @duchamk can use this property? And if that's the case, do we still need to make the change in the configuration (honest question, I would personally be open to all cases)

Yes, there is indeed such a configuration variable quarkus.langchain4j.azure-openai.endpoint, but it has a serious drawback. Easy RAG does not work and switching between models has to be done from scratch.

I like @csotiriou's idea with the domain, if possible add the quarkus.langchain4j.azure-openai.domain-name config variable, then you will be able to use the full potential of the extension.

What do you think about it?