set OPENAI_BASE_URL - openAI

Question

set OPENAI_BASE_URL - openAI

shuther opened this issue 5 months ago · 3 comments

even with updating env.py, it seems the script doesn't want to use the set openai base url:
script used:

Answer 1 · 2024-05-14T16:46:15.000Z

Can you please provide the full details of what you are trying to achieve, a bit more details about your machine (like what OS you are using) and what you have already tried?

Answer 2 · 2024-05-15T08:31:09.000Z

OS: VM linux
I am trying to leverage ollama/litellm openAI compatible endpoints to test further data-to-paper. It means I need to enforce the openai base url.
I tried changing env.py (to update the openai base url as well as the deepinfra baseurl) to set a different endpoint but it is as if only gpt4 was called. I have no logs on my endpoint side (that it tried to connect to).

Answer 3 · 2024-05-15T17:38:46.000Z

Thanks @shuther, for giving more details.

I have created a new branch (attached to this issue), that should support llama3 (as an example), hopefully it will also make it more clear how to generalize to other ollama/litellm models. Specifically, look at the commit c699eeb.