Move model selection to .env
Closed this issue · 7 comments
Either make gpt4 default or allow admin to set in .env. Users may override it with /model
I think you can change default model by changing LLMModels.gpt_3_5_turbo
in the construct_default
function, which is located in app\models\gpt_models.py
.
@classmethod
def construct_default(
cls,
user_id: str,
chat_room_id: str,
gpt_model: LLMModels = LLMModels.gpt_3_5_turbo,
):
return cls(
user_gpt_profile=UserGptProfile(user_id=user_id, chat_room_id=chat_room_id),
gpt_model=gpt_model,
)
I allowed users to change model. If you don't want users to change llm model, just add few lines of validation code in changemodel function, which is located in app\utils\chatgpt\chatgpt_commands.py
.
@staticmethod
@CommandResponse.send_message_and_stop
async def changemodel(model: str, user_gpt_context: UserGptContext) -> str:
"""Change GPT model\n
/changemodel <model>"""
if model not in LLMModels._member_names_:
return f"Model must be one of {', '.join(LLMModels._member_names_)}"
llm_model: LLMModels = LLMModels._member_map_[model] # type: ignore
user_gpt_context.gpt_model = llm_model
await ChatGptCacheManager.update_profile_and_model(user_gpt_context)
return f"Model changed to {model}. Actual model: {llm_model.value.name}"
Yes I did change the default version. However, updates will overwrite it.
Each chat has its own Gpt model. By default, the newly created chat will use the model defined in 'construct_default'. And yes, entering '/model' will overwrite the default, and will permanently change model of the chat. You can check model by entering '/test' in chat. If you want to make it default, just create new chat, and this chat uses default model, which is independent from other chats
The reason why your model wasn't changed as though you changed its default model, is that this previous model name was stored in Redis. So, when entering chat, the model you previosly set will be loaded into the chat.
If you want to change model of all chats, maybe changing the code of reading context. Go to file app\utils\chatgpt\chatgpt_cache_manager.py
, change gpt_model=LLMModels._member_map_[stored_string["gpt_model"]]
to gpt_model=LLMModels.gpt_4
or something.
This will override redis-stored-model.
Now all chats will start with the model you defined, and changing model with /model
will be maintained until you refresh the page.
@classmethod
async def read_context(cls, user_id: str, chat_room_id: str) -> UserGptContext:
stored_string: dict[str, str | None] = {
field: await cache.redis.get(key) for field, key in cls._get_string_fields(user_id, chat_room_id).items()
}
stored_list: dict[str, list | None] = {
field: await cache.redis.lrange(key, 0, -1)
for field, key in cls._get_list_fields(user_id, chat_room_id).items()
}
# if any of stored strings are None, create new context
if any([value is None for value in stored_string.values()]):
default: UserGptContext = UserGptContext.construct_default(
user_id=user_id,
chat_room_id=chat_room_id,
)
await cls.create_context(default)
return default
for field, value in stored_string.items():
if value is not None:
stored_string[field] = orjson_loads(value)
for field, value in stored_list.items():
if value is not None:
stored_list[field] = [orjson_loads(v) for v in value]
return UserGptContext(
user_gpt_profile=UserGptProfile(**stored_string["user_gpt_profile"]), # type: ignore
gpt_model=LLMModels._member_map_[stored_string["gpt_model"]], # type: ignore
user_message_histories=[MessageHistory(**m) for m in stored_list["user_message_histories"]]
if stored_list["user_message_histories"] is not None
else [],
gpt_message_histories=[MessageHistory(**m) for m in stored_list["gpt_message_histories"]]
if stored_list["gpt_message_histories"] is not None
else [],
system_message_histories=[MessageHistory(**m) for m in stored_list["system_message_histories"]]
if stored_list["system_message_histories"] is not None
else [],
)
Yes this is clear. However pulling updates from repo, will over write entire file. .env is safest place to keep it.
def construct_default( cls, user_id: str, chat_room_id: str, gpt_model: LLMModels = LLMModels.gpt_4, // this will be revised by remote update )
ok I will add the feature on next commit.