如何使用v2.13.0的自定义模型功能（how to use 2.13.0 multi models)
lloydzhou opened this issue · 23 comments
- 在v2.13.0版本扩展了CUSTOM_MODELS支持的语法
支持通过
@指定provider，支持通过=指定deploy_name
- 解决Azure和OpenAI的模型不能共存的问题
+gpt-3.5-turbo@openai,+gpt-3.5-turbo@azure=gpt-3.5
以上配置会分别展示两个模型gpt-3.5-turbo(OpenAI)以及gpt-3.5(Azure)
并且使用Azure部署的gpt-3.5-turbo模型会把请求发送到deploy_name=gpt-3.5这个部署的服务上面
- 新增使用字节跳动提供的豆包模型的时候定义deploy_id
+Doubao-lite-4k@bytedance=ep-2024xxxx-xxx
以上配置会新增一个Doubao-lite-4k(ByteDance)的模型，当选择这个模型的时候，会将请求发送到ep-2024xxxx-xxx这个deploy-id服务上面
- 支持自定义厂商发布的不在内置模型列表中的新模型
假如openai发布了gpt-4.5，但是nextchat未及时发布新版
可以配置+gpt-4.5@OpenAI，模型列表会新增gpt-4.5(OpenAI)选项，并且按openai的消息格式发送请求
- 考虑到一些厂商做中转的时候会将多家厂商的模型都按OpenAI的格式做转发，也可以通过
CUSTOM_MODELS配置使用：
例如，厂商同时中转了
gpt-3.5-turbo和claude-2.1，都走兼容openai的格式
配置+gpt-3.5-turbo@OpenAI,+claude-2.1@OpenAI（这里使用OpenAI而不是openai代表不是内置的服务商列表，只是符合OpenAI的消息格式）
会新增gpt-3.5-turbo(OpenAI)和claude-2.1(OpenAI)两个模型选项。请求都会发送到/api/openai/*
这个时候可以在.env通过配置BASE_URL+OPENAI_API_KEY的模式，或者签到配置自定义接口走这家中转供应商的服务
Please follow the issue template to update title and description of your issue.
Bot detected the issue body's language is not English, translate it automatically.
Title: How to use the custom model function of v2.13.0 (how to use 2.13.0 multi models)
- The syntax supported by CUSTOM_MODELS has been expanded in version v2.13.0
Supports specifying provider through
@, and supports specifying deploy_name through=
- Solve the problem that Azure and OpenAI models cannot coexist
+gpt-3.5-turbo@openai,+gpt-3.5-turbo@azure=gpt-3.5
The above configuration will display two modelsgpt-3.5-turbo(OpenAI)andgpt-3.5(Azure)respectively
And using thegpt-3.5-turbomodel deployed by Azure will send the request to the deployed servicedeploy_name=gpt-3.5
- Added the definition of deploy_id when using the beanbao model provided by ByteDance.
+Doubao-lite-4k@bytedance=ep-2024xxxx-xxx
The above configuration will add aDoubao-lite-4k(ByteDance)model. When this model is selected, the request will be sent to theep-2024xxxx-xxxdeploy-id service
- Support new models released by custom manufacturers that are not in the built-in model list
If openai releases gpt-4.5, but nextchat does not release the new version in time
You can configure+gpt-4.5@OpenAI, thegpt-4.5(OpenAI)option will be added to the model list, and the request will be sent according to the message format of openai
- Considering that some manufacturers will forward models from multiple manufacturers according to the OpenAI format when transferring, they can also be configured and used through
CUSTOM_MODELS:
For example, the manufacturer forwarded
gpt-3.5-turboandclaude-2.1at the same time, both in formats compatible with openai.
Configure+gpt-3.5-turbo@OpenAI,+claude-2.1@OpenAI(the use of OpenAI instead of openai here means that it is not a built-in service provider list, but only conforms to the message format of OpenAI)
Two new model options,gpt-3.5-turbo(OpenAI)andclaude-2.1(OpenAI), will be added. Requests will be sent to/api/openai/*
At this time, you can configure theBASE_URL+OPENAI_API_KEYmode in .env, or sign in to configure a custom interface to use the services of this transit provider
感谢提供的帮助。有个疑问，怎么对模型进行排序，比如自定义模型排列在最前面。
Bot detected the issue body's language is not English, translate it automatically.
Thanks for the help. I have a question, how to sort the models, such as custom models at the front.
Bot detected the issue body's language is not English, translate it automatically.
Also, I would like to ask how to support multiple custom deployments #4398
docker compose部署发现按描述配置：+gpt-3.5-turbo@azure=gpt-3.5，无法正常生效，没有配置任何openai的参数，但默认走openai导致返回错误
Bot detected the issue body's language is not English, translate it automatically.
The docker compose deployment found that it was configured as described: +gpt-3.5-turbo@azure=gpt-3.5, which could not take effect normally. No openai parameters were configured, but openai was used by default.
自定义模型的url，在env中是配置哪个变量？
Bot detected the issue body's language is not English, translate it automatically.
Which variable is configured in env for the custom model url?
问答时报错：Unknown parameter: 'path'
chatgpt-next的配置: gpt-4o@openai，代理地址是one-api，one-api对接的是azure的api。
请求one-api的参数：
{
"messages": [
{
"role": "system",
"content": "\nYou are ChatGPT, a large language model trained by OpenAI.\nKnowledge cutoff: 2023-10\nCurrent model: gpt-4o\nCurrent time: Mon Jul 22 2024 16:32:39 GMT+0800 (**标准时间)\nLatex inline: \(x^2\) \nLatex block:
},
{
"role": "user",
"content": "你好呀，gpt"
}
],
"stream": true,
"model": "gpt-4o",
"temperature": 0.5,
"presence_penalty": 0,
"frequency_penalty": 0,
"top_p": 1,
"path": "completions"
}
Bot detected the issue body's language is not English, translate it automatically.
Error during Q&A: Unknown parameter: 'path'
The configuration of chatgpt-next: gpt-4o@openai, the proxy address is one-api, and one-api is connected to the azure api.
Parameters for requesting one-api:
{
"messages": [
{
"role": "system",
"content": "\nYou are ChatGPT, a large language model trained by OpenAI.\nKnowledge cutoff: 2023-10\nCurrent model: gpt-4o\nCurrent time: Mon Jul 22 2024 16:32:39 GMT+0800 (China Standard time)\nLatex inline: \(x^2\) \nLatex block:
},
{
"role": "user",
"content": "Hello, gpt"
}
],
"stream": true,
"model": "gpt-4o",
"temperature": 0.5,
"presence_penalty": 0,
"frequency_penalty": 0,
"top_p": 1,
"path": "completions"
}
谢谢，解决了第三方api的问题
Bot detected the issue body's language is not English, translate it automatically.
Thank you, the problem with the third-party API has been solved
弱弱地问下，插件里面的Artifacts是干嘛的，谷歌、百度了一遍，还是没看明白。。。
Bot detected the issue body's language is not English, translate it automatically.
I asked weakly what the Artifacts in the plug-in are for. I searched Google and Baidu, but I still don’t understand. . .
Bot detected the issue body's language is not English, translate it automatically.
I asked weakly what the Artifacts in the plug-in are for. I searched Google and Baidu, but I still don’t understand. . .
适用Claude模型的一个插件，具体可以看一下官方的说明
https://www.anthropic.com/news/claude-3-5-sonnet
我在使用该项目调用自己微调的大模型接口时，输出的token被限制在了100，请问 我应该如何修改。
Bot detected the issue body's language is not English, translate it automatically.
When I use this project to call the large model interface I fine-tuned, the output token is limited to 100. How should I modify it?