stulzq/azure-openai-proxy

How to deploy gpt-4-1106-preview?

haq1an opened this issue · 5 comments

haq1an commented

I'm bit confused how openai and azure openai name its model.
Can someone help me to check if I'm setting the right parameter to use gpt-4-1106-preview?
My azure openai deployment:
image
My mapper in the proxy:
gpt-4-1106-preview=gpt-4-turbo

And the project I'm using is Kerwin1202/chatgpt-web with the latest docker image with support of gpt-4-1106-preview.

Somehow I'm able to call the model, but instead of gpt-4-1106-preview, it's just gpt-4 (determined by it training data date).

TBH I'm not sure what exactly the issue is, this whole mapping stuff and azure openai's terrible naming standard is just a mess to me, so I hope that someone could help me out here.

lanceli93 commented

@haq1an Here's my settings and it works perfect:

environment:
  AZURE_OPENAI_ENDPOINT: xxx
  AZURE_OPENAI_MODEL_MAPPER: gpt-4=gpt-4-1106
  AZURE_OPENAI_API_VER: 2023-08-01-preview

gpt-4: doesn't matter because it will be mapped to deployment name eventually
gpt-4-1106: deployment name for gpt-4-1106-preview model on azure openai set by myself

PS: you should give a system prompt for gpt-4-turbo on azure: "You are ChatGPT, a large language model trained by OpenAI. Your knowledge has been up to April 2023. "

ripenedcat commented

我也遇到了一样的问题，在azure openai playground里问chatgpt是没问题的，最新训练日期是23年4月。但是通过本项目问他就只有21年9月。看log转换的应该是正确，不知问题具体在哪。难道是后面这段加粗部分的问题?
2023/12/17 12:33:18 proxying request [gpt-4-turbo] /v1/chat/completions?path=v1&path=chat&path=completions -> https://xxxx.openai.azure.com/openai/deployments/gpt-4-vision/chat/completions?api-version=2023-12-01-preview**&path=v1&path=chat&path=completions**
[GIN] 2023/12/17 - 12:33:21 | 200 | 2.81994504s | 40.71.xxx.xx | POST "/v1/chat/completions?path=v1&path=chat&path=completions"

@lanceli93 这个有些自欺欺人了，就算这么写，问他什么是stable diffusion, 他是答不上来的,如下使用API，他并不知道Stable Diffusion是什么
image
但是Azure openai playground中回答的没问题
image

@stulzq 大佬不知是否能指点一二，问题出在哪？

lanceli93 commented

@ripenedcat 必须要放在system prompt中, 目前的Azure 上的gpt-4-turbo原本就不大稳定, 这是微软内部的workaround. 用下面的system prompt, 问2022年世界杯结果即可验证:
You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown. Your knowledge has been up to April 2023.

ripenedcat commented

@ripenedcat 必须要放在system prompt中, 目前的Azure 上的gpt-4-turbo原本就不大稳定, 这是微软内部的workaround. 用下面的system prompt, 问2022年世界杯结果即可验证:
You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown. Your knowledge has been up to April 2023.

ok. 那只能说azure openai有些拉胯了....

liudhzhyym commented

同样的问题，在azure openai playground里问chatgpt是没问题的，最新训练日期是23年4月。但是通过本项目问他就只有21年9月。