本地部署 chatglm3 6B, api 官方案列,关闭流式回复就出错
Opened this issue · 0 comments
YunFenLei commented
OS: Ubuntu 22.04.2 LTS
GPU: 3090
描述:按照官方指导部署在本地,
以下为示例代码:
import openai
if name == "main":
openai.api_base = "http://192.168.100.118:8000/v1"
openai.api_key = "none"
for chunk in openai.ChatCompletion.create(
model="chatglm3-6b",
messages=[
{"role": "user", "content": "李焕英"}
],
stream=False
):
if hasattr(chunk.choices[0].delta, "content"):
print(chunk.choices[0].delta.content, end="", flush=True)
如果 stream=True 那回复内容正常。
如果 stream=False
就会出错,信息如下:
Traceback (most recent call last):
File "D:\python提交到qingyingkj\ooo.py", line 12, in
if hasattr(chunk.choices[0].delta, "content"):
AttributeError: 'str' object has no attribute 'choices'