postman调用api
Closed this issue · 2 comments
Reminder
- I have read the README and searched the existing issues.
Reproduction
Visit http://localhost:8000/docs for API document.
INFO: Started server process [68679]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
1、我用指令CUDA_VISIBLE_DEVICES=0 llamafactory-cli api examples/merge_lora/llama3_lora_sft.yaml启动上述服务（yaml文件中加载的模型为微调的模型chatglm3，可以正常加载这个模型，并进行chat对话）
2、现在我想利用API进行对话，然后进入到fastapi的页面，访问三个API，一开始利用Postman访问http://20.20.136.3:8000/v1/models，返回的结果是
{
"object": "list",
"data": [
{
"id": "gpt-3.5-turbo",
"object": "model",
"created": 1715764012,
"owned_by": "owner"
}
]
}
难道不应该是正在加载的模型名吗？或者是支持加载的模型列表
3、随后我又访问http://20.20.136.3:8000/v1/chat/completions这个接口，给的请求体如下：
{
"model": "string",
"messages": [
{
"role": "user",
"content": "string",
"tool_calls": [
{
"id": "string",
"type": "function",
"function": {
"name": "string",
"arguments": "string"
}
}
]
}
],
"tools": [
{
"type": "function",
"function": {
"name": "string",
"description": "string",
"parameters": {}
}
}
],
"do_sample": true,
"temperature": 0,
"top_p": 0,
"n": 1,
"max_tokens": 0,
"stream": false
}
请问上述的一些参数如何确定，比如"model": "string",有候选值吗？能否给一个请求体样例？谢谢
Expected behavior
No response
System Info
No response
Others
No response
单模型加载时模型名没有意义，我们没有实际使用这个值
单模型加载时模型名没有意义，我们没有实际使用这个值
请问我用上述的请求体,报错：Stop parameter is not supported in Huggingface engine yet.这是为什么？能否给一个完整的请求体实例