[BUG] <opanai_api.py无法分析本地图片>

Question

[BUG] <opanai_api.py无法分析本地图片>

Opened this issue 2 months ago · 10 comments

qinzhenyi1314 commented 2 months ago

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

自己本地环境是OK的，直接hugging face上的调用是可以正确出结果的，改为opanai_api.py这种方式如果不问图片是OK的，带图片的自己搜了下其他的issue看到需要构造成这种形式，自己改动后结果不正常

{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "please describe this image: /tmp/pycharm_project_175/assets/demo.jpeg"
}
],
"temperature": 0.7,
"top_p": 0.9,
"max_length": 100,
"stream": false,
"stop": [
"<|endoftext|>"
]
}

然后打印的结果为

[]
please describe this image: /tmp/pycharm_project_175/assets/demo.jpeg

Received.

INFO: - "POST /v1/chat/completions HTTP/1.1" 200 OK

只有一个Received. 没有具体图片的信息

期望行为 | Expected Behavior

希望可以读取本地文件或者像openai的官方那种方式输入base64进行处理

复现方法 | Steps To Reproduce

运行 openai_api.py 然后http发送请求

运行环境 | Environment

No response

备注 | Anything else?

无

Answer 1 · 2024-04-09T21:59:34.000Z

试试这样搞
{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "/home/a47/桌面/R-2C.jpg描述如下图片 "
}
],
"temperature": 0.7,
"top_p": 0.9,
"max_length": 100,
"stream": false,
"stop": [
"<|endoftext|>"
]
}

Answer 2 · 2024-04-10T17:04:25.000Z

@poppp234
如果图片和模型在同一台机器上，模型回复：I'm sorry, but as an AI language model, I do not have access to files or directories on your computer or any external system.
如果不在同一台机器上，比如模型在服务器上运行，我需要在其他电脑调用api，应该如何上传图片呢？

Answer 3 · 2024-04-11T03:49:05.000Z

@poppp234 如果图片和模型在同一台机器上，模型回复：I'm sorry, but as an AI language model, I do not have access to files or directories on your computer or any external system. 如果不在同一台机器上，比如模型在服务器上运行，我需要在其他电脑调用api，应该如何上传图片呢？

我是自己重新构建了一个图片上传API 图片由客户端上传到服务器端后返回服务器相对路径到客户端然后根据这个相对路径构建json传给服务器

Answer 4 · 2024-04-11T04:15:21.000Z

@poppp234 如果图片和模型在同一台机器上，模型回复：I'm sorry, but as an AI language model, I do not have access to files or directories on your computer or any external system. 如果不在同一台机器上，比如模型在服务器上运行，我需要在其他电脑调用api，应该如何上传图片呢？

我是自己重新构建了一个图片上传API 图片由客户端上传到服务器端后返回服务器相对路径到客户端然后根据这个相对路径构建json传给服务器

那就是我提到的第一种情况，模型和图片都在服务器上。请问你的json是如何构建的呢？可以给一个demo参考一下吗？

另外我注意到一个异常的情况，随着对话轮数的增加，显存占用不断增加，这将会导致无法用较大的数据集对模型进行测评。请问这个问题有解决方案吗？

Answer 5 · 2024-04-11T04:27:43.000Z

看图加标签试试确定本地相对路径是对的吗？关于显存增加的问题我还没注意回头我测试一下。

Answer 6 · 2024-04-11T04:28:40.000Z

构建json要按webui的逻辑来图在先文在后读路径文件权限够

Answer 7 · 2024-04-11T04:32:38.000Z

@poppp234 如果图片和模型在同一台机器上，模型回复：I'm sorry, but as an AI language model, I do not have access to files or directories on your computer or any external system. 如果不在同一台机器上，比如模型在服务器上运行，我需要在其他电脑调用api，应该如何上传图片呢？

我是自己重新构建了一个图片上传API 图片由客户端上传到服务器端后返回服务器相对路径到客户端然后根据这个相对路径构建json传给服务器

那就是我提到的第一种情况，模型和图片都在服务器上。请问你的json是如何构建的呢？可以给一个demo参考一下吗？

另外我注意到一个异常的情况，随着对话轮数的增加，显存占用不断增加，这将会导致无法用较大的数据集对模型进行测评。请问这个问题有解决方案吗？

刚看了一下服务器我是4090显卡昨天开启的服务测试一天下来基本稳定在12G左右显卡占用没有明显变化可能我们的使用方式有区别这个还要多测试看。

Answer 8 · 2024-04-17T14:22:36.000Z

@poppp234 你好按图片在前的方式，显示500报错了如果单纯问一段文字是正常的~

INFO: 10.0.10.7:20191 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 407, in run_asgi result = await app( # type: ignore[func-returns-value] File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__ return await self.app(scope, receive, send) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__ await self.middleware_stack(scope, receive, send) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__ raise exc File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__ await self.app(scope, receive, _send) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__ await self.app(scope, receive, send) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__ await self.middleware_stack(scope, receive, send) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/routing.py", line 776, in app await route.handle(scope, receive, send) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle await self.app(scope, receive, send) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/routing.py", line 77, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/starlette/routing.py", line 72, in app response = await func(request) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app raw_response = await run_endpoint_function( File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(**values) File "/tmp/pycharm_project_175/openai_api.py", line 381, in create_chat_completion response, _ = model.chat( File "/home/work/.cache/huggingface/modules/transformers_modules/Qwen-VL-Chat/modeling_qwen.py", line 939, in chat outputs = self.generate( File "/home/work/.cache/huggingface/modules/transformers_modules/Qwen-VL-Chat/modeling_qwen.py", line 1058, in generate return super().generate( File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/transformers/generation/utils.py", line 1525, in generate return self.sample( File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/transformers/generation/utils.py", line 2622, in sample outputs = self( File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = module._old_forward(*args, **kwargs) File "/home/work/.cache/huggingface/modules/transformers_modules/Qwen-VL-Chat/modeling_qwen.py", line 848, in forward transformer_outputs = self.transformer( File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/home/work/.cache/huggingface/modules/transformers_modules/Qwen-VL-Chat/modeling_qwen.py", line 565, in forward images = self.visual.encode(images) File "/home/work/.cache/huggingface/modules/transformers_modules/Qwen-VL-Chat/visual.py", line 426, in encode return self(images) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = module._old_forward(*args, **kwargs) File "/home/work/.cache/huggingface/modules/transformers_modules/Qwen-VL-Chat/visual.py", line 398, in forward x = self.conv1(x) # shape = [*, width, grid, grid] File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/accelerate/hooks.py", line 160, in new_forward args, kwargs = module._hf_hook.pre_forward(module, *args, **kwargs) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/accelerate/hooks.py", line 297, in pre_forward return send_to_device(args, self.execution_device), send_to_device( File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/accelerate/utils/operations.py", line 152, in send_to_device return honor_type( File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/accelerate/utils/operations.py", line 84, in honor_type return type(obj)(generator) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/accelerate/utils/operations.py", line 153, in <genexpr> tensor, (send_to_device(t, device, non_blocking=non_blocking, skip_keys=skip_keys) for t in tensor) File "/home/work/miniconda3/envs/Qwen-VL-Chat/lib/python3.10/site-packages/accelerate/utils/operations.py", line 171, in send_to_device return tensor.to(device, non_blocking=non_blocking) NotImplementedError: Cannot copy out of meta tensor; no data!

Answer 9 · 2024-05-24T02:33:54.000Z

@poppp234 如果图片和模型在同一台机器上，模型回复：I'm sorry, but as an AI language model, I do not have access to files or directories on your computer or any external system. 如果不在同一台机器上，比如模型在服务器上运行，我需要在其他电脑调用api，应该如何上传图片呢？

我是自己重新构建了一个图片上传API 图片由客户端上传到服务器端后返回服务器相对路径到客户端然后根据这个相对路径构建json传给服务器

请问json有示例吗，在疑惑要怎么传本地的图片到服务器去识别

Answer 10 · 2024-05-24T02:44:45.000Z

@poppp234 如果图片和模型在同一台机器上，模型回复：I'm sorry, but as an AI language model, I do not have access to files or directories on your computer or any external system. 如果不在同一台机器上，比如模型在服务器上运行，我需要在其他电脑调用api，应该如何上传图片呢？

我是自己重新构建了一个图片上传API 图片由客户端上传到服务器端后返回服务器相对路径到客户端然后根据这个相对路径构建json传给服务器

请问json有示例吗，在疑惑要怎么传本地的图片到服务器去识别

1、Json示例：

2、如何上传图片到服务器：
这取决于你的客户端是使用哪种框架来搭建的，如果是streamlit，可以在上传的时候指定服务器的路径img_path，随后在构造json的时候，传入img_path就好了。

另外关于之前提到的内存不断增加的问题，是因为官方demo里面历史记录会一直累加全部丢给模型；通过api使用时，可以设定携带的历史记录数，可以解决GPU内存问题。