使用web_demo输入图片时报错
Closed this issue · 0 comments
Leox0717 commented
System Info / 系統信息
Cuda 12.6 Python 3.10.12
Reproduction / 复现过程
在使用web_demo.py时 输入图片后报错。 运行模型为cogvlm2-llama3-chinese-chat-19B
root@:~/CogVLM2/basic_demo# chainlit run web_demo.py --host=10.10.14.27 --port=8001
Quant = 0
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████| 8/8 [00:01<00:00, 5.06it/s]
2024-10-10 11:13:22 - Your app is available at http://10.10.14.27:8001
2024-10-10 11:13:26 - Translated markdown file for zh-CN not found. Defaulting to chainlit.md.
<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=1278x1714 at 0xFFFF388DAD40>
input_ids torch.Size([1, 2317]) tensor([[128000, 128002, 128002, ..., 29129, 22559, 25]],
device='cuda:0')
inputs_embeds torch.Size([1, 2317, 4096]) tensor([[[ 7.5340e-05, 5.8174e-05, -1.4305e-04, ..., 4.8256e-04,
-7.7248e-05, 5.3406e-04],
[ 0.0000e+00, -0.0000e+00, 0.0000e+00, ..., 0.0000e+00,
-0.0000e+00, -0.0000e+00],
[ 0.0000e+00, -0.0000e+00, 0.0000e+00, ..., 0.0000e+00,
-0.0000e+00, -0.0000e+00],
...,
[ 6.4850e-04, -4.0588e-03, 2.8229e-03, ..., 3.1891e-03,
-4.3030e-03, -3.8147e-03],
[-1.6724e-02, -1.2878e-02, -6.2256e-03, ..., 5.7068e-03,
1.6357e-02, -5.4626e-03],
[ 2.9907e-03, -1.8082e-03, 1.2894e-03, ..., 3.2043e-03,
1.0147e-03, 1.1673e-03]]], device='cuda:0', dtype=torch.bfloat16)
images torch.Size([3, 1344, 1344]) tensor([[[-1.3281, -1.3281, -1.3281, ..., -1.2969, -1.2969, -1.2969],
[-1.3281, -1.3281, -1.3281, ..., -1.2969, -1.2969, -1.2969],
[-1.3281, -1.3281, -1.3281, ..., -1.2969, -1.2969, -1.2969],
...,
[-1.1641, -1.1641, -1.1641, ..., -1.1641, -1.1641, -1.1641],
[-1.1641, -1.1641, -1.1641, ..., -1.1641, -1.1641, -1.1641],
[-1.1641, -1.1641, -1.1641, ..., -1.1641, -1.1641, -1.1641]],
[[-1.2734, -1.2734, -1.2734, ..., -1.2422, -1.2422, -1.2422],
[-1.2734, -1.2734, -1.2734, ..., -1.2422, -1.2422, -1.2422],
[-1.2734, -1.2734, -1.2734, ..., -1.2422, -1.2422, -1.2422],
...,
[-1.1094, -1.1094, -1.1094, ..., -1.1094, -1.1094, -1.1094],
[-1.1094, -1.1094, -1.1094, ..., -1.1094, -1.1094, -1.1094],
[-1.1094, -1.1094, -1.1094, ..., -1.1094, -1.1094, -1.1094]],
[[-1.0234, -1.0234, -1.0234, ..., -0.9961, -0.9961, -0.9961],
[-1.0234, -1.0234, -1.0234, ..., -0.9961, -0.9961, -0.9961],
[-1.0234, -1.0234, -1.0234, ..., -0.9961, -0.9961, -0.9961],
...,
[-0.8672, -0.8672, -0.8672, ..., -0.8672, -0.8672, -0.8672],
[-0.8672, -0.8672, -0.8672, ..., -0.8672, -0.8672, -0.8672],
[-0.8672, -0.8672, -0.8672, ..., -0.8672, -0.8672, -0.8672]]],
device='cuda:0', dtype=torch.bfloat16)
x torch.Size([1, 9217, 1792]) tensor([[[ 1.1797, 0.5195, -0.3750, ..., 0.6953, -0.6562, -1.4219],
[ 2.2969, 2.0625, -1.8906, ..., 1.8984, 0.5547, -0.7148],
[ 2.2031, 2.1562, -1.9219, ..., 1.8203, 0.6250, -0.7422],
...,
[ 1.8125, -0.3242, 0.7930, ..., 0.5898, -0.6797, -1.1406],
[ 1.1797, 0.4297, 1.4141, ..., -0.1729, -0.5117, -0.9727],
[ 0.9180, 0.8633, 1.5938, ..., -0.2969, -0.3008, -0.8203]]],
device='cuda:0', dtype=torch.bfloat16)
2024-10-10 11:14:37 -
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/chainlit/utils.py", line 44, in wrapper
return await user_function(**params_values)
File "/usr/local/lib/python3.10/dist-packages/chainlit/callbacks.py", line 118, in with_parent_id
await func(message)
File "/root/CogVLM2/basic_demo/web_demo.py", line 248, in main
conv = await request(conv, settings)
File "/root/CogVLM2/basic_demo/web_demo.py", line 194, in request
async for response in get_response(query, history, gen_kwargs, images):
File "/root/CogVLM2/basic_demo/web_demo.py", line 85, in get_response
for next_text in streamer:
File "/usr/local/lib/python3.10/dist-packages/transformers/generation/streamers.py", line 223, in __next__
value = self.text_queue.get(timeout=self.timeout)
File "/usr/lib/python3.10/queue.py", line 179, in get
raise Empty
_queue.Empty