`httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)`
Opened this issue · 0 comments
DHaru85 commented
using dumps_kwargs keyword arguments are no longer supported.
while deploying and testing qwen1.5-7b-chat through fastchat v0.2.36, i got an exception :
2024-08-16 03:11:13 | INFO | stdout | INFO: 172.22.0.1:48704 - "POST /v1/chat/completions HTTP/1.1" 200 OK
2024-08-16 03:11:13 | ERROR | stderr | /usr/local/lib/python3.10/site-packages/fastchat/serve/openai_api_server.py:506: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
2024-08-16 03:11:13 | ERROR | stderr | yield f"data: {json.dumps(chunk.dict(exclude_unset=True), ensure_ascii=False)}\n\n"
2024-08-16 03:11:14 | ERROR | stderr | /usr/local/lib/python3.10/site-packages/fastchat/serve/openai_api_server.py:536: PydanticDeprecatedSince20: The `json` method is deprecated; use `model_dump_json` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
2024-08-16 03:11:14 | ERROR | stderr | yield f"data: {chunk.json(exclude_unset=True, ensure_ascii=False)}\n\n"
2024-08-16 03:11:14 | ERROR | stderr | ERROR: Exception in ASGI application
2024-08-16 03:11:14 | ERROR | stderr | Traceback (most recent call last):
2024-08-16 03:11:14 | ERROR | stderr | File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 277, in __call__
2024-08-16 03:11:14 | ERROR | stderr | await wrap(partial(self.listen_for_disconnect, receive))
2024-08-16 03:11:14 | ERROR | stderr | File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 273, in wrap
2024-08-16 03:11:14 | ERROR | stderr | await func()
2024-08-16 03:11:14 | ERROR | stderr | File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 250, in listen_for_disconnect
2024-08-16 03:11:14 | ERROR | stderr | message = await receive()
2024-08-16 03:11:14 | ERROR | stderr | File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 555, in receive
2024-08-16 03:11:14 | ERROR | stderr | await self.message_event.wait()
2024-08-16 03:11:14 | ERROR | stderr | File "/usr/local/lib/python3.10/asyncio/locks.py", line 214, in wait
2024-08-16 03:11:14 | ERROR | stderr | await fut
2024-08-16 03:11:14 | ERROR | stderr | asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f1ffb948160
2024-08-16 03:11:14 | ERROR | stderr |
2024-08-16 03:11:14 | ERROR | stderr | During handling of the above exception, another exception occurred:
2024-08-16 03:11:14 | ERROR | stderr |
2024-08-16 03:11:14 | ERROR | stderr | + Exception Group Traceback (most recent call last):
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
2024-08-16 03:11:14 | ERROR | stderr | | result = await app( # type: ignore[func-returns-value]
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | return await self.app(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 292, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | await super().__call__(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | await self.middleware_stack(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | raise exc
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | await self.app(scope, receive, _send)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 83, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | await self.app(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | raise exc
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | await self.app(scope, receive, sender)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | raise e
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | await self.app(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | await route.handle(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
2024-08-16 03:11:14 | ERROR | stderr | | await self.app(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 69, in app
2024-08-16 03:11:14 | ERROR | stderr | | await response(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 270, in __call__
2024-08-16 03:11:14 | ERROR | stderr | | async with anyio.create_task_group() as task_group:
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 680, in __aexit__
2024-08-16 03:11:14 | ERROR | stderr | | raise BaseExceptionGroup(
2024-08-16 03:11:14 | ERROR | stderr | | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
2024-08-16 03:11:14 | ERROR | stderr | +-+---------------- 1 ----------------
2024-08-16 03:11:14 | ERROR | stderr | | Traceback (most recent call last):
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 273, in wrap
2024-08-16 03:11:14 | ERROR | stderr | | await func()
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 262, in stream_response
2024-08-16 03:11:14 | ERROR | stderr | | async for chunk in self.body_iterator:
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/fastchat/serve/openai_api_server.py", line 536, in chat_completion_stream_generator
2024-08-16 03:11:14 | ERROR | stderr | | yield f"data: {chunk.json(exclude_unset=True, ensure_ascii=False)}\n\n"
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/typing_extensions.py", line 2853, in wrapper
2024-08-16 03:11:14 | ERROR | stderr | | return arg(*args, **kwargs)
2024-08-16 03:11:14 | ERROR | stderr | | File "/usr/local/lib/python3.10/site-packages/pydantic/main.py", line 1011, in json
2024-08-16 03:11:14 | ERROR | stderr | | raise TypeError('`dumps_kwargs` keyword arguments are no longer supported.')
2024-08-16 03:11:14 | ERROR | stderr | | TypeError: `dumps_kwargs` keyword arguments are no longer supported.
2024-08-16 03:11:14 | ERROR | stderr | +------------------------------------
it looks like chunk.json() functions are using dumps_kwargs keyword arguments which have been abondaned by pydantic (version>2.0.0)
fixed by using json.dumps() to get chunk dict string rather than chunk.json() initialy used.
for example:
# initial
yield f"data: {chunk.json(exclude_unset=True, ensure_ascii=False)}\n\n"
# after
yield f"data: {json.dumps(chunk.dict(exclude_unset=True), ensure_ascii=False)}\n\n" # for pydantic >= 2.0 issue