Calling any tool will Azure fails
HAFNIAxSimon opened this issue · 2 comments
HAFNIAxSimon commented
What is the issue?
Hi, thanks for open-sourcing this tool! Looks great ;-)
My issue:
I have setup the toolkit with Cohere endpoint, all good. I can converse and use the tool.
However, I have setup the configuration for Azure endpoint with Command R + on Azure AI Studio.
I can converse with this endpoint in simple chat-mode. However, whenever I add a tool in my selection. It fails with: "Unable to generate a response since an error was encountered. network error".
Running it on Windows
Backend logs:
2024-07-12 11:05:17 backend-1 | The title is: Plot the average 1 bedroom
2024-07-12 11:05:17 backend-1 | 2024-07-12 03:05:17,501 - root - INFO - POST /v1/chat-stream
2024-07-12 11:05:17 backend-1 | Headers({'host': 'localhost:8000', 'connection': 'keep-alive', 'content-length': '232', 'user-id': 'user-id', 'sec-ch-ua': '"Not/A)Brand";v="8", "Chromium";v="126", "Microsoft Edge";v="126"', 'sec-ch-ua-mobile': '?0', 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 Edg/126.0.0.0', 'content-type': 'application/json', 'accept': 'text/event-stream', 'deployment-name': 'Azure', 'deployment-config': '', 'sec-ch-ua-platform': '"Windows"', 'origin': 'http://localhost:4000', 'sec-fetch-site': 'same-site', 'sec-fetch-mode': 'cors', 'sec-fetch-dest': 'empty', 'referer': 'http://localhost:4000/', 'accept-encoding': 'gzip, deflate, br, zstd', 'accept-language': 'en-US,en;q=0.9'})
2024-07-12 11:05:17 backend-1 | 2024-07-12 03:05:17,502 - root - INFO -
2024-07-12 11:05:17 backend-1 |
2024-07-12 11:05:17 backend-1 | curl -X POST -H "Content-Type: application/json" -d '{"signal": {"id": "3ceceb82-c8e7-4ccb-a18c-ef4a64485d8b", "user_id": "user-id", "trace_id": "e4e477ff-2bf3-432e-9dc8-1569a8041065", "method": "post", "endpoint_name": "/v1/chat-stream", "success": true, "timestamp": 1720753025.0923464, "secret": "'$SECRET'", "status_code": 200, "input_nb_tokens": null, "output_nb_tokens": null, "search_units": null, "model": null, "error": null, "object_ids": {}, "duration_ms": 0.029943887995614205, "meta": null, "assistant_id": null, "assistant": null, "user": null}}' $ENDPOINT
2024-07-12 11:05:17 backend-1 |
2024-07-12 11:05:17 backend-1 |
2024-07-12 11:05:17 backend-1 | 2024-07-12 03:05:17,502 - root - ERROR - No report secret set
2024-07-12 11:05:17 backend-1 | INFO: 172.18.0.1:52690 - "POST /v1/chat-stream HTTP/1.1" 200 OK
2024-07-12 11:05:17 backend-1 | Azure chat endpoint url: https://Cohere-command-r-plus-oicts.eastus.models.ai.azure.com/v1
2024-07-12 11:05:17 backend-1 | 2024-07-12 03:05:17,544 - backend.services.logger - INFO - Using deployment AzureDeployment
2024-07-12 11:05:17 backend-1 | 2024-07-12 03:05:17,544 - root - WARNING - kwargs: {'stream': True, 'deployment_name': 'Azure', 'deployment_config': {}, 'file_paths': None, 'managed_tools': True, 'session': <sqlalchemy.orm.session.Session object at 0x7fd7ce37e790>, 'conversation_id': 'caeb9b82-2bd0-42bf-9dbb-50f2ac6ca48c', 'user_id': 'user-id', 'trace_id': 'e4e477ff-2bf3-432e-9dc8-1569a8041065', 'agent_id': None}
2024-07-12 11:05:17 backend-1 | Chat history: []
2024-07-12 11:05:17 backend-1 | 2024-07-12 03:05:17,544 - backend.services.logger - INFO - Step 1
2024-07-12 11:05:19 backend-1 | 2024-07-12 03:05:19,491 - httpx - INFO - HTTP Request: POST https://cohere-command-r-plus-oicts.eastus.models.ai.azure.com/v1/chat "HTTP/1.1 200 OK"
2024-07-12 11:05:19 db-1 | 2024-07-12 03:05:19.496 UTC [25633] ERROR: null value in column "text" of relation "messages" violates not-null constraint
2024-07-12 11:05:19 db-1 | 2024-07-12 03:05:19.496 UTC [25633] DETAIL: Failing row contains (null, null, CHATBOT, user-id, caeb9b82-2bd0-42bf-9dbb-50f2ac6ca48c, 0, t, f85c6346-2a8e-4f57-b4f9-2d6d9dd1a1fe, 2024-07-12 03:05:19.496355, 2024-07-12 03:05:19.496355, null).
2024-07-12 11:05:19 db-1 | 2024-07-12 03:05:19.496 UTC [25633] STATEMENT: INSERT INTO messages (text, user_id, conversation_id, position, is_active, generation_id, tool_plan, agent, id, created_at, updated_at) VALUES (NULL, 'user-id', 'caeb9b82-2bd0-42bf-9dbb-50f2ac6ca48c', 0, true, NULL, NULL, 'CHATBOT', 'f85c6346-2a8e-4f57-b4f9-2d6d9dd1a1fe', now(), now()) RETURNING messages.created_at, messages.updated_at
2024-07-12 11:05:19 backend-1 | ERROR: Exception in ASGI application
2024-07-12 11:05:19 backend-1 | Traceback (most recent call last):
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
2024-07-12 11:05:19 backend-1 | self.dialect.do_execute(
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 924, in do_execute
2024-07-12 11:05:19 backend-1 | cursor.execute(statement, parameters)
2024-07-12 11:05:19 backend-1 | psycopg2.errors.NotNullViolation: null value in column "text" of relation "messages" violates not-null constraint
2024-07-12 11:05:19 backend-1 | DETAIL: Failing row contains (null, null, CHATBOT, user-id, caeb9b82-2bd0-42bf-9dbb-50f2ac6ca48c, 0, t, f85c6346-2a8e-4f57-b4f9-2d6d9dd1a1fe, 2024-07-12 03:05:19.496355, 2024-07-12 03:05:19.496355, null).
2024-07-12 11:05:19 backend-1 |
2024-07-12 11:05:19 backend-1 |
2024-07-12 11:05:19 backend-1 | The above exception was the direct cause of the following exception:
2024-07-12 11:05:19 backend-1 |
2024-07-12 11:05:19 backend-1 | Traceback (most recent call last):
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
2024-07-12 11:05:19 backend-1 | result = await app( # type: ignore[func-returns-value]
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
2024-07-12 11:05:19 backend-1 | return await self.app(scope, receive, send)
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
2024-07-12 11:05:19 backend-1 | await super().__call__(scope, receive, send)
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
2024-07-12 11:05:19 backend-1 | await self.middleware_stack(scope, receive, send)
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
2024-07-12 11:05:19 backend-1 | raise exc
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
2024-07-12 11:05:19 backend-1 | await self.app(scope, receive, _send)
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
2024-07-12 11:05:19 backend-1 | with collapse_excgroups():
2024-07-12 11:05:19 backend-1 | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
2024-07-12 11:05:19 backend-1 | self.gen.throw(typ, value, traceback)
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
2024-07-12 11:05:19 backend-1 | raise exc
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
2024-07-12 11:05:19 backend-1 | await func()
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
2024-07-12 11:05:19 backend-1 | return await super().stream_response(send)
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
2024-07-12 11:05:19 backend-1 | async for chunk in self.body_iterator:
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
2024-07-12 11:05:19 backend-1 | raise app_exc
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
2024-07-12 11:05:19 backend-1 | await self.app(scope, receive_or_disconnect, send_no_error)
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
2024-07-12 11:05:19 backend-1 | with collapse_excgroups():
2024-07-12 11:05:19 backend-1 | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
2024-07-12 11:05:19 backend-1 | self.gen.throw(typ, value, traceback)
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
2024-07-12 11:05:19 backend-1 | raise exc
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sse_starlette/sse.py", line 273, in wrap
2024-07-12 11:05:19 backend-1 | await func()
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sse_starlette/sse.py", line 253, in stream_response
2024-07-12 11:05:19 backend-1 | async for data in self.body_iterator:
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/concurrency.py", line 65, in iterate_in_threadpool
2024-07-12 11:05:19 backend-1 | yield await anyio.to_thread.run_sync(_next, as_iterator)
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
2024-07-12 11:05:19 backend-1 | return await get_async_backend().run_sync_in_worker_thread(
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2177, in run_sync_in_worker_thread
2024-07-12 11:05:19 backend-1 | return await future
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 859, in run
2024-07-12 11:05:19 backend-1 | result = context.run(func, *args)
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/starlette/concurrency.py", line 54, in _next
2024-07-12 11:05:19 backend-1 | return next(iterator)
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/src/backend/services/chat.py", line 579, in generate_chat_stream
2024-07-12 11:05:19 backend-1 | handle_stream_event(
2024-07-12 11:05:19 backend-1 | File "/workspace/src/backend/services/chat.py", line 634, in handle_stream_event
2024-07-12 11:05:19 backend-1 | return handlers[event_type](
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/src/backend/services/chat.py", line 768, in handle_stream_tool_calls_generation
2024-07-12 11:05:19 backend-1 | save_tool_calls_message(
2024-07-12 11:05:19 backend-1 | File "/workspace/src/backend/services/chat.py", line 453, in save_tool_calls_message
2024-07-12 11:05:19 backend-1 | message = create_message(
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/src/backend/services/chat.py", line 319, in create_message
2024-07-12 11:05:19 backend-1 | return message_crud.create_message(session, message)
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/src/backend/crud/message.py", line 19, in create_message
2024-07-12 11:05:19 backend-1 | db.commit()
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2017, in commit
2024-07-12 11:05:19 backend-1 | trans.commit(_to_root=True)
2024-07-12 11:05:19 backend-1 | File "<string>", line 2, in commit
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
2024-07-12 11:05:19 backend-1 | ret_value = fn(self, *arg, **kw)
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1302, in commit
2024-07-12 11:05:19 backend-1 | self._prepare_impl()
2024-07-12 11:05:19 backend-1 | File "<string>", line 2, in _prepare_impl
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
2024-07-12 11:05:19 backend-1 | ret_value = fn(self, *arg, **kw)
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1277, in _prepare_impl
2024-07-12 11:05:19 backend-1 | self.session.flush()
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4341, in flush
2024-07-12 11:05:19 backend-1 | self._flush(objects)
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4476, in _flush
2024-07-12 11:05:19 backend-1 | with util.safe_reraise():
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
2024-07-12 11:05:19 backend-1 | raise exc_value.with_traceback(exc_tb)
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4437, in _flush
2024-07-12 11:05:19 backend-1 | flush_context.execute()
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 466, in execute
2024-07-12 11:05:19 backend-1 | rec.execute(self)
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 642, in execute
2024-07-12 11:05:19 backend-1 | util.preloaded.orm_persistence.save_obj(
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/persistence.py", line 93, in save_obj
2024-07-12 11:05:19 backend-1 | _emit_insert_statements(
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/orm/persistence.py", line 1233, in _emit_insert_statements
2024-07-12 11:05:19 backend-1 | result = connection.execute(
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1418, in execute
2024-07-12 11:05:19 backend-1 | return meth(
2024-07-12 11:05:19 backend-1 | ^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 515, in _execute_on_connection
2024-07-12 11:05:19 backend-1 | return connection._execute_clauseelement(
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1640, in _execute_clauseelement
2024-07-12 11:05:19 backend-1 | ret = self._execute_context(
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
2024-07-12 11:05:19 backend-1 | return self._exec_single_context(
2024-07-12 11:05:19 backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
2024-07-12 11:05:19 backend-1 | self._handle_dbapi_exception(
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2353, in _handle_dbapi_exception
2024-07-12 11:05:19 backend-1 | raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
2024-07-12 11:05:19 backend-1 | self.dialect.do_execute(
2024-07-12 11:05:19 backend-1 | File "/workspace/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 924, in do_execute
2024-07-12 11:05:19 backend-1 | cursor.execute(statement, parameters)
2024-07-12 11:05:19 backend-1 | sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "text" of relation "messages" violates not-null constraint
2024-07-12 11:05:19 backend-1 | DETAIL: Failing row contains (null, null, CHATBOT, user-id, caeb9b82-2bd0-42bf-9dbb-50f2ac6ca48c, 0, t, f85c6346-2a8e-4f57-b4f9-2d6d9dd1a1fe, 2024-07-12 03:05:19.496355, 2024-07-12 03:05:19.496355, null).
2024-07-12 11:05:19 backend-1 |
2024-07-12 11:05:19 backend-1 | [SQL: INSERT INTO messages (text, user_id, conversation_id, position, is_active, generation_id, tool_plan, agent, id, created_at, updated_at) VALUES (%(text)s, %(user_id)s, %(conversation_id)s, %(position)s, %(is_active)s, %(generation_id)s, %(tool_plan)s, %(agent)s, %(id)s, now(), now()) RETURNING messages.created_at, messages.updated_at]
2024-07-12 11:05:19 backend-1 | [parameters: {'text': None, 'user_id': 'user-id', 'conversation_id': 'caeb9b82-2bd0-42bf-9dbb-50f2ac6ca48c', 'position': 0, 'is_active': True, 'generation_id': None, 'tool_plan': None, 'agent': 'CHATBOT', 'id': 'f85c6346-2a8e-4f57-b4f9-2d6d9dd1a1fe'}]
2024-07-12 11:05:19 backend-1 | (Background on this error at: https://sqlalche.me/e/20/gkpj)
Did I do anything wrong in the configuration? Thanks !
Additional information
No response
BeatrixCohere commented
Hey, thanks for trying the toolkit out. Have you tried running make migrate as this appears to be a DB issue?
HAFNIAxSimon commented
Thanks @BeatrixCohere, it fixed my issue.