CORS error or Content type error on Docker image
SeaJungg opened this issue · 4 comments
SeaJungg commented
What is the issue?
Hello :)
I uploaded a written judgment pdf file, which is written on Korean, and asked question.
Has this defendant been punished?
1. What do you think about adding CORS middleware to the backend server?
There is CORS error when I click submit button at the bottom right on UI.
All I did to run cohere toolkit is below command:
docker run -e COHERE_API_KEY='>>MY_API_KEY<<' -p 8000:8000 -p 4000:4000 ghcr.io/cohere-ai/cohere-toolkit:latest
There is an option to turn off CORS as below, but wouldn't adding CORS-related middleware be a more convenient option for users?
2. After disabled CORS on Chrome, but Another error occurred
I started chrome --disable-web-security mode with below command:
open -n -a /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --args --user-data-dir="/tmp/chrome_dev_test" --disable-web-security
Then there is another error on browser:
Expected content-type to be text/event-stream, Actual: text/plain; charset=utf-8
And the backend log is:
2024-05-07 09:25:22 INFO: 172.17.0.1:49940 - "POST /conversations/upload_file HTTP/1.1" 200 OK
2024-05-07 09:25:42 2024-05-07 00:25:42,900 - backend.services.logger - INFO - Using deployment CohereDeployment
2024-05-07 09:25:43 2024-05-07 00:25:43,370 - httpx - INFO - HTTP Request: POST https://api.cohere.ai/v1/chat "HTTP/1.1 200 OK"
2024-05-07 09:25:43 2024-05-07 00:25:43,411 - backend.services.logger - INFO - Search queries generated: []
2024-05-07 09:25:43 2024-05-07 00:25:43,411 - backend.services.logger - INFO - Using retrievers: ['LangChainVectorDBRetriever']
2024-05-07 09:25:48 2024-05-07 00:25:48,159 - chromadb.telemetry.product.posthog - INFO - Anonymized telemetry enabled. See https://docs.trychroma.com/telemetry for more information.
2024-05-07 09:25:49 2024-05-07 00:25:49,234 - langchain_cohere.utils - WARNING - Retrying langchain_cohere.embeddings.CohereEmbeddings.embed_with_retry.<locals>._embed_with_retry in 4.0 seconds as it raised IndexError: list index out of range.
2024-05-07 09:25:53 2024-05-07 00:25:53,238 - langchain_cohere.utils - WARNING - Retrying langchain_cohere.embeddings.CohereEmbeddings.embed_with_retry.<locals>._embed_with_retry in 4.0 seconds as it raised IndexError: list index out of range.
2024-05-07 09:25:57 INFO: 172.17.0.1:57308 - "POST /chat-stream HTTP/1.1" 500 Internal Server Error
2024-05-07 09:25:57 ERROR: Exception in ASGI application
2024-05-07 09:25:57 Traceback (most recent call last):
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
2024-05-07 09:25:57 result = await app( # type: ignore[func-returns-value]
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
2024-05-07 09:25:57 return await self.app(scope, receive, send)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
2024-05-07 09:25:57 await super().__call__(scope, receive, send)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
2024-05-07 09:25:57 await self.middleware_stack(scope, receive, send)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
2024-05-07 09:25:57 raise exc
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
2024-05-07 09:25:57 await self.app(scope, receive, _send)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/cors.py", line 83, in __call__
2024-05-07 09:25:57 await self.app(scope, receive, send)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
2024-05-07 09:25:57 await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
2024-05-07 09:25:57 raise exc
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-05-07 09:25:57 await app(scope, receive, sender)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
2024-05-07 09:25:57 await self.middleware_stack(scope, receive, send)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
2024-05-07 09:25:57 await route.handle(scope, receive, send)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
2024-05-07 09:25:57 await self.app(scope, receive, send)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
2024-05-07 09:25:57 await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
2024-05-07 09:25:57 raise exc
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-05-07 09:25:57 await app(scope, receive, sender)
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
2024-05-07 09:25:57 response = await func(request)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/fastapi/routing.py", line 299, in app
2024-05-07 09:25:57 raise e
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/fastapi/routing.py", line 294, in app
2024-05-07 09:25:57 raw_response = await run_endpoint_function(
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
2024-05-07 09:25:57 return await dependant.call(**values)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/src/backend/routers/chat.py", line 97, in chat_stream
2024-05-07 09:25:57 CustomChat().chat(
2024-05-07 09:25:57 File "/workspace/src/backend/chat/custom/custom.py", line 90, in chat
2024-05-07 09:25:57 retriever.retrieve_documents(query)
2024-05-07 09:25:57 File "/workspace/src/backend/tools/retrieval/lang_chain.py", line 72, in retrieve_documents
2024-05-07 09:25:57 db = Chroma.from_documents(documents=pages, embedding=cohere_embeddings)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/langchain_community/vectorstores/chroma.py", line 778, in from_documents
2024-05-07 09:25:57 return cls.from_texts(
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/langchain_community/vectorstores/chroma.py", line 736, in from_texts
2024-05-07 09:25:57 chroma_collection.add_texts(
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/langchain_community/vectorstores/chroma.py", line 275, in add_texts
2024-05-07 09:25:57 embeddings = self._embedding_function.embed_documents(texts)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/langchain_cohere/embeddings.py", line 151, in embed_documents
2024-05-07 09:25:57 return self.embed(texts, input_type="search_document")
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/langchain_cohere/embeddings.py", line 118, in embed
2024-05-07 09:25:57 embeddings = self.embed_with_retry(
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/langchain_cohere/embeddings.py", line 100, in embed_with_retry
2024-05-07 09:25:57 return _embed_with_retry(**kwargs)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 289, in wrapped_f
2024-05-07 09:25:57 return self(f, *args, **kw)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 379, in __call__
2024-05-07 09:25:57 do = self.iter(retry_state=retry_state)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 325, in iter
2024-05-07 09:25:57 raise retry_exc.reraise()
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 158, in reraise
2024-05-07 09:25:57 raise self.last_attempt.result()
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
2024-05-07 09:25:57 return self.__get_result()
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
2024-05-07 09:25:57 raise self._exception
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 382, in __call__
2024-05-07 09:25:57 result = fn(*args, **kwargs)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/langchain_cohere/embeddings.py", line 98, in _embed_with_retry
2024-05-07 09:25:57 return self.client.embed(**kwargs)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/cohere/client.py", line 153, in embed
2024-05-07 09:25:57 return merge_embed_responses(responses)
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/cohere/utils.py", line 187, in merge_embed_responses
2024-05-07 09:25:57 meta = merge_meta_field([response.meta for response in responses if response.meta])
2024-05-07 09:25:57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-07 09:25:57 File "/workspace/.venv/lib/python3.11/site-packages/cohere/utils.py", line 167, in merge_meta_field
2024-05-07 09:25:57 api_version = metas[0].api_version
2024-05-07 09:25:57 ~~~~~^^^
2024-05-07 09:25:57 IndexError: list index out of range
These two may be separate issues, but I think they may have been caused by me running Chrome in a custom way.
If second issue is an error related to Korean characters, I'd like to contribute for Korean documents.
Additional information
No response
tianjing-li commented
Hi @SeaJungg! Thanks for raising this issue. We do have a CORSMiddleware being used in src/backend/main.py, by default the origins are set to [*].
With this wildcard origin being set you still encountered the issue described? Do you have any backend logs as to what happens when you run the toolkit?
SeaJungg commented
Hi @tianjing-li !
Yes as you said there is CORSMiddleware with wildcard.
There is no backend log because the request was not sent to the backend server in the browser.
Probably responsible for browser settings..
tianjing-li commented
Does the frontend attempt to call OPTIONS on the /chat-stream endpoint? What happens after? Can you provide some frontend console and network calls if possible
tianjing-li commented
Hey @SeaJungg since this has been inactive for a while, I'll go ahead and close this. If you have more questions feel free to file a new bug.