rashadphz/farfalle

500 error: json: cannot unmarshal string into Go struct field ChatRequest.messages of type api.ToolCallFunctionArguments

Opened this issue · 2 comments

lrq3000 commented

I get the following error when I try to do a search (especially when in Expert mode):

500: {"error":"json: cannot unmarshal string into Go struct field ChatRequest.messages of type api.ToolCallFunctionArguments"}

I am using the Docker approach and ollama and phi3:14b on Windows 11.

jandoerntlein commented

Same error here. To me this looks like an issue on ollama side maybe? Refer to ollama/ollama#5909

Update: As the ollama issue suggests, I tried to change the litellm endpoint:

-    ChatModel.LOCAL_LLAMA_3: "ollama_chat/llama3",
+    ChatModel.LOCAL_LLAMA_3: "ollama/llama3.1",

This now results in the following error

[...]
File "/workspace/.venv/lib/python3.12/site-packages/instructor/retry.py", line 173, in retry_sync
    raise InstructorRetryException(
instructor.retry.InstructorRetryException: 1 validation error for RelatedQueries
related_questions
  Input should be a valid array [type=list_type, input_value='[', input_type=str]
    For further information visit https://errors.pydantic.dev/2.9/v/list_type

Looks like the input_value='[' is passed incomplete, instead of '[]' if empty?

A simple fix for this issue (for me) seems to just not use llama3. Gemma2 works fine for me. I did not try phi3.

(Sidenote: I upgraded to python3.12 and pydantic 2.9.0)

jandoerntlein commented

Found a workaround to fix the issue... give it a try.

0001-fix-issues-89-unmarshallling-string-failed.patch

  • 👍1