Validation error for GPT4All when using Llama2

Question

Validation error for GPT4All when using Llama2

oparoz opened this issue a year ago · 13 comments

I'm running Ubuntu and made sure there were no errors at install time
I downloaded exclusively the Llama2 model
I selected the Llama2 model in the admin section and all flags are green
Using the assistant, I asked for a summary of a text
A few minutes later, I get a notification that the process had failed
In the logs, I see this:

Warning | llm | Invalid model file Traceback (most recent call last):   File "/nextcloud/apps/llm/src-py/index.py", line 23, in <module>     llm = GPT4All(model=dir_path + "/models/"+args.model, n_threads=int(threads))   File "/var/www/nextcloud/apps/llm/python/lib/python3.10/site-packages/langchain/load/serializable.py", line 74, in __init__     super().__init__(**kwargs)   File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__ pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All __root__   Unable to instantiate model (type=value_error)

From a user perspective, it looks like the wrong model is being used

Answer 1 · 2023-09-23T17:07:51.000Z

I installed the GPT4All model and get the same error, so not sure what is going on.

Answer 2 · 2023-10-10T12:22:54.000Z

Hi, I'm sorry for the delay. I'll look into this.

From a user perspective, it looks like the wrong model is being used

This is likely not the case. GPT4All is the library and gpt4all falcon is the model, they are separate.

Answer 3 · 2023-10-10T13:00:39.000Z

Which OS, which CPU architecture?

Answer 4 · 2023-10-22T21:24:05.000Z

Thank you for looking into this.

OS: Ubuntu 22.04.3 LTS
Arch: amd64

I updated to the latest version and the error now looks different:

[llm] Warning:  /nextcloud/apps/llm/python/lib/python3.10/site-packages/langchain/__init__.py:40: UserWarning: Importing BasePromptTemplate from langchain root module is no longer supported.
  warnings.warn(
 /nextcloud/apps/llm/python/lib/python3.10/site-packages/langchain/__init__.py:40: UserWarning: Importing PromptTemplate from langchain root module is no longer supported.
  warnings.warn(
Invalid model file
Traceback (most recent call last):
  File " /nextcloud/apps/llm/src-py/index.py", line 23, in <module>
    llm = GPT4All(model=dir_path + "/models/"+args.model, n_threads=int(threads))
  File " /nextcloud/apps/llm/python/lib/python3.10/site-packages/langchain/load/serializable.py", line 97, in __init__
    super().__init__(**kwargs)
  File " /nextcloud/apps/llm/python/lib/python3.10/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
  Unable to instantiate model (type=value_error)

Answer 5 · 2023-10-27T12:10:13.000Z

I encounter the same error. I am running Nextcloud in a Docker container on a VPS. I suspect it has to do with my VPS provider not passing AVX-CPU-flags to the server. These are required for the GPT4ALL ecosystem though

Answer 6 · 2023-10-29T09:37:22.000Z

AVX-CPU-flags [...] These are required for the GPT4ALL ecosystem though

This is not in the README. Is this a requirement for using these LLMs? That could explain my problem as the Celeron in this box doesn't support those instructions.

Answer 7 · 2023-10-30T10:11:28.000Z

Yeah, it's missing in the README of this app, but its a requirement of the underlying architecture, see here: https://github.com/nomic-ai/gpt4all

If this is indeed the cause of the problem, it should probably be in the README of this app as well.

Answer 8 · 2023-11-11T12:28:16.000Z

Thank you so much for highlighting this. I won't try again until I have migrated to a more powerful CPU :)

Answer 9 · 2023-11-11T23:11:20.000Z

I have a Ryzen CPU, which does have AVX2 support. I am also getting a similar error, no matter which of the three models I choose.

{
	"reqId": "h5ZekSOB3CR3x587ry4n",
	"level": 2,
	"time": "2023-11-11T22:58:05+00:00",
	"remoteAddr": "",
	"user": "--",
	"app": "llm",
	"method": "",
	"url": "--",
	"message": "/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/__init__.py:40: UserWarning: Importing BasePromptTemplate from langchain root module is no longer supported.\n  warnings.warn(\n/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/__init__.py:40: UserWarning: Importing PromptTemplate from langchain root module is no longer supported.\n  warnings.warn(\nTraceback (most recent call last):\n  File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/src-py/index.py\", line 23, in <module>\n    llm = GPT4All(model=dir_path + \"/models/\"+args.model, n_threads=int(threads))\n          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/load/serializable.py\", line 97, in __init__\n    super().__init__(**kwargs)\n  File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/pydantic/v1/main.py\", line 341, in __init__\n    raise validation_error\npydantic.v1.error_wrappers.ValidationError: 1 validation error for GPT4All\n__root__\n  Failed to retrieve model (type=value_error)\n",
	"userAgent": "--",
	"version": "27.1.3.2",
	"data": { "app": "llm" }
}

/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/__init__.py:40: UserWarning: Importing BasePromptTemplate from langchain root module is no longer supported.
warnings.warn(
/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/__init__.py:40: UserWarning: Importing PromptTemplate from langchain root module is no longer supported.
warnings.warn(
Traceback (most recent call last):
File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/src-py/index.py\", line 23, in <module>
llm = GPT4All(model=dir_path + \"/models/\"+args.model, n_threads=int(threads))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/load/serializable.py\", line 97, in __init__
super().__init__(**kwargs)
File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/pydantic/v1/main.py\", line 341, in __init__
raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
Failed to retrieve model (type=value_error)

The downloaded models have the following sha256:

259a403e5c4f098936377a62b8d6f05d3c99bb9aebe36166c13bf3cdd5bcf20b  gpt4all-falcon-q4_0.gguf
e4dcdde73bd4ea6975a588c0ace974734cd7419d68ba5d58efa12b37f7eb802f  leo-hessianai-13b-chat-bilingual.Q4_K_M.gguf
08a5566d61d7cb6b420c3e4387a39e0078e1f2fe5f055f3a03887385304d4bfa  llama-2-7b-chat.Q4_K_M.gguf

Answer 10 · 2023-11-12T22:50:23.000Z

I had the same issue and after strace-ing the cron.php job after using the assistant, I found out that the code was looking for a .bin file, not the .gguf file in the apps/llm/src-py/models folder.

So I downloaded the file ggml-model-gpt4all-falcon-q4_0.bin, and renamed it to gpt4all-falcon-q4_0.gguf.bin to match the name of the model (but with a .bin extension added). It worked.

I don't know if the format between the two files are different or if it is just a naming issue.

Answer 11 · 2023-11-13T16:02:42.000Z

These are two different errors though, difference is in the last line. The first is: "Unable to instantiate model (type=value_error)", which in this case I think has to do with missing AVX-support. The second says "Failed to retrieve model (type=value_error)", also reported here #39

Answer 12 · 2023-11-15T13:48:29.000Z

Yes, #39 should be fixed with v1.2.1 now

Answer 13 · 2023-11-29T08:55:09.000Z

I've added the AVX requirement to the README now