Error after connecting to chat UI and sending message (Windows)
InfernalDread opened this issue · 51 comments
Hello again,
I went through the instructions to connect to the Chat UI and it worked! However, after sending a test "Hello" message, I got the following error code on my cmd:
(gptllamaapi) C:\Users\gamet\Documents\Transfer_To_External_Hard_Drive\gpt_llama_cpp\gpt-llama.cpp>npm start
gpt-llama.cpp@0.1.8 start
node index.js
Server is listening on:
- localhost:443
- 192.168.0.34:443 (for other devices on the same network)
--LLAMA.CPP SPAWNED--
C:\Users\gamet\Documents\Transfer_To_External_Hard_Drive\llama_cpp\llama-master-f7d0509-bin-win-avx-x64\ggml-model-q4_0.bin/llama.cpp/main -m C:\Users\gamet\Documents\Transfer_To_External_Hard_Drive\llama_cpp\llama-master-f7d0509-bin-win-avx-x64\ggml-model-q4_0.bin --temp 0 --n_predict 1000 --top_p 0.1 --top_k 40 -b 512 -c 2048 --repeat_penalty 1.1764705882352942 --reverse-prompt user: --reverse-prompt
user --reverse-prompt system: --reverse-prompt
system --reverse-prompt ## --reverse-prompt
--reverse-prompt ### -i -p ### Instructions
Complete the following chat conversation between the user and the assistant. System messages should be strictly followed as additional instructions.
Inputs
system: You are a helpful assistant.
user: How are you?
assistant: Hi, how may I help you today?
system: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.
Response
user: Hello
assistant:
--REQUEST--
user: Hello
node:events:515
throw er; // Unhandled 'error' event
^
Error: spawn C:\Users\gamet\Documents\Transfer_To_External_Hard_Drive\llama_cpp\llama-master-f7d0509-bin-win-avx-x64\ggml-model-q4_0.bin/llama.cpp/main ENOENT
at ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Emitted 'error' event on ChildProcess instance at:
at ChildProcess._handle.onexit (node:internal/child_process:289:12)
at onErrorNT (node:internal/child_process:476:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
errno: -4058,
code: 'ENOENT',
syscall: 'spawn C:\Users\gamet\Documents\Transfer_To_External_Hard_Drive\llama_cpp\llama-master-f7d0509-bin-win-avx-x64\ggml-model-q4_0.bin/llama.cpp/main',
path: 'C:\Users\gamet\Documents\Transfer_To_External_Hard_Drive\llama_cpp\llama-master-f7d0509-bin-win-avx-x64\ggml-model-q4_0.bin/llama.cpp/main',
spawnargs: [
'-m',
'C:\Users\gamet\Documents\Transfer_To_External_Hard_Drive\llama_cpp\llama-master-f7d0509-bin-win-avx-x64\ggml-model-q4_0.bin',
'--temp',
0,
'--n_predict',
1000,
'--top_p',
'0.1',
'--top_k',
'40',
'-b',
'512',
'-c',
'2048',
'--repeat_penalty',
'1.1764705882352942',
'--reverse-prompt',
'user:',
'--reverse-prompt',
'\nuser',
'--reverse-prompt',
'system:',
'--reverse-prompt',
'\nsystem',
'--reverse-prompt',
'##',
'--reverse-prompt',
'\n##',
'--reverse-prompt',
'###',
'-i',
'-p',
'### Instructions\n' +
'Complete the following chat conversation between the user and the assistant. System messages should be strictly followed as additional instructions.\n' +
'\n' +
'### Inputs\n' +
'system: You are a helpful assistant.\n' +
'user: How are you?\n' +
'assistant: Hi, how may I help you today?\n' +
"system: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.\n" +
'\n' +
'### Response\n' +
'user: Hello\n' +
'assistant:'
]
}
Node.js v18.4.0
(gptllamaapi) C:\Users\gamet\Documents\Transfer_To_External_Hard_Drive\gpt_llama_cpp\gpt-llama.cpp>
o crap it must be the front and back slashes i have hardcoded.. let me make a change..
Something that was interesting was that the code would add "/llama.cpp/main" after my path for specifying the .bin file in the llama.cpp folder, I am not sure if it has anything to do with that though
oh I see, lol, similar thought processes
I think the hardcoded values are in the "utils.js" file, near the end of the code. I am assuming that you would have to remove them and include a variable to take the path that the user input?
sorry i'll push out a fix tonight that supports both windows and mac (for front and backslashes). for now, u can try replacing all the front slashes with backslashes and lmk if that works?
oh ok, I can try that, hopefully it works lol
Stumbled upon this issue as well. Will also need to change scriptPath on line 80 in chatRoutes.js
yup, made note of that just now, I will try to make the changes and see if it works
hey @brandonvessel @InfernalDread , i just pushed a change that should "fix" \ and / in paths for windows and mac in 3055f5e
do a fresh pull and lmk if it works! (i haven't published it to npm yet pending this test)
Not sure if this is on my end. Getting the following error:
const readable = new ReadableStream({
^
ReferenceError: ReadableStream is not defined
at file:///E:/Projects/_AI/gpt-llama.cpp/routes/chatRoutes.js:164:20
at Layer.handle [as handle_request] (E:\Projects\_AI\gpt-llama.cpp\node_modules\express\lib\router\layer.js:95:5)
at next (E:\Projects\_AI\gpt-llama.cpp\node_modules\express\lib\router\route.js:144:13)
at Route.dispatch (E:\Projects\_AI\gpt-llama.cpp\node_modules\express\lib\router\route.js:114:3)
at Layer.handle [as handle_request] (E:\Projects\_AI\gpt-llama.cpp\node_modules\express\lib\router\layer.js:95:5)
at E:\Projects\_AI\gpt-llama.cpp\node_modules\express\lib\router\index.js:284:15
at Function.process_params (E:\Projects\_AI\gpt-llama.cpp\node_modules\express\lib\router\index.js:346:12)
at next (E:\Projects\_AI\gpt-llama.cpp\node_modules\express\lib\router\index.js:280:10)
at Function.handle (E:\Projects\_AI\gpt-llama.cpp\node_modules\express\lib\router\index.js:175:3)
at router (E:\Projects\_AI\gpt-llama.cpp\node_modules\express\lib\router\index.js:47:12)```
hmm could you run npm i again?
No updates needed. npm version 9.6.5 if that helps. It did print the --LLAMA.CPP SPAWNED-- log message and the raw query information, but errors out on trying to read the response.
this is actually probably related to readablestream only being supported on node 16+. can you double check your node version? you can check this by doing node -v. i'm on v19.8.1
v16.14.0
Going to upgrade node and see if that fixes it
I still am getting this error (also, yes, I changed the port to 8080):
(venv) C:\Users\Mike's PC\Documents\transfer_to_external_storage\gpt_llama_cpp\gpt-llama.cpp>npm start
gpt-llama.cpp@0.1.8 start
node index.js
Server is listening on:
- localhost:8080
- 192.168.0.33:8080 (for other devices on the same network)
--LLAMA.CPP SPAWNED--
C:\Users\Mike's\llama.cpp\main -m C:\Users\Mike's --temp 0 --n_predict 1000 --top_p 0.1 --top_k 40 -b 512 -c 2048 --repeat_penalty 1.1764705882352942 --reverse-prompt user: --reverse-prompt
user --reverse-prompt system: --reverse-prompt
system --reverse-prompt ## --reverse-prompt
--reverse-prompt ### -i -p ### Instructions
Complete the following chat conversation between the user and the assistant. System messages should be strictly followed as additional instructions.
Inputs
system: You are a helpful assistant.
user: How are you?
assistant: Hi, how may I help you today?
system: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.
Response
user: hello
assistant:
--REQUEST--
user: hello
node:events:515
throw er; // Unhandled 'error' event
^
Error: spawn C:\Users\Mike's\llama.cpp\main ENOENT
at ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Emitted 'error' event on ChildProcess instance at:
at ChildProcess._handle.onexit (node:internal/child_process:289:12)
at onErrorNT (node:internal/child_process:476:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
errno: -4058,
code: 'ENOENT',
syscall: "spawn C:\Users\Mike's\llama.cpp\main",
path: "C:\Users\Mike's\llama.cpp\main",
spawnargs: [
'-m',
"C:\Users\Mike's",
'--temp',
0,
'--n_predict',
1000,
'--top_p',
'0.1',
'--top_k',
'40',
'-b',
'512',
'-c',
'2048',
'--repeat_penalty',
'1.1764705882352942',
'--reverse-prompt',
'user:',
'--reverse-prompt',
'\nuser',
'--reverse-prompt',
'system:',
'--reverse-prompt',
'\nsystem',
'--reverse-prompt',
'##',
'--reverse-prompt',
'\n##',
'--reverse-prompt',
'###',
'-i',
'-p',
'### Instructions\n' +
'Complete the following chat conversation between the user and the assistant. System messages should be strictly followed as additional instructions.\n' +
'\n' +
'### Inputs\n' +
'system: You are a helpful assistant.\n' +
'user: How are you?\n' +
'assistant: Hi, how may I help you today?\n' +
"system: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.\n" +
'\n' +
'### Response\n' +
'user: hello\n' +
'assistant:'
]
}
Node.js v18.4.0
(venv) C:\Users\Mike's PC\Documents\transfer_to_external_storage\gpt_llama_cpp\gpt-llama.cpp>
i'm going to add a minimum node version of 18 if that fixes it. unless something else comes up that forces backwards compatibility with the experimental 16.5-17.5 version i'm going to stick with that
@keldenl That fixed it! Looks like my instance is working as expected now
@InfernalDread does running
C:\Users\Mike's\llama.cpp\main -m C:\Users\Mike's --temp 0 --n_predict 1000 --top_p 0.1 --top_k 40 -b 512 -c 2048 --repeat_penalty 1.1764705882352942 -p "the sky is"
work for you?
update: nvm, i don't see your model path at all, C:\Users\Mike's\llama.cpp\main -m C:\Users\Mike's isn't right. it should show your model path
it says path cannot be specified. I believe I correctly authorized the model. Let me check again
mine looks like this: ../llama.cpp/main -m ../llama.cpp/models/vicuna/13B/ggml-vicuna-unfiltered-13b-4bit.bin --temp 1 --n_predict 1000 --top_p 0.1 --top_k 40 -b 512 -c 2048 --repeat_penalty 1.1764705882352942
can you paste the api key you provided for your model?
C:\Users\Mike's PC\Documents\transfer_to_external_storage\llama_cpp\3\llama-master-f7d0509-bin-win-avx-x64\ggml-model-q4_0.bin
EDIT: do I have to create a specific directory to match something like yours?
ahhhh.. your llama.cpp path looks vastly different from what i expected
i believe the standard setup should be
\llama.cpp\models\ggml-model-q4_0.bin
the problematic parts of your path is the fact that i rely on
llama.cppbeing the project folder name- models being stored in
llama.cpp\models\<HERE>helps, but not required. it relies on llama.cpp being in the path tho
any chance you could tweak your path and folder naming to fix that?
oh ya, of course I can, let me get on that right now!
all the other ".exe" files like "main.exe" should also be there too right?
did you have to include .exe when getting it working with llama.cpp? what's an example command you run in cmd to get llama.cpp working (not gpt-llama.cpp)
I actually do not remember if I had to, I haven't used llama.cpp in a while, but I know that it worked
I am now using this as the model path:
C:\Users\Mike's PC\Documents\llama.cpp\models\ggml-model-q4_0.bin
will this work better?
EDIT: I forgot to include the ".bin" in a folder in the models folder, does that matter?
I think it would be best for me to try and reinstall everything over again
interesting, even after completely installing it fresh, and using this path:
C:\Users\Mike's PC\Documents\llama.cpp\models\vicuna_13b\ggml-model-q4_0.bin
I still get this error:
(venv) C:\Users\Mike's PC\Documents\transfer_to_external_storage\gpt_llama_cpp\gpt-llama.cpp>npm start
gpt-llama.cpp@0.1.8 start
node index.js
Server is listening on:
- localhost:8080
- 192.168.0.33:8080 (for other devices on the same network)
--LLAMA.CPP SPAWNED--
C:\Users\Mike's\llama.cpp\main -m C:\Users\Mike's --temp 0 --n_predict 1000 --top_p 0.1 --top_k 40 -b 512 -c 2048 --repeat_penalty 1.1764705882352942 --reverse-prompt user: --reverse-prompt
user --reverse-prompt system: --reverse-prompt
system --reverse-prompt ## --reverse-prompt
--reverse-prompt ### -i -p ### Instructions
Complete the following chat conversation between the user and the assistant. System messages should be strictly followed as additional instructions.
Inputs
system: You are a helpful assistant.
user: How are you?
assistant: Hi, how may I help you today?
system: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.
Response
user: hello
assistant:
--REQUEST--
user: hello
node:events:515
throw er; // Unhandled 'error' event
^
Error: spawn C:\Users\Mike's\llama.cpp\main ENOENT
at ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Emitted 'error' event on ChildProcess instance at:
at ChildProcess._handle.onexit (node:internal/child_process:289:12)
at onErrorNT (node:internal/child_process:476:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
errno: -4058,
code: 'ENOENT',
syscall: "spawn C:\Users\Mike's\llama.cpp\main",
path: "C:\Users\Mike's\llama.cpp\main",
spawnargs: [
'-m',
"C:\Users\Mike's",
'--temp',
0,
'--n_predict',
1000,
'--top_p',
'0.1',
'--top_k',
'40',
'-b',
'512',
'-c',
'2048',
'--repeat_penalty',
'1.1764705882352942',
'--reverse-prompt',
'user:',
'--reverse-prompt',
'\nuser',
'--reverse-prompt',
'system:',
'--reverse-prompt',
'\nsystem',
'--reverse-prompt',
'##',
'--reverse-prompt',
'\n##',
'--reverse-prompt',
'###',
'-i',
'-p',
'### Instructions\n' +
'Complete the following chat conversation between the user and the assistant. System messages should be strictly followed as additional instructions.\n' +
'\n' +
'### Inputs\n' +
'system: You are a helpful assistant.\n' +
'user: How are you?\n' +
'assistant: Hi, how may I help you today?\n' +
"system: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.\n" +
'\n' +
'### Response\n' +
'user: hello\n' +
'assistant:'
]
}
Node.js v18.4.0
(venv) C:\Users\Mike's PC\Documents\transfer_to_external_storage\gpt_llama_cpp\gpt-llama.cpp>
FINALLY, so apparently, this program does NOT like spaces in the path. I had to put this in a path without a single space. I know that this can happen at times with coding.
This is the final path:
C:\Users\Mike's\llama.cpp\models\vicuna_13b\ggml-model-q4_0.bin
so, you probably want to include in the README file, to not have ANY spaces in your path
Also, one last thing, I got a timeout error from the chat UI cmd, but the response still showed up on the gpt-llama.cpp cmd, so this issue is resolved, but it is weird that I got a timeout error from chat UI
EDIT: And now Chat UI is working fine.......I am confused, but happy confused.
Thank you for all of your help!
glad you figured it out! hopefully this thread will be useful for other folks if they run into similar issues – sorry i had to step away to make some dinner :/
regarding the spaces.. let me see if there's a way around it, there may be a bug perhaps
glad you figured it out! hopefully this thread will be useful for other folks if they run into similar issues – sorry i had to step away to make some dinner :/
regarding the spaces.. let me see if there's a way around it, there may be a bug perhaps
oh no, don't worry about it, we all gotta eat lol. As for the spaces, it's not the worst thing to deal with, but I am glad that this issue was resolved. Hopefully it is something that can be fixed easily, if not, no big deal!
I think I may know why this happened, maybe you have it in your code to always use the %username% of the host machine in the path variable. Hence why, no matter what previous path I chose, it always defaulted to my username.
hey @InfernalDread weird question but how did you get llama.cpp set up? do you have a main file? somebody else is trying to get it working and seem to be stuck on not having the main file
Oh, no problem. So, I went to the main llama.cpp repo and clicked on the releases button. Then, I downloaded the avx zip file that doesn't have a number attached to it (should be the first file of the assets). I unzipped the file and clicked into it and bam, all the .exe files that you need (including of course the "main.exe" file. Then I moved them to a separate folder named "llama.cpp" and created the "models" where I put my folder with the ".bin" file in it. If you need more clarification and or a visual representation, fill free to ask!
are you running on cmd, powershell or git bash or something else?
are you running on cmd, powershell or git bash or something else?
I am running fully on CMD using "venv" python virtual environment
got it! thank you :) also do you think if i made a discord channel it'd be helpful?
got it! thank you :) also do you think if i made a discord channel it'd be helpful?
I'd think so, this way more people can either contribute or ask questions without bloating the repo, but it's up to you of course! I'm just glad I can be of assistance!
Hello guys, how are you doing? Firstly, great job here keldenl! Your work is amazing! And thanks for all the help you gave him InfernalDread.
I'm stuck in the same error as you InfernalDread:
node:events:489
throw er; // Unhandled 'error' event
^
Error: spawn C:/Users/Vitor/Desktop/Hax_Prog/llama.cpp/main ENOENT
at ChildProcess._handle.onexit (node:internal/child_process:285:19)
at onErrorNT (node:internal/child_process:483:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Emitted 'error' event on ChildProcess instance at:
at ChildProcess._handle.onexit (node:internal/child_process:291:12)
at onErrorNT (node:internal/child_process:483:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
errno: -2,
code: 'ENOENT',
syscall: 'spawn C:/Users/Vitor/Desktop/Hax_Prog/llama.cpp/main',
path: 'C:/Users/Vitor/Desktop/Hax_Prog/llama.cpp/main',
spawnargs: [
'-m',
'C:/Users/Vitor/Desktop/Hax_Prog/llama.cpp/models/ggml-vicuna-13b-4bit-rev1.bin',
'--temp',
1,
'--n_predict',
1000,
'--top_p',
'0.1',
'--top_k',
'40',
'-b',
'512',
'-c',
'2048',
'--repeat_penalty',
'1.1764705882352942',
'--reverse-prompt',
'user:',
'--reverse-prompt',
'\nuser',
'--reverse-prompt',
'system:',
'--reverse-prompt',
'\nsystem',
'--reverse-prompt',
'##',
'--reverse-prompt',
'\n##',
'--reverse-prompt',
'###',
'-i',
'-p'
You have mentioned the spaces between the folders in the path, I've checked that already and it's everything ok, any toughts?
Hello guys, how are you doing? Firstly, great job here keldenl! Your work is amazing! And thanks for all the help you gave him InfernalDread.
I'm stuck in the same error as you InfernalDread:
node:events:489 throw er; // Unhandled 'error' event ^ Error: spawn C:/Users/Vitor/Desktop/Hax_Prog/llama.cpp/main ENOENT at ChildProcess._handle.onexit (node:internal/child_process:285:19) at onErrorNT (node:internal/child_process:483:16) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) Emitted 'error' event on ChildProcess instance at: at ChildProcess._handle.onexit (node:internal/child_process:291:12) at onErrorNT (node:internal/child_process:483:16) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { errno: -2, code: 'ENOENT', syscall: 'spawn C:/Users/Vitor/Desktop/Hax_Prog/llama.cpp/main', path: 'C:/Users/Vitor/Desktop/Hax_Prog/llama.cpp/main', spawnargs: [ '-m', 'C:/Users/Vitor/Desktop/Hax_Prog/llama.cpp/models/ggml-vicuna-13b-4bit-rev1.bin', '--temp', 1, '--n_predict', 1000, '--top_p', '0.1', '--top_k', '40', '-b', '512', '-c', '2048', '--repeat_penalty', '1.1764705882352942', '--reverse-prompt', 'user:', '--reverse-prompt', '\nuser', '--reverse-prompt', 'system:', '--reverse-prompt', '\nsystem', '--reverse-prompt', '##', '--reverse-prompt', '\n##', '--reverse-prompt', '###', '-i', '-p'You have mentioned the spaces between the folders in the path, I've checked that already and it's everything ok, any toughts?
What is your node version? It needs to be 18+ to work. Do "node -v" and tell me what the output is.
@Vitorhsantos i think you're missing a / in your path, it should start with C:// right?
also @InfernalDread and @Vitorhsantos , i just made a discord channel that hopefully avoids a 50 reply long thread on GitHub (but hopefully we'll still have actual issues posted here too): https://discord.gg/aWHBQnJaFC come through!
@InfernalDread It says Node.js v20.0.0
@keldenl I've tried both options, same error, any other toughts?
@Vitorhsantos wanna join the discord we could try to figure it out there faster
@InfernalDread It says Node.js v20.0.0
@keldenl I've tried both options, same error, any other toughts?
You made sure to authenticate and use the same model structure and path as in the README?
Actually I got lost in that part, how can I authenticate properly? I would really appreciate it if you could explain it to me
Actually I got lost in that part, how can I authenticate properly? I would really appreciate it if you could explain it to me
Join the discord, it'll be easier for us to help you there!
