trouble generating a response
gsgoldma opened this issue · 2 comments
I've taken steps to authorize the model by pasting the local address to the model and I'm running chatbotui with the env.local set to the local model. It's connecting, but as soon as I try to generate a response, I get this error and it exits the program:
Microsoft Windows [Version 10.0.19044.2846]
(c) Microsoft Corporation. All rights reserved.
D:\gpt-llama.cpp>npm start
gpt-llama.cpp@0.2.2 start
node index.js
Server is listening on:
- http://localhost:443
- http://10.0.0.109:443 (for other devices on the same network)
See Docs
Test your installation
- double click the test-installation.ps1 (powershell) or test-installation.bat (cmd) file
See https://github.com/keldenl/gpt-llama.cpp#usage for more guidance.
REQUEST RECEIVED
PROCESSING NEXT REQUEST FOR /v1/models
PROCESS COMPLETE
REQUEST RECEIVED
PROCESSING NEXT REQUEST FOR /v1/models
REQUEST RECEIVED
PROCESSING NEXT REQUEST FOR /v1/chat/completions
===== CHAT COMPLETION REQUEST =====
===== LLAMA.CPP SPAWNED =====
1i6Ld\llama.cpp\main -m sk- --temp 1 --n_predict 1000 --top_p 0.1 --top_k 40 -b 512 -c 2048 --repeat_penalty 1.1764705882352942 --reverse-prompt user: --reverse-prompt
user --reverse-prompt system: --reverse-prompt
system --reverse-prompt ## --reverse-prompt
--reverse-prompt ### -i -p ### Instructions
Complete the following chat conversation between the user and the assistant. System messages should be strictly followed as additional instructions.
Inputs
system: You are a helpful assistant.
user: How are you?
assistant: Hi, how may I help you today?
system: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.
user: prompt 1
user: hi
Response
user: hjghj
assistant:
===== REQUEST =====
user: hjghj
node:events:491
throw er; // Unhandled 'error' event
^
Error: spawn sk-\llama.cpp\main ENOENT
at ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Emitted 'error' event on ChildProcess instance at:
at ChildProcess._handle.onexit (node:internal/child_process:289:12)
at onErrorNT (node:internal/child_process:476:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
errno: -4058,
code: 'ENOENT',
syscall: 'spawn sk-\llama.cpp\main',
path: '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\llama.cpp\main',
spawnargs: [
'-m',
'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',
'--temp',
1,
'--n_predict',
1000,
'--top_p',
'0.1',
'--top_k',
'40',
'-b',
'512',
'-c',
'2048',
'--repeat_penalty',
'1.1764705882352942',
'--reverse-prompt',
'user:',
'--reverse-prompt',
'\nuser',
'--reverse-prompt',
'system:',
'--reverse-prompt',
'\nsystem',
'--reverse-prompt',
'##',
'--reverse-prompt',
'\n##',
'--reverse-prompt',
'###',
'-i',
'-p',
'### Instructions\n' +
'Complete the following chat conversation between the user and the assistant. System messages should be strictly followed as additional instructions.\n' +
'\n' +
'### Inputs\n' +
'system: You are a helpful assistant.\n' +
'user: How are you?\n' +
'assistant: Hi, how may I help you today?\n' +
"system: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.\n" +
'user: prompt 1\n' +
'user: hi\n' +
'\n' +
'### Response\n' +
'user: hjghj\n' +
'assistant:'
]
}
Node.js v18.13.0
D:\gpt-llama.cpp>
It seems to trying to spawn some API key sk- making my PATH sk-/lllama/main instead of my d: drive. Not sure how this is happening, and I'm not sure where to edit this to fix it.
It was because I had way earlier set my open ai key in the env variables in adv settings! it was taking from that everytime.