keldenl/gpt-llama.cpp

Every Other Chat Response

msj121 opened this issue · 1 comments

msj121 commented

I get a restart on the chatRoute because the last response is recorded in the global messages like this:

{"role":"assistant","content":"\\\b\\\b \b\nUSER:\\\b\\\b \b\ntests\nassistant:\nI'm happy to assist you in finding information related to tests. What specific topic or query are you interested in?"}

Only happens every other response, which might be the oddest piece.

It appears to me that it is doing something odd by combining the previous user response with the assistant response.... Did you run into this?

Using WizardLM

msj121 commented

I altered the following line:
[line #195 of chatRoutes](

gpt-llama.cpp/routes/chatRoutes.js

Line 195 in 1c8b1c1

if (!responseStart && initData.length > initPrompt.length) {
)

to be:

if (!responseStart) {

Now it works properly.

What is that check for? Maybe it can be done in a more reliable way? Happy to make a PR but obviously that logic is for something.