After the new version all local models are unavailable, what ever you ask it will talk about a different topic and is completely uncommunicative.

Question

After the new version all local models are unavailable, what ever you ask it will talk about a different topic and is completely uncommunicative.

ggiidd opened this issue 5 months ago · 2 comments

Answer 1 · 2024-06-01T04:35:31.000Z

Ill look into it. It must be an issue with llama_apply_chat_template

Answer 2 · 2024-06-01T09:41:26.000Z

@ggiidd I dug into the code and it seems the gibberish is occuring because if nPredict is more than or equal to nCtx all the data in the preprompt gets dropped. I can fix this in the next release but in the meantime you should make sure nCtx is set to a value much higher than nPredict