App crashes on loading local model.

Question

Closed this issue 6 months ago · 7 comments

Using StableLm-2-zephyr-1_6b-q1_1.gguf 6gb ram android 11.

sure

Answer 1 · 2024-05-11T13:15:26.000Z

Can you provide a link to the model file used?

Answer 2 · 2024-05-11T13:19:20.000Z

Answer 3 · 2024-05-12T23:32:11.000Z

It seems to be attempting to access memory outside its buffer, likely a llama.rn issue.

Answer 4 · 2024-05-13T05:49:20.000Z

okay, are you sending the whole conversation history as a prompt or just the current message?

Answer 5 · 2024-05-13T05:57:14.000Z

The app fills the context with the latest message history until the context limit is reached.

Answer 6 · 2024-05-13T07:18:06.000Z

Am I doing something wrong here?

No this is simply how LLMs function, they take a step to consume context and build the KV.

Next time, I would recommend not bringing up unrelated topics. This is an issue tracker not code support forum.