undreamai/LLMUnity

Persist user interaction to disk so that it can be continued

Closed this issue · 1 comments

llama.cpp allows to store the conversation along with the processed tokens to disk so that the interaction can be resumed at the next run.
Purpose of this feature is to enable this functionality in LLMUnity.

This is blocked at the moment as the llama.cpp server doesn't expose this functionality