Adding a lot of messages to history, models state that we had no conversational history
Closed this issue · 5 comments
When adding to chat history with
OllamaSharp.Models.Chat.Message m = new OllamaSharp.Models.Chat.Message();
m.Role = OllamaSharp.Models.Chat.ChatRole.User;
m.Content = "whatever";
chat.Messages.Add(m);
What limits the amount of messages and their length? Is it context length times 4, ie. all messages must have less than context length times 4 combined in length?
I am asking, because when adding a lot of messages to history, the models claim that we had no previous conversation.
Seems you're running out of context length indeed you can approach this by increasing num_ctx. Please refer to the Ollama docs.
As far as I know, max context length depends on the model and is not arbitrary.
Of course it depends on the model, but Ollama, for example, limits the num_ctx by default. If you provide more context than the model can process, even num_ctx is useless ... but most models have far more context than I would worry about.
chat.Options.NumCtx scheint die Kontextlänge für die Ausgabe zu sein, hier geht es aber um die Eingabe.
It's not just for generation. As said, please check the Ollama docs, this has nothing to do with the implementation of this repository.