Utilize sessions such that client doesn't need to upload megabytes per chat request
grymmy opened this issue · 1 comments
grymmy commented
Describe the bug
Right now the implementation of the LLM chat feature requires excessive uploads from the web browser client, making it not very useful on low bandwidth/high latency connections.
Expected behavior
Theoretically sessions could be used to ensure we don't need to reupload context multiple times across requests.
JosiasAurel commented
implemented since hackclub/llm-api#4