Add a xxhash based cache
Closed this issue · 0 comments
creatorrr commented
Use xxhash for hashing, create a cozo table for saving these. lookup before every call to openai
xxhash(json.dumps({ chatml, parameters })) => cache lookup => openai
the hash is the key, save the response as the value