julep-ai/julep

Add a xxhash based cache

Closed this issue · 0 comments

Use xxhash for hashing, create a cozo table for saving these. lookup before every call to openai

xxhash(json.dumps({ chatml, parameters })) => cache lookup => openai

the hash is the key, save the response as the value