[Feature]: Support Anthropic API caching in Rivet
justinwetch opened this issue · 3 comments
justinwetch commented
Feature Request
Very recently, Anthropic has added support for API caching which can save on input token costs by up to 90%. Would be a huge boon for using Claude models in Rivet. docs: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
Code of Conduct
- I agree to follow this project's Code of Conduct
manojks1999 commented
Hi @justinwetch , can I work on this?
veloisgood commented
this would be extremely useful. I'm extremely interested.
abrenneke commented
Try running from source, I've implemented prompt caching into the latest commit