A finetuning pipeline for instruct tuning Raven 14bn using QLORA 4bit and the Ditty (0.5.0+) finetuning library
See this blog post: https://blog.kinglycrow.com/rwkv-qlora-4bit/ for how this code is intended to be used, you'll need dev versions of a few major libraries to support 4bit tuning right now and I've listed these in the post along with other supporting information.
I do not offer technical support or bug fixes for this repo, so use at your own risk.