meta-math/MetaMath

May I know how you load QLoRA+LLaMA 70B with vllm?

SuperBruceJia opened this issue · 1 comments

SuperBruceJia commented
May I know how you load QLoRA+LLaMA 70B with vllm?
yulonghui commented

Hi, We first merge the qlora weights and 70B base weights by peft repo. the final weights are saved in a local dir.
After that, you can load with vllm smoothly

  • 👍1