project-baize/baize-chatbot

Features : Can it use 4 bit QLORA ?

x4080 opened this issue · 2 comments

JetRunner commented

Yes, just modify the utils.py following the example from HF blog:

model = AutoModelForCausalLM.from_pretrained("facebook/opt-350m", load_in_4bit=True, device_map="auto")

We'll update the code after testing.

  • 🎉1
x4080 commented

cool, thanks @JetRunner