smol-ai/llama-fine-tuning-hackameetup

The official repository for the Anything But Wrappers: Llama Edition Hackameetup

Fine-Tuning Llama Hackameetup 🦙

Welcome to the most kickass event for all you Actually Open AI aficionados out there: The Fine-Tuning Llama Hackathon! Llama 2 has gone open-ish source and CodeLlama is just around the corner. But we need you to finesse it further. Bring your peculiar data sets and let's build some killer chains and agents.

Hackameetup?

Think of this as a hackathon and a meetup combined. Learn, hack, and win!

Featuring

Speaker	Affiliation
Harrison Chase	CEO, Langchain
Robert Nishihara	CEO, Anyscale
Ben Firshman	CEO, Replicate
Will Bryk	CEO, Metaphor
Michael Royzen	CEO, Phind
Mark Huang	CEO, Gradient AI
Yuze Ma	Engineer @ Lepton AI
Sam L'Huillier	Finetune God @ Brev.dev

Schedule 🗓

• 9:00am: Meet & Greet
• 9:30am: Talks & Intro to Fine-tuning
• 12:00pm: Lunch & Teamforming
• 1:00pm: Hacking Begins
• 4:00pm: Presentations & Awards

Prize Tracks:

🎨 Most Creative

💸 Most Expensive

🤝 Most Practical

No cash prizes, just bragging rights

Finetune Ideas:

• Structured Data
  • YAML
  • JSON
  • Markdown tables
• Code
  • finetune for Python, JS, React, Other code languages
  • finetune for JS→ TypeScript, Python → typehints?
  • finetune CodeLlama
• Text Generation
  • Creative writing
  • Style Replicas - imitate a particular person’s style
• Other forms of creative output
  • drawing ascii art - pator
  • poetry? rap? Music?
  • finetune
• Newer Techniques
  • Gorilla reproduction
  • RETRO reproduction
  • SequenceMatch reproduction
  • CodeLlama reproduction

Example Datasets 📊

Check out the example datasets in this repo.

• astrology-dataset.jsonl
• socialmediamanager-dataset.jsonl
• please PR more!

Getting Started

• https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples
• https://brev.dev/blog/fine-tuning-llama-2
• https://replicate.com/blog/fine-tune-llama-2
• https://huggingface.co/docs/transformers/training
• Talk: WandB on Training and Fine-tuning LLMs
• Notebook: fine-tune LLaMA 2 with qlora via matt shumer
• (Please PR more!)

Prizes 🏆

We've got awards for the weirdest, most practical, and most creative applications of fine-tuning. But remember, the real prize is the friends finetunes we made along the way!

Sponsors & Hosts 🤝

• Hosted by: brev.dev, ai.engineer, celo.org - Contact Nader or Swyx.
• Special Thanks: Amplify & Celo for being awesome sponsors

Code of Conduct 📜

Be cool and respectful. Check the full code of conduct here.

---

Questions? Hit us up! Looking forward to finetuning some llamas with y'all!