Code for Investigating the Effectiveness of HyperTuning via Gisting.
Cite:
@article{phang2024hyperllama,
author = {Phang, Jason},
title = {{I}nvestigating the {E}ffectiveness of {H}yperTuning via {G}isting},
year = {2024},
journal = {arXiv preprint 2402.16817},
}
To-do (as of 02/26/2023):
- Data preparation instructions
- Tokenization scripts
- Hyperpretraining script
- Fine-tuning script
- Prefix Tuning script
- Evaluation script
- Upload model weights to HF Model Hub