Various excellent open-source Large Language Model(LLM) repositories are available as of Feb 2024. This repository learns and pulls from all these great repositories to reimplement different LLM-related techniques and concepts to understand them.
Additionally, this is open-source, so that I can share it with the broader community. This repository organizes different implementations across different branches and does not strive to share code across these branches. The focus is on the techniques and concepts using simple code, not enhancing code reuse and supporting many different use cases in a single codebase. However, each branch strives to be well-designed.
| Branch Name | Description | Status |
|---|---|---|
| old_2022 | Early exploration of training Billion+ parameters LLMs using FDSP | Inactive |
| microsoft/phi | Exploration of SFT and DPO on Small Language Models using microsoft/phi | Active |