/baby_lm

Pre-training language models with limited data

Primary LanguageJupyter Notebook

Pre-training language models with limited data

  • Dataset exploration
  • Tokenizer analysis
  • Baseline training
  • Tuning with task rewards