/machine-learning-list

A curriculum for learning about foundation models, from scratch to the frontier

Elicit Machine Learning Reading List

Purpose

The purpose of this curriculum is to help new Elicit employees learn background in machine learning, with a focus on language models. I’ve tried to strike a balance between papers that are relevant for deploying ML in production and techniques that matter for longer-term scalability.

If you don’t work at Elicit yet - we’re hiring ML and software engineers.

How to read

Recommended reading order:

  1. Read “Tier 1” for all topics
  2. Read “Tier 2” for all topics
  3. Etc

✨ Added after 2024/4/1

Table of contents

Fundamentals

Introduction to machine learning

Tier 1

Tier 2

Tier 3

Transformers

Tier 1

Tier 2

Tier 3

Tier 4+

Key foundation model architectures

Tier 1

Tier 2

Tier 3

Tier 4+

Training and finetuning

Tier 2

Tier 3

Tier 4+

Reasoning and runtime strategies

In-context reasoning

Tier 2

Tier 3

Tier 4+

Task decomposition

Tier 1

Tier 2

Tier 3

Tier 4+

Debate

Tier 2

Tier 3

Tier 4+

Tool use and scaffolding

Tier 2

Tier 3

Tier 4+

Honesty, factuality, and epistemics

Tier 2

Tier 3

Tier 4+

Applications

Science

Tier 3

Tier 4+

Forecasting

Tier 3

Search and ranking

Tier 2

Tier 3

Tier 4+

ML in practice

Production deployment

Tier 1

Tier 2

Benchmarks

Tier 2

Tier 3

Tier 4+

Datasets

Tier 2

Tier 3

Advanced topics

World models and causality

Tier 3

Tier 4+

Planning

Tier 4+

Uncertainty, calibration, and active learning

Tier 2

Tier 3

Tier 4+

Interpretability and model editing

Tier 2

Tier 3

Tier 4+

Reinforcement learning

Tier 2

Tier 3

Tier 4+

The big picture

AI scaling

Tier 1

Tier 2

Tier 3

Tier 4+

AI safety

Tier 1

Tier 2

Tier 3

Tier 4+

Economic and social impacts

Tier 3

Tier 4+

Philosophy

Tier 2

Tier 4+

Maintainer

andreas@elicit.com