yhyu13/Awesome-AI-KnowWhys

Fundamental Researchers/Discoveries on How/Why AI works!

List of Content

• Physics of Language Models

  • Physics of Language Models: Part 1, Context-Free Grammar
  • Physics of Language Models: Part 3.2, Knowledge Manipulation
  • Physics of Language Models: Part 3.1, Knowledge Storage and Extraction

• Tensor Programs

  • Tensor Programs III: Neural Matrix Laws
  • Tensor Programs II: Neural Tangent Kernel for Any Architecture
  • Tensor Programs IVb: Adaptive Optimization in the Infinite-Width Limit
  • Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks
  • Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes
  • Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
  • Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer

• OpenAI

  • 2018,2019 papers on GPT & Scaling laws