dmarx/textbooks-dataset

Curated dataset of open source textbooks

Curated Open Textbooks

An attempt to construct a heavily opinionated, tightly curated dataset of core knowledge. Focus will be textbooks, but may also include lecture transcripts/slides, course notes, etc.

Motivation

Goal is to build an extremely information-dense dataset for model finetuning in support of STEM research activities.

Links will be preferntially to PDFs for standardization. Source code should be linked where available, but for now I'm operating under the assumption that everything will need to go through a PDF processing pipeline.

My main focus here is colelcting materials for machine processing, but I am also including "portal" pages to e.g. help human learniners find associated course content and/or more human-readable presentation.

Textbooks

• 2021 - The Principles of Deep Learning Theory

  • https://arxiv.org/abs/2106.10165

• 2021 - Geometric Deep Learning

  • https://arxiv.org/pdf/2104.13478.pdf
  • source - https://arxiv.org/src/2104.13478
  • portal - https://geometricdeeplearning.com/
  • lectures and slides - https://geometricdeeplearning.com/lectures/

• 2017 - Elements of Statistical Learning

  • https://hastie.su.domains/ElemStatLearn/printings/ESLII_print12_toc.pdf
  • portal - https://hastie.su.domains/ElemStatLearn/

• 2022 - Probabilistic Machine Learning: An Introduction (Bishop 1)

  • https://github.com/probml/pml-book/releases/latest/download/book1.pdf
  • portal https://probml.github.io/pml-book/book1.html
  • project portal - https://github.com/probml/pml-book/

• 2023 - Probabilistic Machine Learning: Advanced Topics (Bishop 2)

  • https://github.com/probml/pml2-book/releases/latest/download/book2.pdf
  • portal - https://probml.github.io/pml-book/book2.html
  • project portal - https://github.com/probml/pml-book/

• CS228 - Probabilistic Graphical Models

  • https://github.com/ermongroup/cs228-notes
  • portal - https://cs.stanford.edu/~ermon/cs228/index.html

• 2021 - Bayesian Statistics With Julia and Turing

  • https://github.com/storopoli/Bayesian-Julia
  • online - https://storopoli.io/Bayesian-Julia/

• Make A Lisp

  • https://github.com/kanaka/mal
  • NB: textbook has been re-implemented in 87 different programming languages. Coding Rosetta Stone.

• 2013 - Probabilistic Programming and Bayesian Methods for Hackers

  • https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers

• 1992 - Paradigms of AI Programming (Norvig)

  • https://github.com/norvig/paip-lisp

• Category Theory for Programmers

  • https://github.com/hmemcpy/milewski-ctfp-pdf/releases/tag/v1.3.0
  • portal - https://bartoszmilewski.com/2014/10/28/category-theory-for-programmers-the-preface/

• 1996 - structure and interpretation of computer programs (SICP, Abelman and Sussman)

  • https://web.mit.edu/6.001/6.037/sicp.pdf
  • https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/index.html

• 2021 - An Introduction to Johnson–Lindenstrauss Transforms

  • https://arxiv.org/pdf/2103.00564.pdf

Needs Categorization

• https://github.com/huggingface/course
• https://github.com/mlc-ai/mlc-en

Misc Intro Deep Learning

Putting these all under a single bucket because I anticipate there to be a lot of repeated content across these.

• 2023 - UVA Deep learning

  • https://github.com/phlippe/uvadlc_notebooks
  • portal - https://uvadlc-notebooks.readthedocs.io/en/latest/

• 2023 - Dive Into Deep Learning

  • https://github.com/d2l-ai/d2l-en
  • portal - https://d2l.ai/

• 2020 - FastAI Book

  • https://github.com/fastai/fastbook

• 2023 - Understanding Deep Learning (Prince)

  • https://github.com/udlbook/udlbook/releases/download/v2.05/UnderstandingDeepLearning_04_18_24_C.pdf
  • https://github.com/udlbook/udlbook/
  • https://udlbook.github.io/udlbook/

• 2020 - An Elementary Introduction to Information Geometry

  • https://www.mdpi.com/1099-4300/22/10/1100
  • https://www.mdpi.com/1099-4300/22/10/1100/pdf?version=1603379009

Limited License

If it's in this section, it's probably only licensed for personal use or the licensing is sufficiently ambiguous that you may not want to train on content in this section. Double check before using any of these.

• 2023 - Bayesian Optimization

  • https://github.com/bayesoptbook/bayesoptbook.github.io?tab=readme-ov-file
  • portal - https://bayesoptbook.com/

• 2016 - Deep Learning (Goodfellow, Bengio, Courville)

  • https://github.com/janishar/mit-deep-learning-book-pdf/raw/master/complete-book-pdf/deeplearningbook.pdf
  • portal - https://www.deeplearningbook.org/

• 2021 - Numerical Methods for Differential Equations with Python

  • No license listed for certain repos, others are MIT licensed. Probably fine, putting it in limited license section just due to the ambiguity.
  • https://johnsbutler.netlify.app/files/Teaching/Numerical_Analysis_for_Differential_Equations.pdf
  • https://github.com/john-s-butler-dit/Numerical-Analysis-Python
  • https://github.com/john-s-butler-dit/NumericalAnalysisBook

• 2003 - Information Theory, Inference, and Learning Algorithms

  • http://www.inference.org.uk/itprnn/book.pdf
  • portal https://www.inference.org.uk/mackay/itila/book.html

• 2008 - Graphical Models, Exponential Families, and Variational Inference

  • https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf
  • alt - https://statistics.berkeley.edu/tech-reports/649
  • source - https://statistics.berkeley.edu/sites/default/files/tech-reports/649.ps.Z
  • Published as a paper, publisher doesn't permit downlaod w/o subscription. Licensing general unclear.

• 2012 - Bayesian Reasoning And Machine Learning

  • http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/140324.pdf
  • http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.HomePage

• 2019 - Applied Stochastic Differential Equations

  • https://users.aalto.fi/~asolin/sde-book/sde-book.pdf
  • code - https://github.com/AaltoML/SDE

• Physically Based Rendering

  • Book - https://pbr-book.org/4ed/contents
  • Code - https://github.com/mmp/pbrt-v4

• Sutton and Bartow RL

  • https://inst.eecs.berkeley.edu/~cs188/sp20/assets/files/SuttonBartoIPRLBook2ndEd.pdf
  • https://mitpress.mit.edu/9780262039246/reinforcement-learning/
  • CC-BY-NC

• Linear Algebra Done Right (Axler)

  • https://linear.axler.net/LADR4e.pdf
  • https://linear.axler.net/
  • CC-BY-NC

• 2004 - Lectures on Differential Geometry

  • https://mysite.science.uottawa.ca/rossmann/Differential%20Geometry%20book_files/Diffgeo.pdf
  • https://mysite.science.uottawa.ca/rossmann/Differential%20Geometry%20book.htm
  • License not specified

• 2022 - Intro to Differential Geometry I (Salamon)

  • https://people.math.ethz.ch/~salamon/PREPRINTS/diffgeo.pdf
  • Lots more lecture notes in book form - https://people.math.ethz.ch/~salamon/

• 2022 - Intro to Differential Geometry II (Salamon)

  • https://people.math.ethz.ch/~salamon/PREPRINTS/difftop.pdf
  • https://people.math.ethz.ch/~salamon/

• 2020 - Measure and Integration (Salamon)

  • https://people.math.ethz.ch/~salamon/PREPRINTS/measure.pdf
  • https://people.math.ethz.ch/~salamon/

• 2015 - Functional Analysis (Salamon) *https://people.math.ethz.ch/~salamon/PREPRINTS/funcana-ams.pdf

  • https://people.math.ethz.ch/~salamon/

• 2023 - Discrete Differential Geometry

  • https://www.cs.cmu.edu/~kmcrane/Projects/DDG/paper.pdf

• 2023 - Stanford Encyclopedia of Philosophy

  • https://huggingface.co/datasets/dmarx/stanford-encyclopedia-of-philosophy_dec23
  • I think if I reach out to the SEP and ask they'd be cool with this kind of application of their data, but I'm not confident this falls under their current terms of use. Buyer beware.

Listicles

• https://github.com/stars/dmarx/lists/pedagogical - This project is largely seeded by the content from this list

• https://github.com/dmarx/bench-warmers

• https://github.com/labmlai/annotated_deep_learning_paper_implementations

• https://github.com/craffel/dl3d-seminar

• https://github.com/floodsung/Deep-Learning-Papers-Reading-Roadmap

• https://scorebasedgenerativemodeling.github.io/

• 8 part (8 papers) RL course with code demos for each - https://github.com/Curt-Park/rainbow-is-all-you-need

• Lots of courses (youtube), last updated Nov 2022 https://deep-learning-drizzle.github.io/

• https://github.com/codecrafters-io/build-your-own-x

• https://github.com/tmabraham/diffusion_reading_group

• https://github.com/nocotan/awesome-information-geometry

• https://github.com/swsnu/aisys2023

• https://github.com/irregular-rhomboid/EAI-Math-Reading-Group

TODO

• Topology
• Info Geo
• Strogatz
• Allen B Downey
• Graph Theory
• Network Theory
• Ecology
• Systems Theory
• Risk Management
• Emergency Management
• Climate Science
• Molecular Biology
• Vector Calculus
• Statistical Mechanics
• Complexity
• Network Science
• Social Networks
• Game Theory
• Physical Chemistry