Interactive companion · Edition 2.0
Machine Learning from First Principles
A first course for readers who know calculus, linear algebra, and probability. Every idea comes with a picture, a derivation, twenty lines of NumPy, and a figure you can drag, step through, and break.
How to use this site
This site and the PDF book cover the same chapters with the same section, equation, and exercise numbers. The PDF is better for slow reading and printing; the site is better for building intuition: every "Try it" box in the PDF has a QR code that opens the matching interactive figure here. Chapters 1–6, 8, 10, 12–14 present the material in a compact form with derivations folded into "Show derivation" panels; the other chapters and the projects show the full text of the book.
The book covers the topics of Stanford's CS229 (Machine Learning) and CS231n (Deep Learning for Computer Vision) and the practical material of CS230; Appendix D maps every course topic to a chapter or project. Read Parts I–III in order. After that, branch out:
| Goal | Suggested path |
|---|---|
| Solid classical foundation | Chapters 1–14, Projects 1–5 and 8 |
| Fastest route to deep learning | Chapters 1–4, 6, 15–17, 20, Projects 3 and 9 |
| Computer vision | Chapters 1–6, 15–17, 19, 21–23, Projects 9–12 |
| Understand large language models | Chapters 1–4, 6, 15, 16, 18, 19, 24, Project 6 |
| Understand generative models | Chapters 1–6, 14–16, 25, Project 7 |
| Reinforcement learning and control | Chapters 1–6, 15, 27 |
The projects need Python 3.10 or newer. Install the dependencies once; every project then runs on a laptop CPU in a few minutes.
python -m venv .venv && source .venv/bin/activate
pip install -r projects/requirements.txt
Contents
Part I · Foundations
Part II · Supervised Learning
Part III · Unsupervised Learning
Part IV · Deep Learning
Part V · Computer Vision
Part VI · Frontier Techniques
Part VII · Projects
Appendices
Notation
| Symbol | Meaning |
|---|---|
| $x$, $\vx$, $\mX$ | scalar, column vector, matrix |
| $n$, $d$ | number of examples, number of features |
| $\vx^{(i)}$, $y^{(i)}$ | $i$-th training input and its label |
| $\mX\in\R^{n\times d}$ | design matrix; row $i$ is $\vx^{(i)\top}$ |
| $\vw$, $b$, $\vtheta$ | weights, bias, all parameters |
| $\hat y$ | a model's prediction |
| $\Loss(\vtheta)$, $\nabla_{\vtheta}\Loss$ | training loss and its gradient |
| $\eta$ | learning rate |
| $\E[X]$, $\Var(X)$ | expectation and variance |
| $\Normal(\vmu,\mSigma)$ | Gaussian with mean $\vmu$ and covariance $\mSigma$ |
| $\sigma(z)$ | logistic sigmoid $1/(1+e^{-z})$ |
| $\log$ | natural logarithm |