Machine Learning from First Principles

Interactive companion · Edition 2.0

Machine Learning from First Principles

A first course for readers who know calculus, linear algebra, and probability. Every idea comes with a picture, a derivation, twenty lines of NumPy, and a figure you can drag, step through, and break.

How to use this site

This site and the PDF book cover the same chapters with the same section, equation, and exercise numbers. The PDF is better for slow reading and printing; the site is better for building intuition: every "Try it" box in the PDF has a QR code that opens the matching interactive figure here. Chapters 1–6, 8, 10, 12–14 present the material in a compact form with derivations folded into "Show derivation" panels; the other chapters and the projects show the full text of the book.

The book covers the topics of Stanford's CS229 (Machine Learning) and CS231n (Deep Learning for Computer Vision) and the practical material of CS230; Appendix D maps every course topic to a chapter or project. Read Parts I–III in order. After that, branch out:

GoalSuggested path
Solid classical foundationChapters 1–14, Projects 1–5 and 8
Fastest route to deep learningChapters 1–4, 6, 15–17, 20, Projects 3 and 9
Computer visionChapters 1–6, 15–17, 19, 21–23, Projects 9–12
Understand large language modelsChapters 1–4, 6, 15, 16, 18, 19, 24, Project 6
Understand generative modelsChapters 1–6, 14–16, 25, Project 7
Reinforcement learning and controlChapters 1–6, 15, 27

The projects need Python 3.10 or newer. Install the dependencies once; every project then runs on a laptop CPU in a few minutes.

python -m venv .venv && source .venv/bin/activate
pip install -r projects/requirements.txt

Contents

Part I · Foundations

Part II · Supervised Learning

Part III · Unsupervised Learning

Part IV · Deep Learning

Part V · Computer Vision

Part VI · Frontier Techniques

Part VII · Projects

Appendices

Notation

SymbolMeaning
$x$, $\vx$, $\mX$scalar, column vector, matrix
$n$, $d$number of examples, number of features
$\vx^{(i)}$, $y^{(i)}$$i$-th training input and its label
$\mX\in\R^{n\times d}$design matrix; row $i$ is $\vx^{(i)\top}$
$\vw$, $b$, $\vtheta$weights, bias, all parameters
$\hat y$a model's prediction
$\Loss(\vtheta)$, $\nabla_{\vtheta}\Loss$training loss and its gradient
$\eta$learning rate
$\E[X]$, $\Var(X)$expectation and variance
$\Normal(\vmu,\mSigma)$Gaussian with mean $\vmu$ and covariance $\mSigma$
$\sigma(z)$logistic sigmoid $1/(1+e^{-z})$
$\log$natural logarithm
Machine Learning from First Principles · Edition 2.0 · Companion site to the PDF book.