Appendix A
Python and NumPy Crash Course
Exactly the Python needed for the code in this book. If you know another programming language, an afternoon with this page and an interpreter is enough. Type the examples; do not just read them.
A.1Setting Up
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -r projects/requirements.txt
Run a script with python script.py, or experiment in python, IPython, or Jupyter. Run all project tests with python -m pytest -q from projects/.
A.2Python Essentials
x = 3 # int
y = 2.5 # float
name = "digits" # str; f-strings format values:
print(f"{name}: {x * y:.2f}")
xs = [3, 1, 4, 1, 5] # list: ordered, mutable
xs.append(9); xs[0] = 2
print(xs[1:3], xs[-1]) # slicing [start:stop), negative index from the end
point = (1.0, 2.0) # tuple: immutable
a, b = point # unpacking
counts = {"spam": 747, "ham": 4827} # dict: key -> value
counts["total"] = counts["spam"] + counts["ham"]
squares = [k * k for k in range(5)] # list comprehension
evens = {k: k * k for k in range(10) if k % 2 == 0}
def mse(pred, target): # function
"""Mean squared error of two equal-length lists."""
return sum((p - t) ** 2 for p, t in zip(pred, target)) / len(pred)
class Counter: # class with state
def __init__(self):
self.n = 0
def step(self, k=1):
self.n += k
return self
print(Counter().step().step(5).n) # 6
Indentation defines blocks. Variables are names bound to objects: b = a does not copy a list. Use a.copy() when you need a copy.
A.3NumPy Arrays
import numpy as np
v = np.array([1.0, 2.0, 3.0]) # shape (3,)
M = np.arange(12).reshape(3, 4) # shape (3, 4)
Z = np.zeros((2, 3)); I = np.eye(3)
M[1, 2] # one element
M[1] # row 1
M[:, 2] # column 2
M[:2, 1:3] # a 2x2 block
M[M > 5] # boolean mask
M[[0, 2]] # rows 0 and 2
M.sum(), M.sum(axis=0), M.mean(axis=1) # reductions
M.T # transpose
np.hstack([M, np.ones((3, 1))]) # append a column (the bias trick)
Axis convention: a data matrix has one row per example, X.shape == (n, d). X.mean(axis=0) gives one mean per feature.
A.4Broadcasting and Vectorization
NumPy compares shapes from the right and stretches dimensions of size 1:
X = np.random.default_rng(0).normal(size=(100, 3))
Xs = (X - X.mean(axis=0)) / X.std(axis=0) # per-feature standardization
A, B = X[:5], X[5:9]
D2 = ((A[:, None, :] - B[None, :, :])**2).sum(-1) # all pairwise squared distances, (5, 4)
If you write a Python loop over examples, look for a matrix formulation: predictions are X @ w, the squared-loss gradient is 2 / n * X.T @ (X @ w - y). Beware: shapes (n,) and (n, 1) broadcast to (n, n), a classic silent bug. Print shapes often.
A.5Linear Algebra and Random Numbers
A = np.array([[3.0, 1.0], [1.0, 2.0]])
b = np.array([9.0, 8.0])
x = np.linalg.solve(A, b) # never inv(A) @ b
w, *_ = np.linalg.lstsq(X, X @ [1., 2., 3.], rcond=None)
vals, vecs = np.linalg.eigh(A) # symmetric eigendecomposition
U, s, Vt = np.linalg.svd(X, full_matrices=False)
rng = np.random.default_rng(42) # reproducible generator
rng.normal(0, 1, size=(2, 3)), rng.uniform(size=5), rng.integers(10, size=4)
rng.permutation(10), rng.choice(10, size=3, replace=False)
A.6Plotting with Matplotlib
import matplotlib.pyplot as plt
xs = np.linspace(0, 3, 200)
fig, ax = plt.subplots(figsize=(6, 4))
ax.scatter(X[:, 0], X[:, 1], s=8, label="data")
ax.plot(xs, np.sin(2 * xs), color="black", label="sin 2x")
ax.set_xlabel("x"); ax.set_ylabel("y"); ax.legend()
fig.savefig("plot.png", dpi=150, bbox_inches="tight")
A.7PyTorch Tensors and Autograd
import torch
w = torch.zeros(3, requires_grad=True)
X = torch.randn(100, 3); y = X @ torch.tensor([1., 2., 3.])
loss = ((X @ w - y)**2).mean()
loss.backward() # fills w.grad = 2/n X^T (Xw - y)
with torch.no_grad():
w -= 0.1 * w.grad
w.grad.zero_()
a = torch.from_numpy(np.ones(3)); b = a.numpy() # shares memory
Chapter 15 explains what backward computes; Chapter 16 shows a full training loop.