Chapter 24 · Part VI: Frontier Techniques
Large Language Models
After this chapter you will be able to
- Write the language-modeling objective with the chain rule of probability and relate cross-entropy to perplexity.
- Run byte-pair encoding by hand and explain how tokenization affects model behavior.
- Use the compute estimate $C\approx6ND$ and derive the compute-optimal trade-off between model size and data.
- Choose among greedy decoding, temperature, top-$k$, and top-$p$ sampling.
- Derive the RLHF objective and the DPO loss, and describe reinforcement learning with verifiable rewards (GRPO) for reasoning models.
- Explain mixture-of-experts layers, quantization, distillation, and LoRA fine-tuning.
- Build a retrieval-augmented generation pipeline and describe tool-using agents.
- Discuss evaluation, hallucination, and safety with appropriate caution.
A large language model (LLM) is a decoder-only Transformer (Chapter 19) with billions of parameters, trained to predict the next token of text. That one objective, applied at enormous scale and followed by a few stages of fine-tuning, produces systems that write code, answer questions, translate, and solve multi-step math problems. This chapter explains every stage of the pipeline with the tools of the earlier chapters: maximum likelihood, cross-entropy, gradient descent, and a little reinforcement learning.
24.1Language Modeling
A language model assigns a probability to every sequence of tokens $x_1,\dots,x_n$. By the chain rule of probability, any joint distribution factors into next-token conditionals:
$$p(x_1,\dots,x_n) = \prod_{t=1}^{n}p(x_t\mid x_1,\dots,x_{t-1}) . \tag{24.1}$$An autoregressive model learns each factor with a neural network: a causally masked Transformer reads $x_{<t}$ and outputs a softmax distribution over the vocabulary for $x_t$. Training maximizes the likelihood of a large text corpus, which by Section 2.6 means minimizing the average negative log-likelihood, the next-token cross-entropy:
$$\Loss(\vtheta) = -\frac1{T}\sum_{t=1}^{T}\log p_{\vtheta}(x_t\mid x_{<t}) . \tag{24.2}$$Thanks to the causal mask, one forward pass over a sequence of length $n$ produces all $n$ predictions in parallel. No labels are needed: the text supervises itself, which is why this is called self-supervised learning.
Perplexity. The exponential of the loss, $\mathrm{PPL}=e^{\Loss}$, is the perplexity. If a model’s loss is $\log k$, it is on average as uncertain as a uniform choice among $k$ tokens. Lower is better; a model that always predicts the next token with certainty has perplexity 1.
Key idea: Why next-token prediction teaches so much
To predict the next word of a physics textbook, a model benefits from knowing physics; to predict the next line of a program, from knowing what the program does; to predict the next move in a recorded chess game, from playing well. Minimizing Equation (24.2) over a sufficiently broad corpus rewards a model for acquiring whatever knowledge and skills reduce its uncertainty about text. This is the core reason a single, simple objective produces general capabilities.
24.2Tokenization
Models operate on integer token IDs, not characters. Word-level vocabularies cannot handle new words, misspellings, or code; character-level sequences are very long and make attention expensive. Modern LLMs use subword tokenizers, most commonly byte-pair encoding (BPE) (Sennrich et al. 2016), which interpolates between the two.
- Input: training text, target vocabulary size $V$
- split the text into words; write each word as a sequence of base symbols (bytes or characters)
- vocabulary $\gets$ all base symbols
- while $|\text{vocabulary}|<V$:
- count every pair of adjacent symbols across the corpus (weighted by word frequency)
- $(a,b)\gets$ the most frequent pair
- replace every occurrence of $a\,b$ by a new symbol $ab$; add $ab$ to the vocabulary
- return vocabulary and the ordered list of merges
To tokenize new text, apply the learned merges in the same order. Frequent words become single tokens; rare words split into meaningful pieces (“tokenization” $\to$ “token” + “ization”). Starting from the 256 possible bytes guarantees that any string, in any language or encoding, can be tokenized. Typical vocabularies have 32,000 to a few hundred thousand tokens; an English word averages about 1.3 tokens.
from collections import Counter
def train_bpe(words, num_merges):
# words: Counter mapping word -> frequency; each word becomes a tuple of characters
corpus = {tuple(w) + ("</w>",): f for w, f in words.items()}
merges = []
for _ in range(num_merges):
pairs = Counter()
for symbols, f in corpus.items():
for a, b in zip(symbols, symbols[1:]):
pairs[a, b] += f
if not pairs:
break
(a, b), _ = pairs.most_common(1)[0]
merges.append((a, b))
new_corpus = {}
for symbols, f in corpus.items():
out, i = [], 0
while i < len(symbols):
if i + 1 < len(symbols) and symbols[i] == a and symbols[i + 1] == b:
out.append(a + b); i += 2
else:
out.append(symbols[i]); i += 1
new_corpus[tuple(out)] = f
corpus = new_corpus
return merges, corpus
merges, corpus = train_bpe(Counter({"low": 5, "lower": 2, "newest": 6, "widest": 3}), 10)
print(merges[:4]) # [('e', 's'), ('es', 't'), ('est', '</w>'), ('l', 'o')]
Tokenization explains several otherwise puzzling behaviors. Models are weaker at character-level tasks (counting letters, reversing strings) because they never see the letters inside a token. Numbers may be split inconsistently, which hurts arithmetic unless digits are tokenized individually. And languages under-represented in the tokenizer’s training data need more tokens per word, which makes them slower and more expensive to process.
24.3Pretraining and Scaling Laws
Data and compute. Pretraining corpora consist of trillions of tokens of web text, books, code, and scientific articles, heavily filtered for quality and deduplicated. A useful rule of thumb gives the training compute in floating-point operations for a model with $N$ parameters trained on $D$ tokens:
$$C \approx 6ND . \tag{24.3}$$The forward pass costs about $2N$ operations per token (one multiply and one add per weight), and the backward pass about twice that.
Scaling laws. Empirically, the pretraining loss of Transformers falls smoothly and predictably as a power law in model size, data size, and compute, over many orders of magnitude (Kaplan et al. 2020). A convenient parametric form (Hoffmann et al. 2022) is
$$L(N,D) = E + \frac{A}{N^{\alpha}} + \frac{B}{D^{\beta}}, \tag{24.4}$$where $E$ is the irreducible entropy of text and the two other terms are the excess loss from a finite model and finite data. The fitted exponents are around $\alpha\approx\beta\approx0.3$.
Compute-optimal training. Given a compute budget $C$, how should we split it between parameters and tokens? Minimize Equation (24.4) subject to $6ND=C$. Substituting $D=C/(6N)$ and setting the derivative with respect to $N$ to zero gives
$$\alpha A N^{-\alpha-1} = \beta B\,(6/C)^{\beta}N^{\beta-1} \quad\Longrightarrow\quad N_{\mathrm{opt}}\propto C^{\beta/(\alpha+\beta)},\qquad D_{\mathrm{opt}}\propto C^{\alpha/(\alpha+\beta)} .$$With $\alpha\approx\beta$, both scale like $\sqrt C$: double the model, double the data. The “Chinchilla” analysis found the optimum near 20 training tokens per parameter, and showed that several earlier large models had been undertrained. In practice, models that will serve many users are often trained on far more tokens than the compute-optimal amount, because a smaller model trained longer is cheaper at inference time for the same quality.
Frontier note
Scaling laws turned model development into something closer to engineering: small pilot runs predict the loss of a large run before it is launched. Two caveats matter. First, the laws predict loss, not specific abilities; performance on a particular task can improve abruptly once the loss crosses some level. Second, high-quality text is finite, so data quality, deduplication, synthetic data, and reuse of data across epochs are now central research topics.
24.4Decoding: From Probabilities to Text
A trained model gives a distribution over the next token; generating text means repeatedly choosing a token and appending it. With logits $\vz$ over the vocabulary:
- Greedy decoding picks $\argmax_i z_i$. Deterministic, but tends to produce repetitive, dull text.
- Temperature sampling samples from $p_i\propto\exp(z_i/T)$. $T<1$ sharpens the distribution toward the most likely tokens; $T>1$ flattens it; $T\to0$ is greedy.
- Top-$k$ sampling keeps only the $k$ most likely tokens, renormalizes, and samples.
- Top-$p$ (nucleus) sampling (Holtzman et al. 2020) keeps the smallest set of tokens whose total probability exceeds $p$ (say 0.9), so the candidate set adapts: small when the model is confident, large when many continuations are plausible.
Low temperatures suit tasks with one right answer (code, math); higher temperatures and nucleus sampling suit creative writing. Each generated token requires a forward pass, made affordable by the KV cache (Section 19.8).
import numpy as np
def sample_next(logits, temperature=1.0, top_k=None, top_p=None, rng=np.random.default_rng()):
z = logits / max(temperature, 1e-6)
p = np.exp(z - z.max()); p /= p.sum()
order = np.argsort(-p)
keep = np.ones_like(p, dtype=bool)
if top_k is not None:
keep[order[top_k:]] = False
if top_p is not None:
cum = np.cumsum(p[order])
cutoff = np.searchsorted(cum, top_p) + 1 # smallest set with mass >= top_p
keep[order[cutoff:]] = False
p = np.where(keep, p, 0); p /= p.sum()
return rng.choice(len(p), p=p)
24.5In-Context Learning and Prompting
Large pretrained models can perform new tasks from instructions and examples given in the prompt, with no change to their weights. Showing a few input–output pairs before a new input (few-shot prompting) often works nearly as well as fine-tuning (Brown et al. 2020). This in-context learning emerges from pretraining: text on the internet is full of patterns that continue earlier patterns, and continuing them well rewards inferring the task.
Asking a model to write out intermediate reasoning steps before its answer, called chain-of-thought prompting (Wei et al. 2022), substantially improves performance on multi-step problems. A simple explanation: each token is produced by a fixed amount of computation, so a problem that needs many sequential steps needs many tokens; written-out steps give the model room to compute and let later steps condition on earlier results. Reasoning models (Section 24.7) are trained to do this systematically.
24.6Post-Training: Instruction Tuning and Preference Optimization
A pretrained model continues text; it does not reliably follow instructions, and it imitates everything in its training data, good and bad. Post-training turns it into a helpful assistant in stages.
Supervised fine-tuning (SFT). Fine-tune on curated (prompt, ideal response) pairs with the same next-token loss, applied only to the response tokens. A few tens of thousands of high-quality examples teach the format and style of a helpful assistant.
Reinforcement learning from human feedback (RLHF). It is often easier for people to compare two responses than to write an ideal one. RLHF (Christiano et al. 2017; Ouyang et al. 2022) collects comparisons and optimizes the model against them in two steps.
Step 1: a reward model. For prompt $x$, a person prefers response $y_w$ over $y_l$. Model the probability of this preference with the Bradley–Terry model, a logistic regression on the difference of scalar rewards:
$$p(y_w\succ y_l\mid x) = \sigma\big(r_{\vphi}(x,y_w)-r_{\vphi}(x,y_l)\big), \tag{24.5}$$where $r_{\vphi}$ is a Transformer with a scalar output head. Fit $\vphi$ by maximizing the likelihood of the observed preferences, i.e. minimizing $-\log\sigma(r_{\vphi}(x,y_w)-r_{\vphi}(x,y_l))$, exactly the binary cross-entropy of Chapter 6.
Step 2: optimize the policy. Treat the language model $\pi_{\vtheta}$ as a policy that generates a response, and maximize the expected reward while staying close to the SFT model $\pi_{\mathrm{ref}}$:
$$\max_{\vtheta}\ \E_{x,\ y\sim\pi_{\vtheta}(\cdot\mid x)}\big[r_{\vphi}(x,y)\big] - \beta\,D_{\mathrm{KL}}\big(\pi_{\vtheta}(\cdot\mid x)\,\Vert\,\pi_{\mathrm{ref}}(\cdot\mid x)\big). \tag{24.6}$$The KL penalty prevents reward hacking: without it the policy drifts toward odd outputs that exploit errors in the learned reward model. The expectation is over the model’s own samples, so the gradient requires reinforcement learning (policy-gradient methods such as PPO; Chapter 27).
Direct preference optimization (DPO). The RL step is complex and unstable. DPO (Rafailov et al. 2023) removes it with a neat observation. For a fixed reward $r$, the maximizer of Equation (24.6) has a closed form (Exercise 24.4):
$$\pi^*(y\mid x) = \frac{1}{Z(x)}\,\pi_{\mathrm{ref}}(y\mid x)\exp\!\big(r(x,y)/\beta\big),$$where $Z(x)$ normalizes. Solve for the reward: $r(x,y)=\beta\log\frac{\pi^*(y\mid x)}{\pi_{\mathrm{ref}}(y\mid x)}+\beta\log Z(x)$. Substitute this into the Bradley–Terry model; the intractable $\log Z(x)$ appears in both rewards and cancels in the difference. Replacing $\pi^*$ by the trainable policy gives a loss that needs no reward model and no sampling:
DPO is supervised learning on preference pairs: it raises the probability of preferred responses relative to the reference model and lowers that of rejected ones. It and its variants are widely used, especially in open models.
AI feedback. Human labels are slow and expensive. In RL from AI feedback, a model judges responses according to a written set of principles, a technique introduced as Constitutional AI (Bai et al. 2022); the resulting preferences train a reward model or are used directly. A comprehensive treatment of these methods is Lambert (2025).
24.7Reasoning Models and Test-Time Compute
Chain-of-thought prompting asks a model to reason. Reasoning models are trained to do it: they produce a long internal chain of thought (often thousands of tokens) before the final answer, and their accuracy on mathematics, programming, and science problems is far higher than that of models answering directly.
Reinforcement learning with verifiable rewards (RLVR). For many problems the final answer can be checked automatically: a math answer against a reference, a program against unit tests. Such checks give a reliable reward without a learned reward model, and therefore without reward hacking of the learned-model kind. Training samples many reasoning attempts per problem, rewards the ones that reach a verified correct answer, and updates the model with a policy-gradient method. Remarkably, behaviors such as checking intermediate results, backtracking, and trying alternative approaches emerge from this outcome-only reward (DeepSeek-AI 2025).
GRPO. A popular algorithm for this setting is group relative policy optimization (Shao et al. 2024). For each prompt, sample a group of $G$ responses $y_1,\dots,y_G$ and score them with rewards $r_1,\dots,r_G$. Each response’s advantage (how much better it was than expected) is its standardized reward within the group:
$$A_i = \frac{r_i-\mathrm{mean}(r_1,\dots,r_G)}{\mathrm{std}(r_1,\dots,r_G)} . \tag{24.8}$$The update increases the log-probability of every token of responses with positive advantage and decreases it for responses with negative advantage, with a clipped importance ratio and a KL penalty toward a reference model to keep steps small. Using the group mean as the baseline replaces the separate value network that PPO would need, which saves memory and simplifies training. If every response in a group gets the same reward, all advantages are zero and the prompt contributes nothing; useful training prompts are those the model sometimes, but not always, solves.
Test-time compute. Reasoning models reveal a second scaling axis: besides spending more compute on training, we can spend more at inference. Ways to do so include generating longer chains of thought, sampling many solutions and taking a majority vote over final answers (self-consistency; (Wang et al. 2023)), and sampling many candidates and selecting with a verifier or reward model. For a fixed budget, the best mix of model size and inference-time computation depends on the difficulty of the problem; on suitable problems, extra test-time compute can let a smaller model outperform a much larger one (Snell et al. 2024).
Example: Why majority voting helps
Suppose each independent sample is correct with probability $p=0.6$ and wrong answers are scattered. The probability that the majority of 15 samples is correct is $\sum_{k=8}^{15}\binom{15}{k}0.6^k0.4^{15-k}\approx0.79$, and it approaches 1 as the number of samples grows. If $p<0.5$ and wrong answers agree with each other, voting makes things worse: test-time compute amplifies whatever the model already tends to do.
24.8Efficient Models: Mixture of Experts, Quantization, and Distillation
Mixture of experts (MoE). In a dense Transformer every parameter is used for every token. An MoE layer (Shazeer et al. 2017; Fedus et al. 2022) replaces the feed-forward network with $E$ expert FFNs and a small router that sends each token to only $k$ of them (typically $k$ between 1 and 8, with $E$ from 8 to several hundred):
$$\mathrm{MoE}(\vx) = \sum_{i\in\mathrm{TopK}(\vg(\vx))} \frac{e^{g_i(\vx)}}{\sum_{j\in\mathrm{TopK}}e^{g_j(\vx)}}\,\mathrm{FFN}_i(\vx),\qquad \vg(\vx)=\mW_r\vx . \tag{24.9}$$The model’s total parameter count (and capacity to store knowledge) grows with $E$, while the compute per token depends only on the $k$ active experts. An auxiliary loss, or a bias adjustment in the router, keeps the load balanced so that no expert is overwhelmed or unused. As of 2026, MoE is the dominant design for large open-weight models; a model may have hundreds of billions of total parameters but activate only a few tens of billions per token.
Quantization. Weights trained in 16-bit can often be stored in 8 or even 4 bits with little loss in quality. Uniform quantization maps each weight to an integer $q=\mathrm{round}(w/s)$ with a scale $s=\max|w|/(2^{b-1}-1)$ chosen per small group of weights, and dequantizes as $\hat w=sq$; the error per weight is at most $s/2$. Quantization reduces memory and speeds up inference, which is usually limited by how fast weights can be read from memory.
Distillation. A large teacher model can train a small student by having the student match the teacher’s full output distribution (minimizing the KL divergence between them) rather than only the one-hot correct token (Hinton et al. 2015). The teacher’s probabilities carry much more information per example (which wrong answers are nearly right), so distilled students often far exceed students trained on the same data alone. Many small models are trained this way, including on reasoning traces generated by larger reasoning models.
24.9Parameter-Efficient Fine-Tuning: LoRA
Fine-tuning all weights of a large model requires storing a full copy of the model per task and the optimizer state for every parameter. Low-rank adaptation (LoRA) (Hu et al. 2022) freezes the pretrained weight matrix $\mW_0\in\R^{d\times k}$ and learns a low-rank update:
Only $\mA$ and $\mB$ are trained: $r(d+k)$ parameters instead of $dk$. For $d=k=4096$ and $r=16$ that is $131{,}072$ instead of $16.8$ million, under 1%. $\mB$ is initialized to zero so that training starts exactly at the pretrained model, and $\alpha/r$ scales the update. After training, $\mB\mA$ can be added into $\mW_0$, so inference costs nothing extra; or many small adapters can be swapped in and out of one base model.
Why does a low-rank update suffice? Fine-tuning usually adjusts a model’s behavior in a few directions rather than rewriting it, and empirically the weight changes from full fine-tuning have rapidly decaying singular values. This is the same structure that PCA exploits (Chapter 14): most of the change lies in a few directions. QLoRA (Dettmers et al. 2023) combines a 4-bit quantized frozen base model with LoRA adapters, making it possible to fine-tune large models on a single GPU.
import torch, torch.nn as nn
class LoRALinear(nn.Module):
def __init__(self, base: nn.Linear, r=8, alpha=16):
super().__init__()
self.base = base.requires_grad_(False) # frozen pretrained W0
self.A = nn.Parameter(torch.randn(r, base.in_features) / r**0.5)
self.B = nn.Parameter(torch.zeros(base.out_features, r)) # zero: start at W0
self.scale = alpha / r
def forward(self, x):
return self.base(x) + self.scale * (x @ self.A.T) @ self.B.T
24.10Retrieval-Augmented Generation and Tool Use
An LLM’s knowledge is frozen at training time, is stored imprecisely in its weights, and cannot include private documents. Retrieval-augmented generation (RAG) (Lewis et al. 2020) supplies relevant text at query time:
- Index. Split the documents into passages and compute an embedding vector for each with an encoder model (Section 19.1). Store the vectors in an approximate nearest-neighbor index (Chapter 8).
- Retrieve. Embed the user’s question and find the passages with the highest cosine similarity. Many systems combine this with keyword search and re-rank the candidates with a more accurate model.
- Generate. Put the retrieved passages into the prompt and ask the model to answer using them, citing its sources.
RAG reduces hallucination on factual questions, keeps answers current without retraining, and lets users check the sources. Its quality is limited by retrieval: if the right passage is not retrieved, the model cannot use it.
Tools and agents. Models can also be trained to emit structured tool calls, such as a search query, a calculator expression, or code to execute; the surrounding system runs the tool and appends the result to the context, and the model continues. Looping this process (think, act with a tool, observe the result, repeat) (Yao et al. 2023) produces agents that can carry out multi-step tasks such as debugging a program or researching a question across many web pages. Agents are trained with the same RL methods as reasoning models, with rewards for completing tasks. Their reliability over long horizons, and the security of letting a model act on outside content (which may contain malicious instructions, called prompt injection), are among the most active engineering and research problems of 2026.
24.11Evaluation, Limitations, and Safety
Evaluation. LLMs are evaluated on benchmarks of questions with known answers (mathematics, coding, science, reading comprehension), on human or model judgments of open-ended responses, and increasingly on realistic multi-step tasks. Benchmarks saturate quickly, and their questions may leak into web-scale training data (contamination), so a high score can overstate real ability. Treat any single number with the skepticism of Chapter 12: ask how it was measured and whether the test resembles your use case.
Hallucination. Models sometimes produce fluent, confident statements that are false. Next-token training rewards plausible continuations, and a model that has learned the form of a citation can produce one that does not exist. Retrieval, calibrated abstention (“I don’t know”), and training that rewards honesty reduce but do not eliminate the problem. Verify important claims.
Bias and misuse. Models absorb patterns, including stereotypes, from their training data, and capable models can be misused. Developers use post-training, filtering, red-teaming, and usage policies to reduce harm, and adversarial prompts (jailbreaks) that bypass these safeguards are an ongoing cat-and-mouse problem.
Alignment. More broadly, making sure that increasingly capable systems reliably do what their users and society intend, and that we can understand why they behave as they do (interpretability), is a central research challenge. The techniques of this chapter (preference learning, AI feedback, RL with careful reward design) are the current tools; none is a complete solution.
24.12Where the Field Is Heading
As of late 2026, the frontier is defined less by bigger pretraining runs alone and more by what happens after pretraining. Reinforcement learning on verifiable tasks has become a major part of training compute; models that reason at length, use tools, and act as agents over many steps are the focus of most effort; and spending compute at inference time is a standard lever alongside model size. Architecturally, sparse mixture-of-experts layers, compressed KV caches, and hybrid local/global attention make long contexts and large models affordable. Open questions include how to provide reliable rewards for tasks that cannot be checked automatically, how to make long-horizon agents dependable and secure, how to keep evaluations honest, and how to understand and steer what large models have learned.
24.13Summary
- An LLM factors text probability by the chain rule and is trained to minimize next-token cross-entropy; perplexity is $e^{\Loss}$.
- BPE tokenization merges frequent symbol pairs; it shapes what models find easy or hard.
- Training compute is about $6ND$; loss follows power laws, and compute-optimal training scales $N$ and $D$ together (about 20 tokens per parameter).
- Decoding chooses tokens by greedy search or by sampling with temperature, top-$k$, or top-$p$.
- Post-training: SFT on demonstrations; RLHF with a Bradley–Terry reward model and a KL-regularized objective; DPO turns this into a supervised loss on preference pairs.
- Reasoning models are trained with RL on verifiable rewards (e.g. GRPO with group-standardized advantages) and benefit from extra test-time compute.
- MoE decouples total from active parameters; quantization and distillation shrink models; LoRA fine-tunes with a low-rank update $\mB\mA$.
- RAG retrieves relevant passages by embedding similarity; tool use and agents extend models to act.
- Evaluate skeptically; hallucination, bias, misuse, and alignment remain open problems.
24.14Exercises
- MathA model assigns probability $0.25$ to every correct next token in a test set. What are its cross-entropy (in nats) and perplexity?
- MathUsing $C\approx6ND$, estimate the training compute for a 7-billion-parameter model trained on 2 trillion tokens. How many days would it take on hardware sustaining $10^{15}$ operations per second?
- MathCarry out the compute-optimal minimization of Equation (24.4) subject to $6ND=C$ in detail, and verify the exponents $\beta/(\alpha+\beta)$ and $\alpha/(\alpha+\beta)$.
- MathShow that the maximizer over distributions $\pi$ of $\E_{y\sim\pi}[r(y)]-\beta D_{\mathrm{KL}}(\pi\Vert\pi_{\mathrm{ref}})$ is $\pi^*(y)\propto\pi_{\mathrm{ref}}(y)e^{r(y)/\beta}$. (Hint: rewrite the objective as $-\beta D_{\mathrm{KL}}(\pi\Vert\pi^*)+\text{const}$.) Then derive Equation (24.7).
- MathIn GRPO, suppose a group of 8 responses has rewards $(1,0,0,1,0,0,0,0)$. Compute the advantages. What happens if all eight rewards are 1?
- MathA LoRA adapter with rank $r=8$ is applied to the four attention matrices ($d\times d$, $d=4096$) of each of 32 layers. How many trainable parameters does it have, as a fraction of the $12Ld^2$ block parameters?
- CodingExtend
train_bpewith a function that tokenizes a new word by applying the merges in order. Train on the text of a public-domain book with 1000 merges and report the average number of tokens per word. - CodingUsing the tiny GPT of Project 6, generate 20 samples each at temperatures 0.5, 1.0, and 1.5 and with top-$p=0.9$. Measure diversity as the fraction of distinct 4-character substrings. Discuss the trade-off between quality and diversity.