Roadmap: ML → DL → agentic models

Modules 01–08 gave you the tools: notation, linear algebra, calculus, probability, statistics, information theory, optimization, and eight algorithms rebuilt from those parts. This module is the payoff and the launchpad — it proves what you actually own, and then shows the same parts doing the same jobs inside neural networks and agents.

The 12 questions, re-stated — same questions, new attempt

These are the twelve cold questions from Module 00, restated compactly. Do not open any module, do not search anything. Ten minutes, short answers. If you have your original answers to hand, do not look at them yet — the comparison happens in Section 1, and grading yourself against an answer you are currently looking at measures nothing.

Six "a month later" cards — the whole course in small questions

The twelve above test what survived from before. These six test what Modules 01–08 were supposed to install. Same rule: write your answer, then open the card.

1 · The diagnostic revisited: measure the gain

You have just answered the twelve questions above a second time. Take out your Module 00 answers — the ones with the date on them — and put the two attempts side by side. For each question, mark one of four states:

1. Still wrong. Same misconception, same blank. That item maps to one module; put it on the repair list.
2. Wrong, differently. The old mistake is gone and a new, subtler one took its place. This is progress — you have moved to a harder error. Treat it as half a win.
3. Right, procedurally. You got the answer by grinding, and cannot say why it is true. Recognition, not ownership — schedule a re-derivation.
4. Right, with a reason. You could have taught it. This is the only state that counts as done.

The count that matters is state 4, and how it moved. This comparison — not any quiz on this site, not any score in a textbook — is the measurement of the relearning, for a plain reason: it is the only place your day-one baseline exists. Every other instrument here is a practice exercise; the diagnostic pair is a before/after on the same subject, cold and unprompted, which is the closest thing to a controlled experiment you can run on yourself.

Three honest readings of the result. If states 3 and 4 dominate: the course did its job, and Section 8 is your maintenance plan. If states 1 and 2 dominate but the explanations now land on re-reading: you learned more than you can retrieve — the fix is not more content, it is more retrieval (Section 8 again). If question 12 is still blank: that is the real alarm, because question 12 is the course's whole thesis — every algorithm is an assembly of these tools — and it is the skill that transfers to deep learning and agents.

2 · The mastery checklist

Eight modules, one falsifiable ability each pair. Every item is phrased so that you either did it or you did not — no partial credit, no "sort of". Work through them aloud, one sitting, no notes. The module tag tells you exactly where the repair lives if an item fails.

• M01 · Read any ML objective aloud: which symbols are data, which are parameters, what is being summed, and which direction is "good".
• M02 · Given $X \in \mathbb{R}^{n \times d}$ and $w \in \mathbb{R}^d$, state the shape of every term in the gradient of the MSE — and the gradient itself.
• M02 · Derive the normal equation $w = (X^\top X)^{-1} X^\top y$ in four lines, and say two situations where you must not use it.
• M02 · Say what an eigenvector is for, and how PCA uses it — geometrically, not procedurally.
• M03 · Explain why $\sigma'(x) = \sigma(x)(1-\sigma(x))$ and why that causes vanishing gradients in a deep stack.
• M03 · Run a finite-difference gradient check on a loss you just wrote, and say what tolerance is acceptable.
• M04 · State Bayes' rule and compute a base-rate posterior without a calculator — the 1%-prevalence test above is the standard item.
• M04 · Say which loss MLE gives you under Gaussian noise and under Bernoulli noise — squared error and cross-entropy, and why the log turns one into the other.
• M05 · Say what a p-value is and is not, and what a 95% confidence interval does and does not claim.
• M05 · Write the bias–variance decomposition and name which knob moves which term.
• M06 · Derive cross-entropy from surprise; state KL's asymmetry and one place the direction of the asymmetry matters in practice.
• M07 · Write the gradient-descent, momentum and Adam updates from memory, including what $\beta_1$ and $\beta_2$ each smooth.
• M07 · Explain why the learning rate cannot be regularised away, and what warmup and gradient clipping actually fix.
• M08 · Name the mathematical backbone of eight classic algorithms: linear and logistic regression, k-means, Naive Bayes, decision trees, SVM, PCA, k-NN.
• M08 · Say what an SVM maximises, and why the kernel trick is a dot product you never have to compute explicitly.

3 · The same math, bigger: deep learning

Here is the central claim of this module, and it is a fact, not encouragement: deep learning introduces no new mathematics from this course. It uses the same linear algebra, the same chain rule, the same maximum-likelihood argument, the same cross-entropy, the same statistics — with bigger matrices, more composition, and automatic differentiation doing the mechanical derivative steps. Read the map first, then the three pieces that deserve prose.

Attention, shape by shape

A layer is one affine map plus a non-linearity: $h = \sigma(Wx + b)$, and a network composes such maps. The one operation that made the current generation of models possible — attention — is nothing but matrix products and a softmax. Three token sequences of length $n$ become matrices of queries, keys and values:

Read the shapes: $QK^\top$ is $n \times n$ — entry $(i,j)$ is token $i$'s query dotted with token $j$'s key, i.e. Module 02's alignment score. Row-normalising with a softmax makes every row a probability distribution over the $n$ keys. Multiplying by $V$ then averages value vectors, and each output row is an $d_v$-vector. Nothing here is beyond Module 02 plus Module 04's softmax.

The $\sqrt{d_k}$ deserves its own sentence, because it is a variance argument (Module 05's tool). If the components of a query and a key are independent with unit variance, then their dot product $q \cdot k = \sum_{i=1}^{d_k} q_i k_i$ has variance $d_k$ — logits get wilder as the vectors get longer, the softmax saturates onto one key, and gradients die. Dividing by $\sqrt{d_k}$ divides variance by $d_k$, restoring unit variance at every width. Block 1 measures exactly this: with four random keys, the unscaled softmax at $d_k = 64$ has mean row-entropy $0.22$ nats against a maximum of $\ln 4 \approx 1.39$, while the scaled version sits at $1.11$ — and at $d_k = 256$ the unscaled version collapses further, to $0.11$, while the scaled one barely moves.

Vanishing gradients: the chain rule, multiplied too many times

Backpropagation is the chain rule over a computation graph — Module 03, section 7. To reach a parameter in layer 1 you multiply through every layer in between, and each sigmoid contributes a factor bounded by $0.25$: twenty such factors give roughly $10^{-12}$. The gradient is not "small", it is gone, indistinguishable from zero in float arithmetic. Exploding gradients are the mirror case, when the weight matrices have spectral radius above 1.

Every architectural fix of the last decade is a patch on this one product. ReLU makes the derivative exactly 1 on the active side. Residual connections give the gradient a highway: with $h_{l+1} = h_l + F(h_l)$ the backward pass picks up an additive identity path, so the gradient is a sum of routes instead of a pure product — one route can never zero the whole thing. Initialisation keeps variance at 1 through depth, normalisation re-enforces it every layer, and clipping caps the damage when it still spikes. Four techniques, one equation.

Note also what backprop hands you for a whole layer at once — the gradient of a layer family. For a linear layer $h = Wx + b$ with incoming error signal $\delta \in \mathbb{R}^{d_{\text{out}}}$ and layer input $x \in \mathbb{R}^{d_{\text{in}}}$, every weight gets its sensitivity from the same outer product, $\partial L / \partial W = \delta\, x^\top \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$ — a matrix shaped exactly like $W$. A million parameters, one sweep, one matrix product. That is also why "the framework does the derivatives for you" never made the calculus optional: the sweep is mechanical, but whether those derivatives are meaningful — finite, non-saturating, pointing downhill — is not.

Optimization at scale: same engine, harsher landscape

Modules 03 and 07 gave you convexity, descent and the update rules; deep learning removes convexity and keeps everything else. The loss surface of a large network is non-convex, but the practical failure point is rarely a "local minimum" — it is a saddle point, where the gradient is near zero because some directions rise and others fall. Saddles are generic in high dimensions and rare in two, which is one honest reason textbook intuition misleads here. The update rules you wrote from memory still run the loop; the extra machinery manages noise and scale:

Around those updates sit the tricks, each now legible. Warmup starts the learning rate tiny because Adam's per-coordinate statistics are meaningless in the first steps — a variance argument. Schedules decay the rate as the landscape flattens. Gradient clipping bounds the size of each step. Learning-rate finding sweeps $\eta$ once and reads off the largest stable value — the cheapest hyperparameter experiment there is. And batch normalisation is best read through Module 07's conditioning section: it re-centres and re-scales intermediate activations so the optimizer sees a less elongated loss surface, which is what makes aggressive learning rates survivable. Initialisation is the same distributional reasoning as Module 04: choose weight variances so activations keep variance $\approx 1$ at every depth, and the signal neither dies nor explodes.

The ELBO: a lower bound you can actually optimise

A latent-variable model defines $p(x) = \int p(x \mid z)\, p(z)\, dz$, an integral nobody can compute. Jensen's inequality (Module 06 — concavity of $\log$) turns that intractable quantity into a tractable lower bound, the evidence lower bound or ELBO, by sliding the expectation inside the log:

Two quantities trade off, and that trade is the entire design space of the model: reconstruction (the decoder must explain $x$ from $z$) against regularity (the encoder's posterior must stay close to the prior, so that drawing a fresh $z \sim p(z)$ produces something sensible). Push the first too hard and the latent space stops being usable; push the second and you get the posterior collapse failure mode, where the model ignores $z$ entirely. Everything else in a VAE is engineering.

Information theory and statistics, scaled up

Information theory becomes the vocabulary of losses. Every classification loss you will meet is cross-entropy, $H(p, q) = -\sum_v p(v)\log q(v)$ — at 2 classes or 100 000, same formula. Perplexity is exponentiated mean cross-entropy, quoted instead of nats because it reads as "the model is about as unsure as if it were choosing uniformly between $k$ equally likely tokens". KL shows up twice more: as a regulariser pulling a latent distribution toward a prior (the ELBO above, the VAE below), and as the distillation penalty — matching a teacher's soft outputs is $\mathrm{KL}(p_{\text{teacher}} \| q_{\text{student}})$, and the direction is chosen so the teacher's full distribution is honoured, not just its argmax.

Statistics gets stranger, and more important, at scale. Generalisation is still the question, but deep networks routinely have more parameters than data points and still generalise — the classical bias–variance story predicted they should not. Double descent is the observed phenomenon: test error falls, spikes around the interpolation threshold (the point where the model first fits the training data exactly), then falls again as the model gets larger still, eventually below the first minimum. Be honest about the state of play: theory is catching up (implicit regularization, the role of noise and architecture) but lags the practice, so treat double descent as a measured fact with competing explanations, not settled doctrine. Two things did not change and matter more than ever: calibration — whether a stated 0.9 happens 90% of the time, measured by log-loss and repaired by temperature scaling — and evaluation discipline, because the larger the model, the more opportunities for leakage, contamination and test-set overfitting, and the more expensive a wrong "it works" claim becomes.

4 · Transformers and language models, in your math

A language model is a stack of the operations you already own. Walk it once, in shapes and objectives, and the architecture stops being exotic:

1. Token embeddings — a vocabulary of size $|V|$ is a matrix $E \in \mathbb{R}^{|V| \times d}$; looking up token $i$ takes row $i$. A sequence of $n$ tokens is a matrix $X \in \mathbb{R}^{n \times d}$, one vector per token. Module 02: rows are data points.
2. Positional encoding — embeddings alone are order-blind (a set, not a sequence), so each position $m$ gets a fixed vector of sines and cosines at geometrically spaced frequencies: bounded, unique per position, and arithmetic on positions shows up as linear operations on the vectors. Added to $X$, no training required.
3. Self-attention — $A = \mathrm{softmax}(QK^\top / \sqrt{d_k}) V$, as in Section 3: every token gathers a weighted summary of every token.
4. Residual stream — each block reads from and writes an additive update to one shared vector per token. Additive writes are why very deep stacks train at all: the gradient flows down the stream as a sum of paths (Section 3, again).
5. Next-token objective — the model outputs $q(w_{t+1} \mid w_{\lt t})$, a distribution over the vocabulary; the data supplies a one-hot target $p$. The loss is cross-entropy $H(p, q) = -\log q(\text{actual next token})$ — Module 06's payoff, at $|V|$ classes, millions of times per step.

The other quantity people quote for a language model is perplexity — just exponentiated mean cross-entropy, a change of units from nats back into "effective number of equally plausible next tokens":

Two knobs on the output distribution, both pure Module 06. Temperature divides the logits before the softmax, which is equivalent to raising the base distribution to $1/T$ and renormalising:

Block 3 measures it: on a ten-token distribution with overconfident logits, entropy goes from $0.046$ nats at $T = 0.5$ to $0.319$ at $T = 1$ to $0.909$ at $T = 2$, while the top probability falls from $0.992$ to $0.701$. Top-p (nucleus) sampling is the other knob: keep the smallest set of tokens whose cumulative probability reaches the threshold — 0.9 is a common choice — renormalise, and sample only from there. That is a support truncation — a hard edit of the distribution, not a smoothing — and it is where entropy meets the engineering constraint of never emitting the long tail of garbage.

Context length is a memory-and-compute trade you can now price exactly: self-attention computes a dot product for every query–key pair, so attention work grows as $n^2$ in sequence length, and the cached key/value matrices grow linearly in $n \times$ layers $\times d$. Doubling the context quadruples the attention compute. Every architectural variant you will read about — sparse attention, sliding windows, state-space models — is an answer to that one line of arithmetic.

5 · Generative models, in your math

Three model families, three honest sketches. Each is one paragraph because each is one paragraph of mathematics from this course, applied.

VAEs. An encoder $q_\phi(z \mid x)$ performs amortised inference: instead of fitting a separate posterior per input, one network maps any input to the parameters of a posterior over its latent code. A decoder $p_\theta(x \mid z)$ reconstructs. The loss is exactly the negative ELBO from Section 3: reconstruction minus $\mathrm{KL}(q_\phi(z \mid x) \| p(z))$. The KL term is what makes generation possible at all — it is what allows a brand-new $z \sim p(z)$ to decode to a plausible $x$. There is even a coding view of the same trade — the bits-back argument, optional reading — where the KL term reads as the bitrate the latent code costs, which is a neat reminder that information theory and likelihood are one subject.

Diffusion models. A forward process corrupts data by adding small Gaussian noise over many steps until it is essentially pure noise; this is a Markov chain whose per-step distribution is known in closed form, so training pairs $(x_t, \text{noise})$ are free. The model learns to predict the noise at any step — equivalently, the score, $\nabla_x \log p_t(x)$, the gradient of the log-density with respect to the data (Module 03: a gradient is a direction of steepest change). Sampling is iterated denoising: start from noise, and repeatedly step in the direction that increases log-density — toward the data manifold. The reason these models are stable to train is that each step is a small regression problem, not a game.

GANs. A generator and a discriminator in a minimax game:

There is no likelihood to write down and no single objective descending — training instability and mode collapse are problems of game dynamics (Module 07's optimizers assume one player's loss; here two optimisers chase each other), which is why GAN training needed a decade of tricks and why diffusion largely displaced it.

6 · Agentic models, in your math

Your stated destination. An agent is a loop — perceive, think (an LLM call), act (a tool), observe — and the mathematics of that loop is decision-making under uncertainty. Nothing more, nothing less.

The belief state. The world has state you cannot see; you hold a distribution $b_t(s)$ over what it might be, and each observation revises it — Bayes' rule from Module 04, applied repeatedly, which is called filtering:

A partially observable Markov decision process (POMDP) is the formal frame for exactly this — hidden state, actions, observations, a belief over states — and the standard trick is that maintaining a correct belief turns the partially observable problem into an ordinary one over beliefs. An agent keeping a to-do list, a hypothesis about a bug, or a running summary of a document is maintaining $b_t$; the LLM's context window is its (lossy, uncompressed) belief state.

Planning. A policy $\pi$ is a distribution over actions, and the value of a state is the expected reward under it:

Exploration vs exploitation — the smallest complete example is the multi-armed bandit, and its best-known rule closes this notebook's loop. UCB picks the arm maximising

Unpack the second term and look at what it is: $\hat\mu_a$'s uncertainty scales as $1/\sqrt{n_a}$ — the standard error from Module 05, appearing as a decision rule. The bonus pays for information: an arm you have never pulled has $n_a = 0$ and an infinite bonus, so it is tried; an arm you have sampled 500 times has almost no bonus left, so it must win on merit. Statistics does not merely describe your measurements here — it tells the agent which experiment to run next. Block 2 shows UCB beating $\varepsilon$-greedy on a ten-armed bandit with cumulative regret $75$ against $363$ over 2 000 steps.

Tool use and structured output are constrained decoding: at each step the token distribution is masked down to tokens that keep the output syntactically valid (JSON keys, closing brackets), then renormalised — another softmax over a restricted support. Retries and verification loops are Monte Carlo search over strategies: sample $k$ candidate plans or answers, run a checker, keep what survives. Self-consistency is the cleanest case — sample $k$ independent chains, take the majority answer. That is a committee of estimators, and voting reduces the variance of the final decision (Module 05, ensemble logic) at $k$-times the cost.

Memory and retrieval are linear algebra wearing a product name. A memory item is an embedding vector $e \in \mathbb{R}^d$; retrieval is a dot product between the query embedding and every stored one, ranked — nearest neighbours in $\mathbb{R}^d$ (Module 02, again). The similarity metric is a modelling choice with consequences: raw dot product also rewards long vectors, so systems normalise and use cosine when magnitude should not mean popularity. RAG is then just conditional generation, $p(\text{answer} \mid \text{question}, \text{retrieved context})$ — the retrieval step decides which conditioning information the model ever sees.

Uncertainty and calibration are the last piece, and the one practitioners skip. An agent that knows when it does not know is exactly a calibrated model: its stated confidence matches observed frequency, which you measure with log-loss (Module 06 — a strictly proper scoring rule, it rewards honesty) and repair with temperature scaling (Section 4). A poorly calibrated model makes a bad agent for a specific, mechanical reason: overconfidence turns a coin-flip into a committed tool call, and the agent has no reason to abstain, ask, or check. Calibration is what converts a predictor into a colleague.

7 · The 8-week plan

Paced for a working engineer: 25–45 focused minutes on most weekdays, not five-hour Sundays. The companion ML notebook carries the practical half of each week. If a week slips, shrink the scope — never the reviews, because the reviews are the mechanism, not the reward.

The daily rhythm inside every week, in order: (1) 5 minutes of cold recall — the previous module's questions, written, before opening anything; (2) 25–45 minutes on today's section, with its code typed rather than copied; (3) the 1/3/7-day rule — re-answer a module's questions one day, then three, then seven after you finish it. The weekly test is the explain-aloud: pick one concept, set a five-minute timer, explain it out loud with no notes, ideally to a person or a recording. If you cannot, that concept goes back into this week's rotation — not into next month's reading list.

8 · How to keep it: retention as a system

Everything so far decays without maintenance, and the maintenance is not re-reading. It is four mechanisms, all cheap, all boring, all evidence-backed.

1. Spaced repetition. Build one card per checklist item, in the form: front = a question that names a context, back = the smallest complete answer. Examples: front "gradient of BCE with respect to the logit", back "$\sigma(z) - y$"; front "why $1/\sqrt{d_k}$ in attention", back "keeps logit variance at 1 so the softmax does not saturate"; front "what does a 95% CI claim", back "the procedure, not the parameter". Five to ten minutes a day, forever.
2. Monthly re-derivation drills. Once a month, pick one algorithm — logistic regression, PCA, k-means, the normal equation, backprop through a two-layer net — and derive it from definitions on paper, no notes, 30 minutes max. Derivation is the skill that survives; the ability to re-derive means the structure is stored, not the symbol string.
3. Project-based retention. Knowledge you use stays. The ladder: linear regression → logistic regression → a two-layer MLP in NumPy → one transformer attention block, each with shape assertions. By the end you will have written every operation in Section 3 yourself, once, slowly — which is exactly what makes the fast frameworks legible later.
4. The teaching test. Every quarter, explain the course to a colleague, or write it up. Teaching exposes the difference between "I follow this" and "I can rebuild this", and it costs nothing but willingness.

9 · Resources, chosen not listed

Nine items, each with a job. Notice what is not here: a syllabus to replace this notebook. These are supplements with specific purposes.

10 · Closing: what "done" means, and what deliberately remains undone

"Done" with this notebook does not mean knowing more mathematics. It means a smaller, testable list: you can cold-derive the checklist items; when a model misbehaves your first suspects are a shape, a gradient, or a probability assumption; you can read a formula in a paper and say what it is for before you understand its details; and you know which of your gaps matter for your goal and which are academic.

What was deliberately left out, so that the boundary is explicit rather than discovered later: measure-theoretic probability (the foundations under everything in Modules 04–06; needed only if you read or write theory); rigorous convex analysis (Module 07 states the results you need and why; the proofs are a separate, academic pursuit); representation-learning theory (why over-parameterised networks generalise is an open research question — no course can honestly teach it); and hardware and systems knowledge — mixed precision, kernels, distributed training. That last one is the exception to "academic": it is practically essential for training anything large, but it is engineering, not mathematics, and it belongs in the companion ML notebook and in your day job, not here.

For your stated destination — agentic models — be equally blunt about what is needed: everything in Modules 01–08, plus evaluation discipline (the companion notebook's modules 04, 12 and 19), plus systems literacy, plus a working knowledge of how language models behave. Not measure theory. Not functional analysis. The mathematics that decides whether your agents work is the mathematics in this notebook, applied carefully and maintained.

Which leaves one action, and it is today's, not next Monday's: pick one algorithm and rebuild it from the math, on paper or in NumPy, in the next 45 minutes. Not a tutorial, not a video — the normal equation, or logistic regression's gradient, or one attention head. That single act of production is worth more than everything you have read this week, including this page.

Block 1 · The attention scaling, verified

Random $Q$, $K$ matrices; compute the softmax of $QK^\top$ with and without the $1/\sqrt{d_k}$ scaling and measure the entropy of each attention row. Saturation means low entropy; the scaling should hold entropy near its maximum $\ln n$ at any width.

import numpy as np
rng = np.random.default_rng(0)

def softmax(z, axis=-1):
    z = z - z.max(axis=axis, keepdims=True)   # stability shift; cancels in the ratio
    e = np.exp(z)
    return e / e.sum(axis=axis, keepdims=True)

def mean_row_entropy(dk, scale, n=500):
    """Entropy of softmax(QK^T / scale) rows, averaged over random Q, K (nats)."""
    ent = []
    for _ in range(n):
        Q = rng.standard_normal((4, dk))      # 4 queries, dk dims each
        K = rng.standard_normal((4, dk))      # 4 keys
        p = softmax(Q @ K.T / scale)          # (4, 4) rows sum to 1
        ent.append(-(p * np.log(p + 1e-12)).sum(axis=1).mean())
    return np.mean(ent)

print("max possible entropy (4 keys) =", round(np.log(4), 3))     # 1.386
for dk in (64, 256):
    print(f"dk={dk}: unscaled {mean_row_entropy(dk, 1.0):.3f}"
          f"   scaled {mean_row_entropy(dk, np.sqrt(dk)):.3f}")
# dk= 64: unscaled 0.224   scaled 1.113   -- scaling keeps the softmax usable
# dk=256: unscaled 0.108   scaled 1.119   -- unscaled gets WORSE as d_k grows

# Shapes of one attention head, end to end (n tokens, dk keys, dv values)
Q = rng.standard_normal((5, 64)); K = rng.standard_normal((7, 64)); V = rng.standard_normal((7, 32))
A = softmax(Q @ K.T / np.sqrt(64))
print(Q.shape, K.shape, V.shape, A.shape, (A @ V).shape)  # (5,64) (7,64) (7,32) (5,7) (5,32)
print("rows of A sum to 1:", np.allclose(A.sum(axis=1), 1.0))     # True

Block 2 · The standard error as a decision rule

A ten-armed bandit. Compare UCB (exploitation estimate + uncertainty bonus, $c = 1$) against $\varepsilon$-greedy, measuring cumulative regret — how much reward you forgo versus always pulling the best arm. Same arms, same noise, 200 independent runs.

import numpy as np
rng = np.random.default_rng(1)

true_means = rng.normal(0, 1, 10)             # unknown to the agent

def run(policy, T=2000, runs=200):
    regrets = []
    for _ in range(runs):
        q = np.zeros(10); n = np.zeros(10); total = 0.0
        for t in range(1, T + 1):
            if policy == "ucb":
                # bonus = c * sqrt(ln t / n_a): shrinks like a standard error; untried arms get inf
                bonus = np.where(n > 0, np.sqrt(np.log(t) / np.maximum(n, 1)), np.inf)
                a = int(np.argmax(q + bonus))
            else:
                a = rng.integers(10) if rng.random() < 0.1 else int(np.argmax(q))
            r = rng.normal(true_means[a], 1.0)   # noisy reward
            n[a] += 1; q[a] += (r - q[a]) / n[a]  # running mean = the exploitation term
            total += true_means.max() - r         # opportunity cost of this pull
        regrets.append(total)
    return np.mean(regrets)

print("best arm:", true_means.argmax(), "mean:", round(true_means.max(), 2))
print("UCB regret        :", round(run("ucb"), 1))       # 75.2
print("eps-greedy (0.1)  :", round(run("eps"), 1))       # 363.3

Block 3 · Temperature as an entropy knob

Take an overconfident ten-way logit vector and watch entropy, top probability and the probability assigned to a specific less-likely token move as $T$ moves. This is the exact quantity a sampling parameter changes — nothing else in the model.

import numpy as np
rng = np.random.default_rng(2)

logits = rng.normal(0, 4, 10)                  # overconfident: logits 4 std devs apart
true_tok = int(np.argsort(logits)[-3])         # pretend the "right" token ranks 3rd

def probs(T):
    z = logits / T; z -= z.max()
    e = np.exp(z); return e / e.sum()          # = p(v)^(1/T) renormalised

for T in (0.5, 1.0, 2.0):
    p = probs(T)
    H = -(p * np.log(p)).sum()                 # entropy in nats; max = ln 10 = 2.303
    print(f"T={T}: entropy {H:.3f}, max prob {p.max():.3f}, "
          f"p(true token) {p[true_tok]:.4f}, its NLL {-np.log(p[true_tok]):.3f}")
# T=0.5: entropy 0.046, max prob 0.992, NLL 10.814  -- nearly argmax decoding
# T=2.0: entropy 0.909, max prob 0.701, NLL  3.057  -- distribution actually usable

Block 4 · The retention deck as data

Your recall deck is a data structure, and the 1/3/7-day rule is a scheduler over it. This prints the review dates for a module finished today, and the day's queue — the whole maintenance system, ten lines.

import numpy as np

# One card per checklist item: (front, back). Keep backs to one line.
deck = [
    ("gradient of BCE w.r.t. the logit", "sigmoid(z) - y"),
    ("why 1/sqrt(d_k) in attention", "keeps logit variance at 1, softmax out of saturation"),
    ("what a p-value is", "P(data this extreme | H0) — never P(H0 | data)"),
    ("UCB's second term", "uncertainty bonus, shrinks like 1/sqrt(n_a)"),
]
today = np.datetime64("2026-10-14")            # the day you finished a module
for offset in (1, 3, 7):
    print(f"day +{offset}: {today + offset}")   # 2026-10-15, 2026-10-17, 2026-10-21
print(f"{len(deck)} cards due today; budget 5-10 minutes, closed-book")

import numpy as np
rng = np.random.default_rng(3)

def bce_with_logits(z, y):
    # numerically stable BCE on logit z, label y in {0,1} -- no sigmoid-then-log blowups
    return np.maximum(z, 0) - z * y + np.log1p(np.exp(-np.abs(z)))

def sigmoid(z): return 1 / (1 + np.exp(-z))

z = rng.normal(0, 1, 5); y = rng.integers(0, 2, 5).astype(float)
claim = sigmoid(z) - y                       # the gradient we claim dBCE/dz equals

eps = 1e-6; I = np.eye(5)                    # central differences, one coordinate at a time
check = np.array([(bce_with_logits(z + eps * I[i], y).sum() -
                   bce_with_logits(z - eps * I[i], y).sum()) / (2 * eps) for i in range(5)])
print(np.round(claim, 6)); print(np.round(check, 6))
print("max abs diff:", np.abs(claim - check).max())   # ~1e-10: the claim stands

Multiple choice — instant feedback

Open questions — answer aloud, no notes

Re-answer all six in 1 day, then 3 days, then 7 days — from memory, out loud. If question 1 or 2 comes back fuzzy, that is not a surprise; it is the item going onto the deck described in Section 8.