Ten ways to be wrong, each with a diagnostic. Label every entry in your error log with one of these; the pattern that emerges is your actual curriculum.
Mistakes in this material are not random. They fall into ten families, and knowing which family you are prone to is worth more than knowing another result.
Use the log honestly. One line per error: what you did, what was right, and the class. After twenty entries the distribution will be lopsided, and the two heaviest classes are where your remaining study time should go.
Symptom: the method is right and the number is slightly wrong. Dropped bias terms in a parameter count; a floor not taken in a convolution size; a missing continuity correction; a strict inequality read as non-strict; “the smallest \( t \) exceeding” confused with “the \( t \) solving.”
Diagnostic: recompute the smallest instance by hand. A one-layer network, a three-block stack, a two-token vocabulary — if the small case disagrees with your formula, the formula has a missing term.
Symptom: an optimum that looks plausible and is wrong. The canonical instance is writing the compute-optimal condition as \( AN^{-a} = BD^{-b} \) rather than the exponent-weighted \( aAN^{-a} = bBD^{-b} \). A close cousin is adding standard deviations where variances should be added.
Diagnostic: never assert a balance condition; derive it by substitution. If you did not differentiate, you guessed.
Symptom: concluding that a technique which helps must have improved the aggregate you first reached for. The reversal trick leaves the mean lag exactly unchanged and makes the maximum worse, yet helps. Two agents with equal average performance across environments need not have equal weighted scores.
Diagnostic: when a technique helps, ask which statistic actually moved — and be prepared for the answer to be “the shape of the distribution, not its mean.”
Symptom: attributing to the model what belongs to the ruler. An exact-match cliff read as a discontinuous capability; a contaminated benchmark read as skill; coverage quoted as accuracy; a perplexity drop that sounds enormous and is half a bit.
Diagnostic: for any reported number, ask what would change it besides the model. If the answer is “the threshold,” “the test set's provenance,” or “the selector,” you are looking at a measurement, not a capability.
Symptom: the magnitude is right and the sign of the reasoning is inverted. Believing noisy weights are the expensive ones (they are the cheap ones); that quantization slows decoding (it speeds it up); that factorizing a large kernel costs expressiveness (it adds a nonlinearity); that voting always helps (below \( p = \tfrac12 \) it hurts); that dilation reduces resolution (it preserves it).
Diagnostic: state the claim, then construct the extreme case. Infinite precision, zero bits, \( p = 0 \), one micro-batch. Extremes make reversed signs obvious.
Symptom: a factor of \( \ln 2 \) adrift, or a decade confused with a factor. Mixing natural and base-two logarithms; losing the sign of \( \ln(0.x) \); reporting “1.36” when the answer is \( 10^{1.36} \).
Diagnostic: carry units. “Decades” and “factors” are different quantities; bits and nats are different quantities. Writing the unit after every number catches this class almost entirely.
Symptom: a per-unit quantity reported as a total or vice versa. A KV cache quoted per token when the total was wanted; tokens per cycle reported as tokens per second; a bubble fraction confused with an absolute idle time.
Diagnostic: say the quantity aloud with its denominator. “Bytes per token,” “tokens per second,” “operations per byte.” A quantity you cannot name with a denominator is one you have not pinned down.
Symptom: believing a computational improvement changed what is being computed. Streaming softmax is exact; speculative decoding is exact; the preference-optimization collapse changed the estimator, not the objective. Quantization is the one technique in this group that genuinely does change the function — which is precisely why the distinction is worth policing.
Diagnostic: for each technique, ask whether two implementations would produce identical outputs given identical randomness. If yes, it is an estimator change and nothing about the model's behavior may be attributed to it.
Symptom: “this makes it faster” or “this makes it smaller,” with no resource specified. Low-rank adaptation saves optimizer state, not training arithmetic. Grouped-query attention saves cache, not weight reads. A mixture-of-experts saves per-token compute, not memory.
Diagnostic: refuse to evaluate any efficiency claim until the budget is named. There are four: training operations, serving operations, memory bytes and bandwidth — which splits into weights, optimizer state and activations — and human preference data. Most confused arguments in this field dissolve at this question.
Symptom: a formula applied where its assumptions have collapsed. A group of samples all correct, giving zero advantage and no learning signal. Voting below the threshold. A pipeline with one micro-batch. “Diverse” samples drawn at temperature zero. Guidance with a negative weight.
Diagnostic: for every formula you derive, evaluate it at both extremes of its parameter before using it. This is also where examiners and interviewers live, because the degenerate case reveals whether you derived the result or memorized it.
Two habits make the taxonomy work. First, log the error when it happens, not at the end of a session — the reasoning that produced it is only recoverable while it is fresh. Second, when a class accumulates five entries, do not simply resolve to be careful. Go back to the chapter that introduced the underlying derivation and reproduce it from scratch; a repeated trap almost always signals that a derivation was memorized rather than built.
A note on T4 and T9, which between them account for most professional disagreements in this field: they are not really arithmetic errors at all. They are failures to ask what a number refers to. That question — what exactly is being measured, and out of which budget — is the habit this book most wants to leave you with.