Gradient descent and momentum on an ill-conditioned valley — Chapter 3

The Arithmetic
of Intelligence

A problem course in modern machine learning, from the pathological valley to direct preference optimization.

Anshu Avinash
Download PDF edition ↓

Preface

This book teaches machine learning the way physics is taught to engineers: as a small set of results you derive, drill, and then recognize everywhere.

Most introductions to the field are surveys. They tell you that residual connections help, that larger models should see more data, that a preference-tuning method called DPO replaced a more complicated pipeline. A survey leaves you fluent in names and helpless in front of a design decision. When someone asks whether your serving cluster is memory-bound, whether a benchmark jump is capability or measurement, or why a reinforcement-learning recipe deleted its critic network, the names do not answer. A short calculation does.

The premise here is that modern machine learning — for all its scale and noise — rests on roughly forty short calculations. Each is the kind of thing a first-year engineering student could do, if someone set it up properly: a stability condition, a variance count, a constrained minimization, a geometric series, a Bayes flip. The field's landmark papers are these calculations wearing costumes. Learn the calculations and the costumes turn transparent; when next quarter's paper arrives, you will see within a page which budget it spends and which derivation prices it.

Accordingly, nothing in this book asks you to memorize a benchmark score, a release date, or an author list. Where a constant from the literature is needed — a scaling exponent, a data-law slope, an accelerator's bandwidth — it is given, the way g = 9.8 m/s² is given in a physics paper. What is never given is the model you must build to use it. That is the work, and it is the only thing examined.

The book covers two eras in one arc. Part II is the classical era, 2012–2022: why training deep networks was hard, what architecture did about it, how scale became a budgeting problem, and why compression is the master currency underneath all of it. Part III is the modern era, 2022 onward: the economics of inference, attention rebuilt around memory traffic, sparsity and thrift, the alignment pipeline, inference-time reasoning, diffusion, and the full-lifecycle cost of a deployed model. The ordering inside Part III is deliberate — systems before alignment — because without the cost vocabulary of bytes, bandwidth, and budgets, the alignment era's design choices read as fashion instead of forced moves.

How to use this book

The unit of progress is a derivation, not a chapter

Appendix A lists the thirty-eight core derivations. You are done with a chapter when you can produce its derivations on blank paper, unprompted — not when you have read it. Three of them (the compute-optimal allocation of Chapter 7, the DPO collapse of Chapter 14, and one you meet early in Chapter 3) carry a special protocol: reproduce them from nothing on three separate days before moving on.

Work every drill; keep an error log

Drills appear inside sections with their answers one click away; end-of-chapter exercise sets are graded A (drills), B (problems), and C (challenges), with full solutions in Appendix D. Keep a notebook with one line per mistake: what you did, what was right, and which trap class it belongs to. The trap taxonomy (Appendix B) names the ten ways students reliably go wrong. Twenty hours of honest error-logging is worth more than any rereading.

Let code confirm the math

From Chapter 11 onward a lab manual (Appendix C) runs in parallel. Every lab's acceptance criterion is a number your derivation predicted: your streaming softmax must match a library implementation to six decimal places; your preference-tuning loss must begin at exactly ln 2 — and you must prove why before you run it. When code and derivation disagree, one of them is wrong, and finding which is the curriculum.

Prerequisites

School-level mathematics — logarithms, elementary calculus, basic probability — and, for the labs, working Python. Chapter 1 sharpens eight mathematical reflexes and nothing else; if its gate feels easy, skip ahead without guilt. You are also assumed to know roughly what a neural network is — a stack of weighted sums and nonlinearities, trained by following a gradient. Chapter 3 states every such object precisely before it is used, but it states them; it does not motivate them.

Contents

Part I · Foundations
  1. 1The Mathematical ToolkitEight reflexes: power laws, variance algebra, the binomial, constrained minima, geometric series, counting in bits, Bayes, counting operations
  2. 2The ObjectsLayer, gradient step and momentum, softmax and cross-entropy, Jacobians, tokens, normalization, divergence
Part II · The Classical Era (2012–2022)
  1. 3The Geometry of TrainingThe pathological valley, conditioning, momentum's √κ, and the statistics of initialization
  2. 4The Architecture of DepthConvolution arithmetic, the indestructible identity, depth as an ensemble of shallow paths, dilation
  3. 5Memory and GatesGradient horizons, the forget gate's bias, why dropout must respect the recurrent path, data power laws
  4. 6AttentionScore statistics and √d, cost crossovers, path lengths, and the reversal trick's unchanged mean
  5. 7The Economics of ScaleThe compute-optimal allocation, exponent-weighted balance, and what a halving of loss really costs
  6. 8The Machinery of ScalePipeline bubbles from a timing diagram, sharding arithmetic, and the memory wall
  7. 9MeasurementEmergence as a property of the ruler, contamination algebra, and the Bayes of a leaked benchmark
  8. 10Compression and OccamMDL in five costumes, the length prior's theorem, grokking, double descent, and a measure of intelligence
Part III · The Modern Era (2022– )
  1. 11The Price of a TokenKV caches, rooflines, and why decoding is a memory problem
  2. 12Bytes over FLOPsOnline softmax, IO-aware attention, and the exact economics of speculative decoding
  3. 13Sparsity and ThriftMixture-of-experts accounting, quantization error, and LoRA's attack on the optimizer budget
  4. 14Teaching PreferencesBradley–Terry, PPO's pessimism, the DPO collapse, and GRPO's deleted critic
  5. 15Thinking at Inference TimeCoverage versus selection, matched-compute comparisons, and what elicits versus what creates
  6. 16Generation by DenoisingThe closed-form forward process, signal-to-noise schedules, and guidance as extrapolation
  7. 17The Whole LifecycleTraining cost plus serving cost, break-even lifetimes, distillation, and the four budgets
Appendices
  1. AThe Derivation BankAll thirty-eight results, stated for blank-paper reproduction
  2. BThe Trap TaxonomyTen classes of reliable error, with diagnostics
  3. CThe Lab ManualEight builds, each accepted only when it matches a predicted number
  4. DSolutionsWorked answers to the B and C exercises, and the final examination blueprint
  5. ENotation and GlossaryEvery symbol, the thirteen that carry more than one meaning, and where each term was defined