Destroy an image gradually with noise until nothing remains, then train a network to undo one step of the destruction. Run it backward and you have a generator — and the whole construction rests on the fact that Gaussians compose.
Take a photograph and add a little noise. Add a little more. Keep going until nothing is left but static. Now train a network to undo one step of that, and run it backwards from pure noise. What comes out is a photograph that never existed. It is a strange way to build a generator, and the reason it is tractable at all is a fact about Gaussians you met in Chapter 1.
The corruption is defined one step at a time: blend the current image with fresh Gaussian noise,
\[ x_t = \sqrt{1-\beta_t}\,x_{t-1} + \sqrt{\beta_t}\,\epsilon, \qquad \epsilon \sim \mathcal{N}(0, I), \]with a small schedule \( \beta_t \). Applied a thousand times, this leaves pure noise. Simulating a thousand steps to produce a training example would be intolerable — but we do not have to, because a chain of Gaussian blends is itself a single Gaussian blend.
There is a practical problem to notice before the mathematics. Training requires showing the network images corrupted to every possible degree. If reaching corruption level eight hundred meant actually running eight hundred steps of noising for every training example, the method would be far too slow to use. So the question that has to be answered first is whether we can jump straight to any level in one operation. The answer is yes, and the reason is a fact about Gaussians you already know.
A thousand steps in one jump
Take two steps with constants \( \beta_1, \beta_2 \), and write \( \alpha_t = 1-\beta_t \). Then \( x_1 = \sqrt{\alpha_1}x_0 + \sqrt{1-\alpha_1}\,\epsilon_1 \) and
\[ x_2 = \sqrt{\alpha_2}\,x_1 + \sqrt{1-\alpha_2}\,\epsilon_2 = \sqrt{\alpha_2\alpha_1}\,x_0 + \sqrt{\alpha_2(1-\alpha_1)}\,\epsilon_1 + \sqrt{1-\alpha_2}\,\epsilon_2. \]The two noise terms are independent Gaussians, so — variances add — their sum is a single Gaussian of variance \( \alpha_2(1-\alpha_1) + (1-\alpha_2) = 1 - \alpha_1\alpha_2 \). Hence \( x_2 = \sqrt{\bar\alpha_2}\,x_0 + \sqrt{1-\bar\alpha_2}\,\epsilon \) with \( \bar\alpha_2 = \alpha_1\alpha_2 \). By induction,
\[ q(x_t \mid x_0) = \mathcal{N}\bigl(\sqrt{\bar\alpha_t}\,x_0,\ (1-\bar\alpha_t)I\bigr), \qquad \bar\alpha_t = \prod_{s \le t}(1-\beta_s). \]Any noise level is reachable in one draw. Note the coefficients: \( \sqrt{\bar\alpha} \) multiplies the signal and \( \sqrt{1-\bar\alpha} \) the noise, so their variances sum to 1 — the check that catches nearly every algebra slip here.
The quantity that gives the schedule meaning is the signal-to-noise ratio, \( \mathrm{SNR}(t) = \bar\alpha_t/(1-\bar\alpha_t) \), which slides from very large to nearly zero. Training samples a random \( t \), corrupts a real image to that level, and asks the network to predict the noise that was added. The network thus learns to denoise at every level at once — a curriculum in a single objective.
Reading a schedule
Take a constant \( \beta = 0.02 \), so \( \bar\alpha_t = 0.98^t \). The half-signal point, where \( \mathrm{SNR} = 1 \), is where \( 0.98^t = 0.5 \): \( t = \ln 0.5/\ln 0.98 = 0.693/0.0202 = \) 34. By \( t = 100 \), \( \bar\alpha = 0.98^{100} = e^{-2.02} = 0.133 \) and the SNR is \( 0.133/0.867 = 0.15 \) — structure nearly gone.
Observe that this is arithmetically the same computation as Chapter 9's emergence frontier \( p^k = \tfrac12 \). Different subject, identical reflex.
(a) With \( \beta = 0.01 \), at what step does \( \mathrm{SNR} = 1 \)? (b) If \( \bar\alpha = 0.36 \), write \( x_t \) explicitly in terms of \( x_0 \) and \( \epsilon \).
(a) \( 0.99^t = 0.5 \Rightarrow t = 0.693/0.01005 = 69 \). (b) \( x_t = 0.6\,x_0 + 0.8\,\epsilon \), since \( \sqrt{0.36} = 0.6 \) and \( \sqrt{0.64} = 0.8 \); note \( 0.36 + 0.64 = 1 \). ✓
Generation reverses the ladder: start from noise and repeatedly remove the network's predicted noise. One reverse step is
\[ x_{t-1} = \frac{1}{\sqrt{\alpha_t}}\left(x_t - \frac{\beta_t}{\sqrt{1-\bar\alpha_t}}\,\epsilon_\theta(x_t, t)\right) + \sigma_t z, \qquad z \sim \mathcal{N}(0, I), \]where \( \epsilon_\theta \) is the network's estimate of the noise in \( x_t \) and \( \sigma_t \) sets how much fresh noise is injected on the way back down. This update is given: deriving it requires the variational argument this chapter's readings set aside, and nothing below depends on where it came from. What matters is its shape — one network evaluation per step, and a step index that must walk down the ladder it walked up.
That shape is the cost. Run it as written and generating one image takes as many network evaluations as the forward process had steps, often a thousand. Setting \( \sigma_t = 0 \) makes the reverse process deterministic, and a deterministic trajectory can be sampled at a subset of its steps: take every twentieth \( t \) instead of every \( t \), and the same schedule needs fifty evaluations rather than a thousand.
Now price it, because the arithmetic here is the whole practical story and it is one line. Cost is proportional to the number of network evaluations \( n_{\text{eval}} \), so a stride \( k \) over a \( T \)-step schedule costs
\[ n_{\text{eval}} = \left\lceil \frac{T}{k} \right\rceil, \]and the saving is the factor \( k \), exactly and with nothing hidden in it. At \( T = 1000 \) and \( k = 20 \) that is fifty evaluations against a thousand, a twentyfold reduction in the cost of every image the model ever generates. Quality degrades as \( k \) grows, gently at first and then not, and where that knee sits is measured rather than derived — the readings give the table.
To make generation follow a condition — a text prompt, say — the model is trained both with the condition and, some fraction of the time, without it. At sampling time the two predictions are combined:
\[ \tilde\epsilon = \epsilon_u + w\,(\epsilon_c - \epsilon_u), \]where \( \epsilon_u \) is the unconditional prediction and \( \epsilon_c \) the conditional. At \( w = 0 \) this is unconditional; at \( w = 1 \) it is exactly the conditional prediction. At \( w > 1 \) — the usual setting — it goes past the conditional prediction, in the direction that distinguishes conditional from unconditional.
Outside the interval
Let \( \epsilon_u = 0.20 \), \( \epsilon_c = 0.30 \), \( w = 7.5 \). Then \( \tilde\epsilon = 0.20 + 7.5(0.10) = \) 0.95 — far outside \( [0.20, 0.30] \). Guidance is extrapolation, not interpolation. That is both why it sharpens adherence to the prompt and why large \( w \) over-saturates and collapses diversity: you are pushing well beyond where either prediction was trained to be valid.
Three recurring slips. Reading \( w \) as a mixing weight in \( [0,1] \) — it is not; values of 5 to 10 are routine. Confusing \( \alpha_t = 1-\beta_t \) with the cumulative \( \bar\alpha_t \) — the bar denotes the product over all steps so far. And placing the square roots wrongly: it is \( \sqrt{\bar\alpha} \) on the signal and \( \sqrt{1-\bar\alpha} \) on the noise, verifiable at a glance because the variances must sum to one.
With \( w = 3 \), \( \epsilon_u = 0.5 \), \( \epsilon_c = 0.4 \), compute \( \tilde\epsilon \) and note its direction relative to the two inputs.
\( 0.5 + 3(0.4 - 0.5) = 0.5 - 0.3 = 0.2 \) — below both, because the conditional lies below the unconditional and guidance extrapolates in that direction.
Running this process at full image resolution is wasteful, because most pixel-level detail is texture that a decoder can supply. The remedy is to compress the image into a much smaller latent representation, run the entire diffusion process there, and decode at the end. The saving is roughly the compression ratio in both computation and activation memory — often a factor of dozens.
Structurally this is the same division of labour we saw in Chapter 4's bottleneck block and Chapter 13's expert routing: do the expensive work in the narrow space, and let cheap machinery handle the wide one. It is worth recognizing the pattern, because it recurs whenever a field gets serious about cost.
The forward corruption collapses to a single Gaussian because variances add — making training tractable at any noise level in one draw. The signal-to-noise ratio gives the schedule its meaning, and its half-signal point is a logarithm. Sampling can be strided, turning a thousand evaluations into fifty. Guidance extrapolates beyond the conditional prediction, buying adherence at the cost of diversity. And running the whole process in a compressed latent space is the same narrow-middle economy that recurs throughout this book. One chapter remains: what all of this costs over a model's entire life, and how that changes what model you should have trained in the first place.
A · Drills
B · Problems
C · Challenge
Reproduce Derivation 16.1 — the two-step composition and the induction — on blank paper, and produce the SNR half-point calculation from nothing. Attempt B-1 and B-2 closed book. You pass when the coefficient placement is automatic and you check it every time by summing variances to one.