49 · Capstone: comparing the families
AE, VAE, GAN, diffusion, autoregressive — five generative families, five sets of trade-offs across quality, diversity, speed, stability, and likelihood. There is no single winner; choosing the right one is the skill this track gives you.
There is no single best generative model: autoencoder, VAE, GAN, diffusion, and autoregressive each trade off sample quality, diversity, sampling speed, training stability, and likelihood differently — so the skill is matching the family to the job.
Without this:
Knowing WHICH generative model to reach for — and being able to measure how good a generator is — is what turns five separate lessons into a usable mental map of generative AI.
You've now met five generative families. This capstone lays them side by side so you can choose the right tool — because there is no single winner, only different trade-offs.
- Autoencoder (AE). Not really a generator: it reconstructs through a bottleneck. Great for compression, denoising, and anomaly detection, but its latent space has holes, so you can't sample new data cleanly.
- VAE. A true generator with a smooth, sampleable latent space and stable training (an explicit likelihood via the ELBO). The price is blurry samples, because its averaging objective hedges.
- GAN. Strikingly sharp samples from an adversarial game — but unstable training, prone to mode collapse (low diversity), and no likelihood to evaluate.
- Diffusion. High quality AND stable training by iterated denoising — the current state of the art for images. The catch is slow sampling (many reverse steps), though distillation and better samplers keep shrinking that.
- Autoregressive (GPT-style). Generate one token/pixel at a time, conditioning on all prior outputs. Gives exact likelihood and dominates text/sequences, but sampling is sequential (one step per element) and so can be slow for long outputs.
Lay them on five axes — sample quality, diversity, sampling speed, training stability, likelihood — and no row wins every column. VAEs are stable but blurry; GANs are sharp but unstable; diffusion is high-quality but slow; autoregressive gives exact likelihood but samples sequentially. Engineering is picking the trade-off that fits the job.
To compare generators you need a metric. The standard for images is FID (Fréchet Inception Distance): embed real and generated images with an Inception network, fit a Gaussian to each set of features, and measure the Fréchet distance between the two Gaussians — lower is better (closer to the real distribution). The cell below builds a 1-D toy version so the formula is concrete, then the read-along table turns all of this into a practical "which model should I use?" decision guide.
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
A decision table mapping goals to families: sharp+fast → GAN; stable high-quality → diffusion; smooth latent/reconstruction → VAE; text with exact likelihood → autoregressive/LLM; compression/anomaly detection → autoencoder.
Which generative family is known for sharp samples but unstable adversarial training and no explicit likelihood?
- Five generative families trade off differently: AE (compress/denoise, not a generator), VAE (stable, sampleable, blurry), GAN (sharp, unstable, no likelihood), diffusion (high-quality, stable, slow), autoregressive (exact likelihood, sequential).
- There is no single winner across quality, diversity, sampling speed, training stability, and likelihood — matching the family to the job is the core skill.
- FID (Fréchet Inception Distance) measures generator quality as the Fréchet distance between Gaussians fit to Inception features of real vs generated images — lower is better.
Every generative-AI decision starts here: a startup choosing diffusion for product imagery, a research team reporting FID, an engineer reaching for a VAE for anomaly detection or an LLM for text — all of today's frontier systems (Stable Diffusion, GPT, Claude) are these families composed.
If you remove it: You'd know five generative models in isolation but couldn't choose between them or measure them — missing the mental map that connects the perceptron of lesson 1 to every modern generative system.