Variational Autoencoder
A VAE is a probabilistic autoencoder: the encoder outputs a distribution q(z|x), the decoder p(x|z), and training maximizes an ELBO with a KL term that keeps the latent well-behaved.
Definition
Kingma & Welling’s VAE turns the AE bottleneck into inference of a latent Gaussian (usually). The loss is reconstruction (or likelihood) plus KL(q(z|x) ‖ p(z)). The reparameterization trick lets gradients flow through the sampler.
Why it matters
VAEs are the cleanest intro to latent-variable deep generative models. They underly many disentangling and anomaly papers, and they are the conceptual sibling of diffusion’s variational view.
Case
A VAE on faces learns a smooth z you can interpolate. A VAE on returns with a too-strong KL collapses z to the prior (“posterior collapse”) and the decoder ignores the latent — pretty ELBO, useless codes.
How to read it
Watch the KL term. Near-zero KL means the latent is unused. Huge KL means the decoder is ignored. β-VAE and free-bits are knobs, not proofs of “disentanglement.”
Key papers
Landmark references with real publisher or arXiv links — not token-overlap noise.
- Auto-Encoding Variational BayesKingma, Welling · 2014 · ICLRThe VAE paper.