Autoencoder
An autoencoder learns to reconstruct its input through a bottleneck, producing a compressed latent that can be used for denoising, retrieval, or as a generative seed.
Definition
An encoder maps x → z; a decoder maps z → x̂; training minimizes reconstruction error. A linear AE with squared loss spans PCA. Nonlinear and overcomplete AEs need regularization (sparsity, denoising, contractive) or they learn the identity.
Why it matters
Bottlenecks are a way to force structure. In desks this shows up as compressed factor representations, anomaly scores (reconstruction error), and as the ancestor of VAEs and modern token autoencoders.
Case
Hinton & Salakhutdinov (2006) showed deep AEs could beat PCA on images. A credit-card fraud AE that flags high reconstruction error is the same idea with a different loss.
How to read it
If the bottleneck is wider than the information in x, reconstruction proves nothing. Report latent size, noise, and a downstream probe — not only MSE.
Key papers
Landmark references with real publisher or arXiv links — not token-overlap noise.
- Reducing the Dimensionality of Data with Neural NetworksHinton, Salakhutdinov · 2006 · ScienceDeep autoencoders vs PCA.