Residual Network
A ResNet learns a residual f(x) added back to x, so extra layers can default to identity. That skip connection made 100+ layer nets trainable.
Definition
He et al. formulate a block as y = x + F(x). If F is easier to learn than a full mapping H(x), deep stacks stop degrading. The same idea is the residual stream of every Transformer block.
Why it matters
Without residuals, depth hurts. With them, you can scale depth until other limits (data, optimization, memory) bind. Pre-norm Transformer + residual is the modern default.
Case
ImageNet training of a 56-layer plain CNN was worse than a 20-layer net. The 152-layer ResNet reversed that and won ILSVRC 2015. The skip was the invention, not “more layers” as a slogan.
How to read it
If a deep train is worse than a shallow one, first ask whether the skip path is intact (dtype, init, LN placement) before adding more tricks.
Key papers
Landmark references with real publisher or arXiv links — not token-overlap noise.
- Deep Residual Learning for Image RecognitionHe, Zhang, Ren, Sun · 2016 · CVPRResNet.