Overfitting
Overfitting is when a model fits training idiosyncrasies instead of the transferable regularity, so held-out or live error rises even as train loss falls.
Definition
A model overfits when train performance keeps improving while true risk does not. Capacity, small N, leakage, and non-stationarity all cause it. The diagnostic is a proper validation protocol — not a lower training loss.
Why it matters
In markets, overfitting is the default. A factor with 40 years of daily data still has few independent regimes. A backtest that never saw 2008, 2020, or a QT episode is an overfit story waiting to print.
Case
A 200-feature equity model with walk-forward Sharpe 2.4 and no cost model dies to 0.3 live. The train window memorized a factor-crowding episode. That is overfitting, not “regime change” as an excuse.
How to read it
Require a locked validation set, nested CV or purged CV for time series, and a pre-registered metric. If you tuned on the test set, you no longer have a test set.
Key papers
Landmark references with real publisher or arXiv links — not token-overlap noise.
- The Problem of OverfittingHawkins · 2004 · J. Chem. Inf. Comput. Sci.Clear statement of overfit vs signal in high-capacity models.