Backtest Overfitting
Backtest Overfitting — False discovery from mining historical patterns that do not persist out-of-sample.
Definition
Backtest Overfitting refers to false discovery from mining historical patterns that do not persist out-of-sample. Keep that definition fixed when comparing series, managers, or regimes — renaming the same tape does not create a new signal.
Why it matters
It shows up in factor research, attribution, and capacity debates — whether a return slice is skill, style, or fee drag. When false discovery from mining historical patterns that do not persist out-of-sample shifts, related hedges, limits, and narratives usually need an explicit update rather than a quiet assumption.
Case
Suppose a desk is positioned for the opposite of what backtest overfitting is saying. If false discovery from mining historical patterns that do not persist out-of-sample moves against that book, the first question is not “is the story clever?” but whether size, hedges, and stop logic still match the observation.
How to read it
Check definition stability across universes, costs, and regimes before treating a backtest as portable. Prefer a short written null hypothesis for Backtest Overfitting: what would falsify the current reading in the next window?