Entropy
Shannon entropy H(P) = −Σ p log p is the expected surprise of a distribution — a measure of uncertainty used in information theory, portfolio tilts, and some max-ent priors.
Definition
Entropy is maximized by the uniform (discrete) or by Gaussians under a variance constraint (differential entropy). Conditional entropy and mutual information are the multivariate language. It is not thermodynamic entropy unless you build that bridge on purpose.
Why it matters
Max-ent views (and some risk-budgeting stories) use it as a ‘least assumption’ prior. Model calibration that matches a few moments and maxes entropy is a standard trick.
Case
A implied-vol smile can be turned into a risk-neutral density; its entropy says how ‘wild’ that density is versus a lognormal.
How to read it
Discrete vs differential entropy are not in the same units. Do not compare a bit-entropy of a classifier to a differential entropy of returns without a conversion.
Key papers
Landmark references with real publisher or arXiv links — not token-overlap noise.
- A Mathematical Theory of CommunicationShannon · 1948 · Bell System Technical JournalShannon entropy.