Neural Network
A neural network is a layered function approximator: units compute a weighted sum, apply a nonlinearity, and pass the result forward so the whole stack can learn a mapping from inputs to outputs.
Definition
A neural network composes linear maps and elementwise nonlinearities. A multilayer perceptron is the simplest useful form: x ↦ f(Wₙ … f(W₁x + b₁) … + bₙ). Depth lets the model build hierarchical features; width controls capacity at each layer. Training usually minimizes a loss by gradient descent, with gradients obtained by backpropagation.
Why it matters
Almost every modern AI system — CNN, LSTM, Transformer, diffusion U-Net — is a neural network with a particular connectivity pattern. If you cannot say what the inputs, loss, and inductive bias are, you cannot debug the model when it fails in production or in a trading book.
Case
A feed-forward net on yesterday’s returns will often look sharp in-sample and collapse live: the architecture can fit noise as easily as signal. The same math, given images or tokens and the right inductive bias, is what made AlexNet and GPT work.
How to read it
Name the architecture, loss, data, and regularization before trusting a result. Universal approximation says a wide enough net can fit a function — it does not say the fit will generalize, or that the learned features are causal.
Key papers
Landmark references with real publisher or arXiv links — not token-overlap noise.
- Learning representations by back-propagating errorsRumelhart, Hinton, Williams · 1986 · NatureThe paper that made gradient training of multilayer nets practical.
- Gradient-based learning applied to document recognitionLeCun, Bottou, Bengio, Haffner · 1998 · Proc. IEEEConvolutional nets + gradient learning on a real task (LeNet).