Reinforcement Learning
Reinforcement learning trains a policy to maximize expected return by interacting with an environment: states, actions, rewards, and (usually) a discount factor.
Definition
An agent observes state s, takes action a, receives reward r and next state s′. The objective is expected discounted return. Methods are value-based (learn Q), policy-based (learn π), or actor–critic (both). The environment may be a game, a simulator, or a market — the last is non-stationary and only partially observed.
Why it matters
RL is how agents act, not only predict. Execution algorithms, market making, and RLHF all sit on this formalism. The hard part in markets is the reward and the simulator, not the update rule.
Case
DQN on Atari learned from pixels because the reward was the game score and the env was resettable. The same DQN on fills, with a handmade reward, will harvest the reward bug (markouts, leftover inventory) rather than “make money.”
How to read it
Write the MDP down: state, action, reward, discount, episode boundary. If those are vague, the algorithm paper you cite is decoration.
Key papers
Landmark references with real publisher or arXiv links — not token-overlap noise.
- Playing Atari with Deep Reinforcement LearningMnih et al. · 2013 · NeurIPS workshopDeep Q-learning from pixels — the modern deep RL spark.