arXiv · arXiv q-fin · 2025
This paper applies deep reinforcement learning (DRL) to optimize liquidity provisioning in Uniswap v3, a decentralized finance (DeFi) protocol implementing an automated market maker (AMM) model with concentrated liquidity. We model the liquidity provision task as a Markov Decision Process (MDP) and train an active liquidity provider (LP) agent using the Proximal Policy Optimization (PPO) algorithm. The agent dynamica…
Haonan Xu, Alessio Brini
arXiv · arXiv q-fin · 2019
Financial portfolio management is one of the problems that are most frequently encountered in the investment industry. Nevertheless, it is not widely recognized that both Kelly Criterion and Risk Parity collapse into Mean Variance under some conditions, which implies that a universal solution to the portfolio optimization problem could potentially exist. In fact, the process of sequential computation of optimal compo…
Yoshiharu Sato
arXiv · arXiv q-fin · 2024
This study investigates the development of an optimal execution strategy through reinforcement learning, aiming to determine the most effective approach for traders to buy and sell inventory within a finite time horizon. Our proposed model leverages input features derived from the current state of the limit order book and operates at a high frequency to maximize control. To simulate this environment and overcome the …
Yadh Hafsi, Edoardo Vittori
arXiv · arXiv q-fin · 2024
Markov decision processes (MDPs) are used to model a wide variety of applications ranging from game playing over robotics to finance. Their optimal policy typically maximizes the expected sum of rewards given at each step of the decision process. However, a large class of problems does not fit straightforwardly into this framework: Non-cumulative Markov decision processes (NCMDPs), where instead of the expected sum o…
Maximilian Nägele, Jan Olle, Thomas Fösel, Remmy Zen, Florian Marquardt
arXiv · arXiv q-fin · 2024
In recent years, there has been a growing trend of applying Reinforcement Learning (RL) in financial applications. This approach has shown great potential to solve decision-making tasks in finance. In this survey, we present a comprehensive study of the applications of RL in finance and conduct a series of meta-analyses to investigate the common themes in the literature, such as the factors that most significantly af…
Yahui Bai, Yuhe Gao, Runzhe Wan, Sheng Zhang, Rui Song
arXiv · arXiv q-fin · 2024
The formulaic alphas are mathematical formulas that transform raw stock data into indicated signals. In the industry, a collection of formulaic alphas is combined to enhance modeling accuracy. Existing alpha mining only employs the neural network agent, unable to utilize the structural information of the solution space. Moreover, they didn't consider the correlation between alphas in the collection, which limits the …
Tao Ren, Ruihan Zhou, Jinyang Jiang, Jiafeng Liang, Qinghao Wang
arXiv · arXiv q-fin · 2021
In recent years, quantitative investment methods combined with artificial intelligence have attracted more and more attention from investors and researchers. Existing related methods based on the supervised learning are not very suitable for learning problems with long-term goals and delayed rewards in real futures trading. In this paper, therefore, we model the price prediction problem as a Markov decision process (…
Sihang Chen, Weiqi Luo, Chao Yu
arXiv · arXiv q-fin · 2021
We describe the bailout of banks by governments as a Markov Decision Process (MDP) where the actions are equity investments. The underlying dynamics is derived from the network of financial institutions linked by mutual exposures, and the negative rewards are associated to the banks' default. Each node represents a bank and is associated to a probability of default per unit time (PD) that depends on its capital and i…
Daniele Petrone, Neofytos Rodosthenous, Vito Latora
arXiv · arXiv q-fin · 2020
We study the minimization of a spectral risk measure of the total discounted cost generated by a Markov Decision Process (MDP) over a finite or infinite planning horizon. The MDP is assumed to have Borel state and action spaces and the cost function may be unbounded above. The optimization problem is split into two minimization problems using an infimum representation for spectral risk measures. We show that the inne…
Nicole Bäuerle, Alexander Glauner
arXiv · arXiv q-fin · 2019
In this paper we explore the usage of deep reinforcement learning algorithms to automatically generate consistently profitable, robust, uncorrelated trading signals in any general financial market. In order to do this, we present a novel Markov decision process (MDP) model to capture the financial trading markets. We review and propose various modifications to existing approaches and explore different techniques like…
Souradeep Chakraborty
arXiv · arXiv q-fin · 2019
Periodic Double Auctions (PDAs) are commonly used in the real world for trading, e.g. in stock markets to determine stock opening prices, and energy markets to trade energy in order to balance net demand in smart grids, involving trillions of dollars in the process. A bidder, participating in such PDAs, has to plan for bids in the current auction as well as for the future auctions, which highlights the necessity of g…
Susobhan Ghosh, Sujit Gujar, Praveen Paruchuri, Easwar Subramanian, Sanjay P. Bhat
arXiv · arXiv q-fin · 2019
Dividend yields have been widely used in previous research to relate stock market valuations to cash flow fundamentals. However, this approach relies on the assumption that dividend yields are stationary. Due to the failure to reject the hypothesis of a unit root in the classical dividend-price ratio for the US stock market, Polimenis and Neokosmidis (2016) proposed the use of a modified dividend price ratio (mdp) as…
Vassilis Polimenis, Ioannis Neokosmidis
arXiv · arXiv q-fin · 2018
This paper studies optimal market making for large-tick assets in the presence of latency. We consider a random walk model for the asset price, and formulate the market maker's optimization problem using Markov Decision Processes (MDP). We characterize the value of an order and show that it plays the role of one-period reward in the MDP model. Based on this characterization, we provide explicit criteria for assessing…
Xuefeng Gao, Yunhan Wang
arXiv · arXiv q-fin · 2018
An automatic program that generates constant profit from the financial market is lucrative for every market practitioner. Recent advance in deep reinforcement learning provides a framework toward end-to-end training of such trading agent. In this paper, we propose an Markov Decision Process (MDP) model suitable for the financial trading task and solve it with the state-of-the-art deep recurrent Q-network (DRQN) algor…
Chien Yi Huang
arXiv · arXiv q-fin · 2012
Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations. In this paper, we propose and analyze a new method for solving large risk-averse MDPs with hybrid continuous-discrete state spaces and continuous action spaces. The proposed method …
Marek Petrik, Dharmashankar Subramanian
arXiv · arXiv · 2022
Deep Reinforcement Learning (DRL) algorithms can scale to previously intractable problems. The automation of profit generation in the stock market is possible using DRL, by combining the financial assets price "prediction" step and the "allocation" step of the portfolio in one unified process to produce fully autonomous systems capable of interacting with their environment to make optimal decisions through trial and …
Taylan Kabbani, Ekrem Duman