TradeMaster is a first-of-its kind, best-in-class open-source platform for quantitative trading (QT) empowered by reinforcement learning (RL).
It covers the full pipeline for the design, implementation, evaluation and deployment of RL-based trading methods. It contains: 1) a toolkit for efficient data collection, preprocessing and analysis; 2) a high-fidelity data-driven market simulator for mainstream QT tasks (e.g., portfolio management and algorithmic trading); 3) standard implementation of over 10 novel FinRL methods; 4) a systematic evaluation benchmark called PRUDEX-Compass.
TradeMaster could be beneficial to a wide range of communities including leading trading firms, startups, financial service providers and personal investors. We hope TradeMaster can make a change for the whole pipeline of FinRL to prevent untrustworthy results and lead successful industry deployment.
We provide a video tutorial of using docker to build a proper environment of running this project.
To help you better understand the step discribed in the video, Here are the installation tutorials for different operating systems:
We provide tutorials for users to get start with.
Algorithm | Dataset | Code link | Description |
---|---|---|---|
Classic RL | FX | tutorial | Classic RL Algorithms for Portfolio Management on FX |
DeepScalper | Bitcoin | tutorial | DeepScalper for Algorithm Trading on Crypto |
EIIE | DJ30 | tutorial | EIIE for Portfolio Management on DJ30 |
IMIT | DJ30 | tutorial | Investor Imitator for Portfolio Management on DJ30 |
SARL | DJ30 | tutorial | SARL for Portfolio Management on DJ30 |
- Colab Version: Use Google Colab resource to run TradeMaster on Cloud
The evaluation module of TradeMaster is mainly based on PRUDEX-Compass, a systematic evaluation toolkit of FinRL methods with 6 axes and 17 measures. We show some results here:
PRUDEX-Compass provides an intuitive visual means to give readers a sense of comparability and positioning of FinRL methods. The inner level maps out the relative strength of FinRL methods in terms of each axis, whereas the outer level provides a compact way to visually assess which set-up and evaluation measures are practically reported to point out how comprehensive the evaluation are for FinRL algorithms.
PRIDE-Star is a star plot to evaluate profitability,risk-control and diversity. It contains the normalized score of 8 measures.
(a) A2C | (b) PPO | (c) SAC |
Rank distribution plot is a bar plot, where the i-th column in the rank distribution shows the probability that a given method is assigned rank i in the corresponding metrics.
(a) All 4 datasets | (b) DJ30 | (c) FX |
Performance profile reports FinRL methods' score distribution of all runs across the different financial markets that are statistically unbiased and more robust to outliers.
(a) All 4 datasets | (b) DJ30 | (c) FX |
For more information of the usage of this part, please refer to this tutorial and this project
DeepScalper based on Pytorch (Shuo Sun et al, CIKM 22)
DeepTrader based on Pytorch (Wang et al, AAAI 21)
SARL based on Pytorch (Yunan Ye et al, AAAI 20)
ETTO based on Pytorch (Lin et al, 20)
Investor-Imitator based on Pytorch (Yi Ding et al, KDD 18)
EIIE based on Pytorch (Jiang et al, 17)
Classic RL based on Pytorch and Ray: PPO A2C SAC DDPG DQN PG TD3
Dataset | Data Source | Type | Range and Frequency | Raw Data | Datasheet |
---|---|---|---|---|---|
DJ30 | YahooFinance | US Stock | 2012/01/01-2021/12/31, 1day | OHLCV | DJ30 |
FX | Kaggle | FX | 2000/01/01-2019/12/31, 1day | OHLCV | FX |
Crypto | Kaggle | Crypto | 2013/04/29-2021/07/06, 1day | OHLCV | Crypto |
SZ50 | YahooFinance | CN Securities | 2009/01/02-2021-01-01, 1day | OHLCV | SZ50 |
Bitcoin | Kaggle | Crypto | 2021-04-07 11:33-2021-04-19 09:54 , 1min | LOB | Bitcoin |
OHLCV: open, high, low, and close prices; volume: corresponding trading volume
Users may download data from the following data source with personal account:
Data Source | Type | Range and Frequency | Request Limits | Raw Data |
---|---|---|---|---|
Alpaca | US Stocks, ETFs | 2015-now, 1min | Account-specific | OHLCV |
Baostock | CN Securities | 1990-12-19-now, 5min | Account-specific | OHLCV |
Binance | Cryptocurrency | API-specific, 1s, 1min | API-specific | Tick-level daily data |
CCXT | Cryptocurrency | API-specific, 1min | API-specific | OHLCV |
IEXCloud | NMS US securities | 1970-now, 1 day | 100 per second per IP | OHLCV |
JoinQuant | CN Securities | 2005-now, 1min | 3 requests each time | OHLCV |
QuantConnect | US Securities | 1998-now, 1s | NA | OHLCV |
RiceQuant | CN Securities | 2005-now, 1ms | Account-specific | OHLCV |
Tushare | CN Securities, A share | -now, 1 min | Account-specific | OHLCV |
WRDS | US Securities | 2003-now, 1ms | 5 requests each time | Intraday Trades |
YahooFinance | US Securities | Frequency-specific, 1min | 2,000/hour | OHLCV |
TradeMaster supports financial data with open, high, low, close, volume (OHLCV) raw informations as:
We compute 10 technical indicators to describe the financial markets:
Users can adapt their data with prefered features by changing the data loading and feature calculation part with corresponding input and output size. We plan to support limit order book (LOB) and altervative data such as text and graph in the future.
|-- agent
| |-- ClassicRL
| |-- DeepScalper
| |-- DeepTrader
| |-- EIIE
| |-- Investor_Imitator
| |-- SARL
|-- config
| |-- input_config
| |-- output_config
|-- data
| |-- download_data.py
| |-- preprocess.py
| |-- data
| |-- BTC
| |-- dj30
| |-- exchange
| |-- sz50
|-- env
| |-- AT
| |-- OE
| |-- PM
|-- experiment
|-- figure
|-- result
|-- tutorial
| |-- ClassRL_for_PM_on_FX.ipynb
| |-- DeepScalper_for_AT_on_Bitcoin.ipynb
| |-- EIIE_for_PM_on_DJ30.ipynb
| |-- IMIT_for_PM_on_DJ30.ipynb
| |-- SARL_for_PM_on_DJ30.ipynb
| |-- Visualization.ipynb
|-- visualization
| |-- compass
| |-- exen
| |-- ocatgon
| |-- performance_profile
| |-- rank
|-- README.md
|-- requirement.txt
Deep Reinforcement Learning for Quantitative Trading: Challenges and Opportunities (IEEE Intelligent Systems 2022)
Commission Fee is not Enough: A Hierarchical Reinforced Framework for Portfolio Management (AAAI 21)
Reinforcement Learning for Quantitative Trading (Survey)
PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets
- This repository is developed and maintained by AMI group at Nanyang Technological University
- If you want to make contributions to
TradeMaster
, please create pull requests.
We have positions for software engineer, RA and postdoc. If you are interested in working at the intersection of RL and financial trading, feel free to send an email to shuo003@e.ntu.edu.sg with your CV.