/FinRL-Library

A Deep Reinforcement Learning Library for Automated Trading in Quantitative Finance. NeurIPS 2020. Please star.

Primary LanguageJupyter NotebookMIT LicenseMIT

FinRL: A Deep Reinforcement Learning Library for Quantitative Finance Twitter Facebook Google+ Linkedin

Downloads Downloads Python 3.6 PyPI

FinRL is the open source library with a unified framework for practitioners to pipeline the strategy development. In reinforcement learning (or deep RL), an agent learns by interacting with an environment, in a trial-and-error manner, achieving a balance between exploration and exploitation. The open source community AI4Finance (to efficiently automate trading) provides educational resources to learn about deep reinforcement learning (DRL) in quantitative finance.

To contribute? Please check the call for contributions at the end of this page.

Feel free to report bugs using Github issues, join our mailing list: AI4Finance, and discuss FinRL in the slack channel:



Roadmaps of FinRL:

FinRl 1.0: entry-level toturials for beginners, with a demonstrative and educational purpose.

FinRl 2.0: intermediate-level framework for full-stack developers and professionals.

FinRL provides a unified machine learning framework for various markets, SOTA DRL algorithms, benchmark finance tasks (portfolio allocation, Cryptocurrency trading), live trading, etc.

Table of Contents

Prior Arts:

We published papers in FinTech and now arrive at this project:

4). FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance, Deep RL Workshop, NeurIPS 2020.

3). Deep Reinforcement Learning for Automated Stock Trading: An Ensemble Strategy, paper and codes, ACM International Conference on AI in Finance, ICAIF 2020.

2). Multi-agent Reinforcement Learning for Liquidation Strategy Analysis, paper and codes. Workshop on Applications and Infrastructure for Multi-Agent Learning, ICML 2019.

1). Practical Deep Reinforcement Learning Approach for Stock Trading, paper and codes, Workshop on Challenges and Opportunities for AI in Financial Services, NeurIPS 2018.

News

[Towardsdatascience] FinRL for Quantitative Finance: Tutorial for Single Stock Trading

[Towardsdatascience] FinRL for Quantitative Finance: Tutorial for Multiple Stock Trading

[Towardsdatascience] FinRL for Quantitative Finance: Tutorial for Portfolio Allocation

[Towardsdatascience] Deep Reinforcement Learning for Automated Stock Trading

[Analyticsindiamag.com] How To Automate The Stock Market Using FinRL (Deep Reinforcement Learning Library)?

[量化投资与机器学习] 基于深度强化学习的股票交易策略框架(代码+文档)

[Neurohive] FinRL: глубокое обучение с подкреплением для трейдинга

Overview

A YouTube video about FinRL library. [YouTube] AI4Finance Channel for quant finance.

DRL Algorithms

We implemented Deep Q Learning (DQN), Double DQN, DDPG, A2C, SAC, PPO, TD3, GAE, MADDPG, InterSAC, InterAC, MuZero, etc. using PyTorch and OpenAI Gym.

Status

Version History [click to expand]
  • 2020-12-14 Upgraded to Pytorch with stable-baselines3; Remove tensorflow 1.0 at this moment, under development to support tensorflow 2.0
  • 2020-11-27 0.1: Beta version with tensorflow 1.5

Installation:

Clone this repository

git clone https://github.com/AI4Finance-LLC/FinRL-Library.git

Install the unstable development version of FinRL:

pip install git+https://github.com/AI4Finance-LLC/FinRL-Library.git

Docker Installation

Build the container:

$ docker build -f docker/Dockerfile -t finrl docker/

Start the container Note: The default container run starts jupyter lab in the root directory, allowing you to run scripts, notebooks, etc.

$ docker run -it --rm -v ${PWD}:/home -p 8888:8888 finrl

Prerequisites

For OpenAI Baselines, you'll need system packages CMake, OpenMPI and zlib. Those can be installed as follows

Ubuntu

sudo apt-get update && sudo apt-get install cmake libopenmpi-dev python3-dev zlib1g-dev libgl1-mesa-glx

Mac OS X

Installation of system packages on Mac requires Homebrew. With Homebrew installed, run the following:

brew install cmake openmpi

Windows 10

To install stable-baselines on Windows, please look at the documentation.

Create and Activate Python Virtual-Environment (Optional but highly recommended)

cd into this repository

cd FinRL-Library

Under folder /FinRL-Library, create a Python virtual-environment

pip install virtualenv

Virtualenvs are essentially folders that have copies of python executable and all python packages.

Virtualenvs can also avoid packages conflicts.

Create a virtualenv venv under folder /FinRL-Library

virtualenv -p python3 venv

To activate a virtualenv:

source venv/bin/activate

To activate a virtualenv on windows:

venv\Scripts\activate

Dependencies

The script has been tested running under Python >= 3.6.0, with the folowing packages installed:

pip install -r requirements.txt

Stable-Baselines3 using Pytorch

Stable-Baselines3 is a set of improved implementations of reinforcement learning algorithms in PyTorch. It is the next major version of Stable Baselines. If you have questions regarding Stable-baselines package, please refer to Stable-baselines3 installation guide. Install the Stable Baselines package using pip:

pip install stable-baselines3[extra]

A migration guide from SB2 to SB3 can be found in the documentation.

Stable-Baselines using Tensorflow 2.0

Still Under Development

Run

python main.py --mode=train

Backtesting

Use Quantopian's pyfolio package to do the backtesting.

Data

The stock data we use is pulled from Yahoo Finance API

(The following time line is used in the paper; users can update to new time windows.)

Contributions

  • FinRL is an open source library specifically designed and implemented for quant finance. Trading environments incorporating market frictions are used and provided.
  • Trading tasks accompanied by hands-on tutorials with built-in DRL agents are available in a beginner-friendly and reproducible fashion using Jupyter notebook. Customization of trading time steps is feasible.
  • FinRL has good scalability, with a broad range of fine-tuned state-of-the-art DRL algorithms. Adjusting the implementations to the rapid changing stock market is well supported.
  • Typical use cases are selected and used to establish a benchmark for the quantitative finance community. Standard backtesting and evaluation metrics are also provided for easy and effective performance evaluation.

Citing FinRL

@article{finrl2020,
    author  = {Liu, Xiao-Yang and Yang, Hongyang and Chen, Qian and Zhang, Runjia and Yang, Liuqing and Xiao, Bowen and Wang, Christina Dan},
    journal = {Deep RL Workshop, NeurIPS 2020},
    title   = {FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance},
    url     = {},
    year    = {2020}
}

Call for Contributions

We will maintain the open source FinRL library for the "AI + finance" community and welcome you to join as contributors!

Contributors

This project exists thanks to all the people who contribute.

Support various markets

We would like to support more asset markets, so that the users can test their stategies.

SOTA DRL algorithms

We will continue to maintian a pool of DRL algorithms that can be treated as SOTA implementations.

Benchmarks for typical trading tasks

To help quants have better evaluations, here we maintain benchmarks for many trading tasks, upon which you can improve for your own tasks.

Support live trading

Supporting live trading can close the simulation-reality gap, it will enable quant to switch to the real market when they are confident with their strategies.

LICENSE

MIT