PyStrategies: Deep Learning Framework for HFT Trading Strategy Development

PyStrategies is a collection of Python tools for implementing, testing, and optimizing algorithmic trading strategies. This framework is built to be used with depth-of-book US equities data. Tools are included for deep learning, parameter optimization, and high-fidelity single-stock backtesting.

This project relies on another one of my libraries: PyLimitBook

The bottom of this README includes important setup instructions.

Toolset

Writing Strategies

Strategies are implemented by extending the BaseStrategy base class and adding logic for event callbacks. Refer to the Sample Strategy for an (unprofitable) example.

Backtesting a Strategy

A single-stock backtester is included that accurately simulates the limit order book.

To run a backtest:

    ./bin/backtest.sh quotes/XOM_BATS_2010-06-23.csv strategy/pyStrategy.py

Graphing Results

A sample Jupyter notebook is included to analyze signals generated from your strategy.

To generate data and interactively graph it:

    # Generate book snapshots - make sure CREATE_ON_NBBO_CHANGE_ONLY variable in the script is set appropriately
    python PyLimitBook/create_graphing_data.py ./quotes/XOM_BATS_2010-06-23.csv ./analyze/data/book.csv

    # Move signals generated by backtest
    mv ./strategy/signals_log.csv ./analyze/data/signals.csv

    jupyter notebook --notebook-dir=./analyze

Strategy Parameter Optimization

Optunity is used for multi-threaded parameter optimization. Parameter ranges must be defined in the parameters.csv file.

To run the parameter optimizer for 1000 iterations:

    # Specify parameter ranges in optimize/parameters.csv
    python optimize/optimize.py quotes/XOM_BATS_2010-06-23.csv strategy/pyStrategy.py 1000

Machine Learning

Wrapper scripts are included to generate features and do deep learning using Theano and Keras:

    # Generate features
    python machine_learning/generate_features.py quotes/XOM_BATS_2010-06-23.csv

    # Generate labels to predict
    python machine_learning/generate_labels.py quotes/XOM_BATS_2010-06-23.csv

    # Train the model
    python machine_learning/train_nn.py quotes/XOM_BATS_2010-06-23.csv

    # Test the model
    python machine_learning/test_nn.py quotes/XOM_BATS_2010-06-24.csv

Setup Instructions

Install Python libraries

Setup virtualenv: virtualenv --no-site-packages venv
Activate virtualenv: source venv/bin/activate
Install libraries: pip install -r requirements.txt

Install jpy for bridging Python and Java code

git clone https://github.com/bcdev/jpy.git
cd jpy
export JDK_HOME=`/usr/libexec/java_home`
export JAVA_HOME=$JDK_HOME
python setup.py --maven build
Copy lib/jpy-0.8.jar to the Java project's lib directory
Copy properties file from build/lib.*/jpyconfig.properties into the Java project's strategy directory

Install CuDNN for Speeding up Deep Learning

If your GPU supports CUDA, the machine learning with be much, much faster if you install CuDNN from NVIDIA.

Download CuDNN: https://developer.nvidia.com/rdp/cudnn-download

The following configuration works on Mac OS X:
- nvcc version: release 7.5, V7.5.26
- cuda driver: 7.5.27
- clang version: clang-703.0.29 (XCODE 7.3.0)
- osx version: 10.11.4

Install libraries:

cd to where you want the libraries to live
Linux: export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH
Mac: export DYLD_LIBRARY_PATH=`pwd`:$DYLD_LIBRARY_PATH
- Copied *.h files to CUDA_ROOT/include and .so files to CUDA_ROOT/lib64
  - By default, CUDA_ROOT is /usr/local/cuda on Linux and /Developer/NVIDIA/CUDA-* on mac
- Add the following to the end of ~/.profile:
```
export PATH="/Developer/NVIDIA/CUDA-7.5/bin:$PATH"
export DYLD_LIBRARY_PATH="/Developer/NVIDIA/CUDA-7.5/lib:$DYLD_LIBRARY_PATH"
```
Add the install path to your build and link process by adding -Iinstallpath to your compile line and -Linstallpath -lcudnn to your link line.

Create a ~/.theanorc file: (adjust cnmem as necessary)

 [global]
 device=gpu
 floatX=float32
 allow_gc=False
 warn_float64=warn

 [lib]
 cnmem=0.50

 [nvcc]
 fastmath=True

danielktaylor/PyStrategies