ScrapeWithYuri/pairstrade-fyp-2019

A Comparative Study of Three Pairs Trading Approaches - Distance, Cointegration, and Reinforcement Learning.

Pairs Trading: Testing 3 Approaches - Distance Method, Cointegration Method, and Reinforcement Learning Agent (RL)

Introduction

In this project, I discuss my experience in implementing the reinforcement learning (RL) approach for pairs trading based on work from pairstrade-fyp-2019. The project provides helpful instructions on running the RL_train code using specific parameters. I also share the datasets used by the authors and the processed raw data results. However, I observed differences in log prices and believe the authors' dataset may have had look ahead bias. I achieved marginally positive results with the authors' processed raw data but was not able to achieve 40%+ results using the authors' dataset. I adjusted the incur_commission function and propose another modification to include shorted value in the compute reward function. This project explores the three main approaches for performing pairs trading: distance method, cointegration method (rolling OLS, Kalman Filter), and the RL agent approach.

Additional Information

The authors added helpful instructions how to run the code here. I ran the rl_train code by using these parameters:

--job_name train_0_1_2_test_3 --run_mode "train" --train_indices 0 1 2 --test_indices 3

I added datasets used by the authors here. However, when I proccessed the raw data into the transformed data via the process_data.py script, the resulting log prices were different than the data used by the authors. After training and testing the reinforcement code with my processed raw data, my results were marginally positive. I believe the authors' dataset may have had look ahead bias when they originally transformed the data. My results are here (slightly positive but not large enough to warrant trading), while the results I was able to achieve using the authors' dataset is located here (40%+ as mentioned in their paper).

I made one adjustment to the code, and believe another is warranted.

1. The incur_commission function added 10%+ cost over the course of a year. Since the strategy generally executed few trades, I expected this to be lower. Based on my knowledge, this should be 0.25% - 0.50%, so I divided the cost by 50.

2. I believe the compute reward function should include the shorted value to self.port_val_minus_com[i] when the shorted value goes below zero by adding spv_nex. In a real-world environment, shorted values can go "below zero" (i.e. you owe money to cover the short). I did not make this change to the code though.

Documents

The authors' final report can be found here. The authors' presentation slides can be found here.