/optimal-stopping-cnn

Using CNN architeture of ANN to solve the optimal stopping problem of Bermundan options

Primary LanguagePythonMIT LicenseMIT

Solving the optimal stopping problem with reinforcement learning: an application in financial option exercise

Abstract:

The optimal stopping problem is a category of decision problems with a specific constrained configuration. It is relevant to various real-world applications such as finance and management. To solve the optimal stopping problem, state-of-the-art algorithms in dynamic programming, such as the least-squares Monte Carlo (LSMC), are employed. This type of algorithm relies on path simulations using only the last price of the underlying asset as a state representation. Also, the LSMC was thinking for option valuation where risk-neutral probabilities can be employed to account for uncertainty. However, the general optimal stopping problem goals may not fit the requirements of the LSMC showing auto-correlated prices. We employ a data-driven method that uses Monte Carlo simulation to train and test artificial neural networks (ANN) to solve the optimal stopping problem. Using ANN to solve decision problems is not entirely new. We propose a different architecture that uses convolutional neural networks (CNN) to deal with the dimensionality problem that arises when we transform the whole history of prices into a Markovian state. We present experiments that indicate that our proposed architecture improves results over the previous implementations under specific simulated time series function sets. Lastly, we employ our proposed method to compare the optimal exercise of the financial options problem with the LSMC algorithm. Our experiments show that our method can capture more accurate exercise opportunities when compared to the LSMC. We have outstandingly higher (above 974% improvement) expected payoff from these exercise policies under the many Monte Carlo simulations that used the real-world return database on the out-of-sample (test) data.

Requirements

Usage

First, install prerequisites

$ pip install fbm
$ pip install timesynth	
$ pip3 install torch==1.8.1

Check pytorch address for the compatible version

To train and test the model use main.py calling the necessaru libraries that train and test the specific RL models. To use the LSMC implementation check the LSMC.py.

To use real data, add the file to the Dataset file and replace the file variable in main and replace change the type of data variable:

file = r'\SP500- daily - 30Y_train.csv'
type_of_type_of_data = real_val # or choose the type of sythetic data to use

All the data was extracted from Yahoo Finance All synthetic data is generated by the code in utils.py

Plotting Results

To reproduce the figures presented in the paper, it is possible to use the plotting code.

References

If you re-use this work, please cite (will be updated soon):

@INPROCEEDINGS{Felizardo2022,
  author={Leonardo Kanashiro Felizardo and Elia Yathie Matsumoto and Emilio Del-Moral-Hernandez},
  booktitle={2022 International Joint Conference on Neural Networks (IJCNN)}, 
  title={Solving the optimal stopping problem with reinforcement learning: an application in financial option exercise}, 
  year={2022},
  volume={},
  number={},
  pages={1-8},
  doi={}}

The main references for this paper is the work of Becker