Market Reporter automatically generates short comments that describe time series data of stock prices, FX rates, etc. This is an implementation of Murakami et al. (ACL 2017) [bib] [paper] and Aoki et al. (INLG 2018) [bib] [paper] [poster].
The architecture is illustrated below.
- Tick data
We purchased tick data from Thomson Reuters DatScope Select and downloaded them by using the REST API it provides. - Text data
We purchased news articles provided by Nikkei Quick News.
This tool stores data to Amazon S3.
Ask the manager to give you AmazonS3FullAccess
and issue a credential file.
For details, please read AWS Identity and Access Management.
When you have a credentials file for AWS (default: ~/.aws/credetials
), please edit config.toml
to set profile_name
in the s3
section.
Otherwise, you need to type AWS access key ID and AWS secret access key when you launch the image.
cd envs
docker build -t market-reporter .
docker run -d \
--name demo \
--user root \
-e AWS_ACCESS_KEY_ID=your_access_key_id \
-e AWS_SECRET_ACCESS_KEY=your_secret_access_key \
market-reporter
docker exec -it --user reporter demo /bin/bash
We recommend Anaconda.
The code never runs on Python 2.
After you install Anaconda, create a new environment from environment.yaml
.
conda env create -f environment.yaml -n NAME
source activate NAME
Suppose you have a database named master
on your local machine.
Then, edit config.toml
as the following.
[postgres]
- uri = 'postgresql://USERNAME:PASSWORD@SERVER:PORT/DATABASE'
+ uri = 'postgresql:///master'
While you are connecting to a server by SSH port forwarding by a command such as ssh -fNT -L 2345:localhost:5432 kirito@dbserver
, edit config.toml
as the following.
- uri = 'postgresql://USERNAME:PASSWORD@SERVER:PORT/DATABASE'
+ uri = 'postgresql://kirito:PASSWORD@localhost:2345/master'
Create config.toml
based on example.toml or murakami-et-al-2017.example.toml.
Execute the following command for the training of model. When you use GPU (CPU), you specify cuda:n
(cpu
) for --device
option, where n
is the device index to use.
python -m reporter --device 'cuda:0'
After the program finishes, it saves three files (reporter.log
, reporter.model
, and reporter.vocab
) to config.output_dir/reporter-DATETIME
, where config.output_dir
is a variable set in config.toml
and DATETIME
is the timestamp of the starting time.
Prediction submodule generates a single comment of a financial instrument at specified time by loading a trained model.
# -r, --ric: Reuters Instrument Code (e.g. '.N225' for Nikkei Stock Average)
# -t, --time: timestamp in '%Y-%m-%d %H:%M:%S%z' format
# -o, --output: directory that contains 'reporter.model' and 'reporter.vocab'
python -m reporter.predict \
-r '.N225' \
-t '2018-10-03 09:03:00+0900' \
-o output/reporter-2018-10-07-18-47-41
Execute the following command and access http://localhost:5000/
in a web browser.
make # for the first time
python -m reporter.webapp
When you launch it on a server, execute the following command instead.
nohup uwsgi --ini uwsgi.ini &
You can see a page as the following picture.
python setup.py test
Market Reporter is available under different licensing options:
- GNU General Public License (v3 or later).
- Commercial licenses.
Commercial licenses are appropriate for development of proprietary/commercial software where you do not want to share any source code with third parties or otherwise cannot comply with the terms of the GNU. For details, please contact us at kirt-contact-ml@aist.go.jp
This software uses a technique applied for patent (patent application number 2017001583).
When you write a paper using this software, please cite either or both of the followings.
@InProceedings{murakami2017,
author = {Murakami, Soichiro
and Watanabe, Akihiko
and Miyazawa, Akira
and Goshima, Keiichi
and Yanase, Toshihiko
and Takamura, Hiroya
and Miyao, Yusuke},
title = {Learning to Generate Market Comments from Stock Prices},
booktitle = {Proceedings of the 55th Annual Meeting of
the Association for Computational Linguistics (Volume 1: Long Papers)},
year = {2017},
publisher = {Association for Computational Linguistics},
pages = {1374--1384},
location = {Vancouver, Canada},
doi = {10.18653/v1/P17-1126},
url = {http://www.aclweb.org/anthology/P17-1126}
}
@InProceedings{aoki2018,
author = {Aoki, Tatsuya
and Miyazawa, Akira
and Ishigaki, Tatsuya
and Goshima, Keiichi
and Aoki, Kasumi
and Kobayashi, Ichiro
and Takamura, Hiroya
and Miyao, Yusuke},
}
title = {Generating Market Comments Referring to External Resources},
booktitle = {Proceedings of the 11th International Conference on Natural Language Generation},
year = {2018},
publisher = {Association for Computational Linguistics},
pages = {135--139},
location = {Tilburg University, The Netherlands},
url = {http://aclweb.org/anthology/W18-6515}
}