Streaming Data Saver

Working with Tinkoff Invest OpenAPI. Dumps realtime streams of orderbooks and candles to JSON file for ML purposes.

Releases

statically-linked binary for: linux/amd64, linux/arm64, linux/armv6, linux/armv7
docker image with amd64, arm64, armv6 and armv7 manifests: a00s/tinkoff-invest-dumper

Options

tinkoff-invest-dumper --help
  --candle string
        list of tickers to subscribe for candles
  --candle-interval string
        interval of candles: 1min,2min,3min,5min,10min,15min,30min,hour,2hour,4hour,day,week,month (default "1min")
  --orderbook string
        list of tickers to subscribe for orderbooks
  --orderbook-depth int
        depth of orderbook: from 1 to 20 (default 20)
  --path string
        path to storage dir (default ".")
  --time-suffix-enabled
        add the time suffix to every filename on (re)start
  --time-suffix-format string
        go format of the time suffix (see https://golang.org/src/time/format.go) (default "2006010215")
  --token string
        your sandbox's token
  --version
        show version info

File rotation

You can split and rotate JSON-files using --time-suffix-enabled. For e.g, if you start tinkoff-invest-dumper as systemd service, you may use cron to restart this service at a certain time during an inactive part of a trading day. During restarting, tinkoff-invest-dumper closes previous files and opens a new one with a new-day time suffix. The inactive period for Tinkoff Invest is 2:00-10:30 MSK.

Run as Docker image

docker run \
  --rm -it \
  --env TOKEN=$TINKOFF_SANDBOX \
  --volume `pwd`/data:/data \
  a00s/tinkoff-invest-dumper \
    --token "$TOKEN" \
    --path /data \
    --time-suffix-enabled \
    --candle NVDA,MSFT,TSLA \
    --orderbook NVDA,MSFT,TSLA

Example

tinkoff-invest-dumper \
    --token "$TINKOFF_SANDBOX" \
    --candle NVDA,MSFT,TSLA \
    --orderbook NVDA,MSFT,TSLA \
    --orderbook-depth 2

2020/08/24 12:49:15 Subscribed to orderbook NVDA BBG000BBJQV0
2020/08/24 12:49:15 Subscribed to orderbook MSFT BBG000BPH459
2020/08/24 12:49:15 Subscribed to orderbook TSLA BBG000N9MNX3
2020/08/24 12:49:15 Subscribed to candles NVDA BBG000BBJQV0
2020/08/24 12:49:15 Subscribed to candles MSFT BBG000BPH459
2020/08/24 12:49:15 Subscribed to candles TSLA BBG000N9MNX3

tail -f NVDA-obk

{"a":[[514.31,75],[514.71,6]],"b":[[514.3,6],[514.25,10]],"figi":"BBG000BBJQV0","lt":"2020-08-24T12:49:24.866749+03:00","t":"2020-08-24T09:49:24.850272182Z","ticker":"NVDA"}
{"a":[[514.31,75],[514.71,6]],"b":[[514.3,6],[514.25,10]],"figi":"BBG000BBJQV0","lt":"2020-08-24T12:49:25.225449+03:00","t":"2020-08-24T09:49:25.26326835Z","ticker":"NVDA"}
{"a":[[514.31,75],[514.71,6]],"b":[[514.3,6],[514.25,10]],"figi":"BBG000BBJQV0","lt":"2020-08-24T12:49:25.480208+03:00","t":"2020-08-24T09:49:25.50689026Z","ticker":"NVDA"}

tail -f NVDA-cdl

{"c":514.48,"figi":"BBG000BBJQV0","h":514.71,"i":"1min","l":514.48,"lt":"2020-08-24T12:49:15.203217+03:00","o":514.5,"t":"2020-08-24T09:49:15.241791397Z","ticker":"NVDA","ts":"2020-08-24T09:49:00Z","v":11}
{"c":514.48,"figi":"BBG000BBJQV0","h":514.71,"i":"1min","l":514.48,"lt":"2020-08-24T12:49:19.747036+03:00","o":514.5,"t":"2020-08-24T09:49:19.786563182Z","ticker":"NVDA","ts":"2020-08-24T09:49:00Z","v":13}

koi8-r/tinkoff-invest-dumper