Example Algorithmic Trading bot - broken - crashes the server
123mitnik opened this issue · 0 comments
123mitnik commented
when running the Example Algorithmic Trading bot
and the server is started with:
RUST_BACKTRACE=full ./tdb-server -vv -a -f "/tectonicdb/test/test-data" -i 10000
from tectonic import TectonicDB
import json
import asyncio
async def subscribe(name):
db = TectonicDB(host="localhost", port=9001)
_success, _text = await db.subscribe(name)
while 1:
_, item = await db.poll()
if b"NONE" == item:
await asyncio.sleep(0.01)
else:
yield json.loads(item)
class TickBatcher(object):
def __init__(self, db_name):
self.one_batch = []
self.db_name = db_name
async def batch(self):
async for item in subscribe(self.db_name):
self.one_batch.append(item)
async def timer(self):
while 1:
await asyncio.sleep(1) # do work every n seconds
print(len(self.one_batch)) # do work here
self.one_batch = [] # clear queue
if __name__ == '__main__':
loop = asyncio.get_event_loop()
proc = TickBatcher("bt_btceth")
loop.create_task(proc.batch())
loop.create_task(proc.timer())
loop.run_forever()
loop.close()
where the bt_btceth - is the file located inside the test-data directory
when running the code it crashes the server with:
[2021-01-08][21:58:28:763015000][tdb_server_core::server][INFO] Accepting from: 127.0.0.1:54466
thread 'async-std/executor' panicked at 'range end index 1398096467 out of range for slice of length 1048576', /Users/megalodon/hook/tectonicdb/crates/tdb-server-core/src/server.rs:97:32
stack backtrace:
0: 0x100fb9bb4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha0848bb2602b5d05
1: 0x100fd91e0 - core::fmt::write::h9f3ccac2ef682b93
2: 0x100fb35e6 - std::io::Write::write_fmt::h0a47673aab280496
3: 0x100fbb9b9 - std::panicking::default_hook::{{closure}}::h850c6aaf5e80c2f5
4: 0x100fbb67d - std::panicking::default_hook::h037801299da6e1c6
5: 0x100fbc03b - std::panicking::rust_panic_with_hook::h76436d4cf7a368ac
6: 0x100fbbb65 - std::panicking::begin_panic_handler::{{closure}}::h516c76d70abf04f6
7: 0x100fba028 - std::sys_common::backtrace::__rust_end_short_backtrace::h653290b4f930faed
8: 0x100fbbaca - _rust_begin_unwind
9: 0x100fe790f - core::panicking::panic_fmt::hde9134dd19c9a74f
10: 0x100fe79e6 - core::slice::index::slice_end_index_len_fail::h1abfffb7603f7340
11: 0x100e2b7a5 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::ha8ee21bb24b9c334
12: 0x100e1b2db - async_task::raw::RawTask<F,R,S,T>::run::h3048d31c3e65fa36
13: 0x100f19d9b - std::thread::local::LocalKey<T>::with::h22726983dfed64e7
14: 0x100f1e039 - std::sys_common::backtrace::__rust_begin_short_backtrace::h751ab56d9cd682d2
15: 0x100f1969d - core::ops::function::FnOnce::call_once{{vtable.shim}}::hef7b80dd4fa7d5bb
16: 0x100fbf4cd - std::sys::unix::thread::Thread::new::thread_start::hedb7cc0d930a8f40
17: 0x7fff2036a950 - __pthread_start
[1] 21857 abort RUST_BACKTRACE=full ./tdb-server -vv -a -f -i 10000
rustc 1.49.0 (e1884a8e3 2020-12-29)
the order book seems ok.. running the ffi.py with big limit, gives:
ts seq is_trade is_bid price size
0 1509862963012 7050 True True 0.040190 81.000000
1 1509862963012 7050 True True 0.040190 28.550123
2 1509862963012 7050 True True 0.036790 0.000000
3 1509862963012 7050 True True 0.032791 0.000000
4 1509862963012 7050 True True 0.040313 0.990000
... ... ... ... ... ... ...
6104686 1510340961171 16144 True True 0.052534 0.000000
6104687 1510340961171 16144 True True 0.060045 0.045147
6104688 1510340961171 16144 True True 0.060048 0.350615
6104689 1510340961171 16144 True True 0.060054 0.382365
6104690 1510340961171 16144 True True 0.060059 0.067691
[6104691 rows x 6 columns]
53.56917119026184