bitauth/chaingraph

Scalenet Support

Opened this issue · 0 comments

To implement and test support for very large blocks, Chaingraph should add default support for scalenet, which currently uses a 256 MB maximum block size. (It should be as easy to add a scalenet node as it is to add a testnet4 node.)

In my earlier testing with very large blocks, Chaingraph's block-saving queries were hitting Postgres' 2MB max_stack_depth. If that's still an issue, we should change the block-saving strategy to accommodate very large blocks without tuning Postgres. This should probably be implemented at the same time as #12, which is closely related.

Additionally, we should consider saving incoming transactions in batches rather than one transaction at a time – at a rate of hundreds or thousands of transactions per second, Chaingraph instances with high DB latency and a very limited connection pool might not keep up.

As part of this work, we should also implement a "stress test" benchmark which measures Chaingraph's performance with 256MB blocks, both for blocks with most transactions heard first over P2P and blocks full of previously-unheard transactions. (Would be good to also have benchmarks for ~1MB blocks and ~32MB blocks.) All benchmarks should run in CI.