mikeal/IPSQL

chore: refactor to allow lazy loading of all nodes

mikeal opened this issue · 0 comments

Right now, there’s an optimization intended to speed up the loading of a database.

When you used IPSQL.from() it doesn’t return until the heads of all the data structures for every table and column are loaded. This is intended to speed up subsequent performance by pre-loading all the block data.

The problem is that this also means you MUST have all these heads in order to load a database, which means that query exports have to include all that data even though it isn’t actually necessary to satisfy their queries.

It’s an annoying refactor, but all the loading of tables and columns needs to be done lazily as each node is accessed (and then cached). We can add a preload() method that loads all the head nodes into memory so that people can still get the pre-load performance if they wish.