duality-labs/hapi-indexer

Add some form of DB snapshot saving for quicker syncing

Opened this issue · 0 comments

dib542 commented

When an indexer service is updated, sometimes this does not necessitate re-indexing the chain data from height 0 (for example: minor npm dependencies should be updated) it would be great if the indexer could start from a known snapshotted position.

This could be:

  • saving the DB file to a cloud file storage service and being able to resume from that

There is a challenge in how to identify what is a "breaking" upgrade (requiring a full indexing) and "non-breaking" (ok to use DB snapshots). And there is also the issue of ensuring a snapshot file isn't used as a base or the wrong or a new chain. So it's likely that any DB file snapshot must be properly identified with an ID representing

  • current chain ID
  • current non-breaking indexer version (and this currently conflicts with our strategy to keep the indexer version the same as the Chain release version number)