stellar/go

services/horizon: Benchmark reingestion based on precomputed ledger meta tx files

Opened this issue · 1 comments

sreuland commented

What problem does your feature solve?

  • absence of performance benchmark numbers for time it takes to reingest at least 1 year of latest pubnet data.
  • public docs haven't been updated for new reingestion from datastore capability.

What would you like to see?

  • A dedicated runtime environment with at least provides the horizon required compute resources

  • Run db reingest range --parallel-workers <?> --ledgerbackend datastore --datastore-config config.toml <1 year ago> <current>

  • Monitor the performance of cpu/memory, runtime duration.

  • Try different performance config parameters:

    • parallel workers: likely 5 or less.
    • buffered storage config per results from #5390

  • Update the public docs for reingestion to state this new --ledgerbackend datastore option.

What alternatives are there?

sreuland commented

@urvisavla , should we move this ticket up to sprint 49 backlog?

  • 👍1