go-faster/oteldb

benchmark: single-node load generator

Closed this issue · 1 comments

To benchmark storage, we need to generate realistic load for all observability signals.
Let's start with benchmark that runs locally on single node.

All components should start with os.Exec. Dependencies on database binaries (i.e. ytsaurus, clickhouse) is acceptable, but ideally should be downloaded automatically. Dependency on docker is acceptable on first stage, but probably should be avoided.

The test input should be benchmark configuration that describes a pattern of simulated load and benchmark time (if applicable, we can use "number of requests" mode instead).

The test output should be machine readable, probably with additional human readable variant.

  • Model production services
    • Multiple nodes (1x, 10x, 100x, 10_000x, 100_000x, 1_000_000x)
    • Multiple interconnected services (user -> frontend -> service A -> service B -> ...)
  • Implement configurable service that generates load via OpenTelemetry protocol
  • Select and track basic performance metrics of system, including those for supported
  • ...

Assume that test is run on bare metal with enough resources.

Stages of benchmark integration:

  1. Implementation
  2. Manual runs on PR branch
  3. Automated, manual-triggered runs
  4. CI integration, GitHub runners
  5. CI integration, external ephemeral runners
ernado commented

Done in ch-bench, good enough for now, focusing on metrics performance.