This is a home for the common benchmarking infrastructure described in the accompanying RFC. It aims to be a common benchmark suite that is compiler-agnostic and can be used in standalone comparative benchmark workflows and regression benchmarking resident in each compiler project.
There are two components in this repository:
- common_benchmark_suite: The compiler-agnostic benchmark suite.
- comparative_benchmark: Tools to run comparative benchmarks with the benchmark suite.
The common_benchmark_suite is standalone and should not have dependency on the comparative_benchmark.
This repository is still under early development.
To add new models and benchmarks, see Onboarding New Models and Benchmarks.
TODO(pzread): Add instructions to run comparative benchmarks locally.
TODO(pzread): Add a link to the comparative benchmark dashboard.
- GitHub issues: Feature requests, bugs, and other work tracking
- OpenXLA discord: Daily development discussions with the core team and collaborators
OpenXLA Benchmark is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.