/awesome-benchmark

Creative Commons Zero v1.0 UniversalCC0-1.0

Awesome system benchmark tools

CPU

  • sysbench (cpu) : Scriptable multi-threaded benchmark tool based on LuaJIT.
  • openssl speed: Cryptography performance measurement
  • V-Ray Benchmark: Free standalone application to test how fast your system renders.
  • Phoronix Test Suite: Testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems.
  • SPEC CPU: SPEC's next-generation, industry-standardized, CPU intensive suites for measuring and comparing compute intensive performance, stressing a system's processor, memory subsystem and compiler.
  • VASP: The Vienna Ab initio Simulation Package is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.
  • CoreMark: Methods for testing only a processor's core features
  • CoreMark®-PRO: Comprehensive, advanced processor benchmark that works with and enhances the market-proven industry-standard EEMBC CoreMark®
  • Geekbench : Cross-platform benchmark that measures your system's performance with the press of a button.
  • db-benchmark: Reproducible benchmarking of database-like operations in single-node environment.
  • Renaissance Benchmark Suite: Aggregates common modern JVM workloads, including, but not limited to, Big Data, machine-learning, and functional programming.
  • The Computer Language Benchmarks Game: Toy-program performance measurements for ~24 language implementations.
  • phpbenchmarks: Test suite for vanilla PHP, Laravel or Symfony
  • Compute Benchmarks: A benchmark suite developed and maintained by NEO driver development team in order to provide a stable environment, which allows checking and reproducing performance for various parts of the driver
  • Large Text Compression Benchmark: Methodologies and set of results from Wikipedia's text compression through various tool
  • benchdnn: Extended and robust correctness verification and performance benchmarking tool for the primitives provided by oneDNN

RAM

Storage

  • Flexible I/O Tester (FIO): Spawns a number of threads or processes doing a particular type of I/O action as specified by the user.
  • vdbench: Disk and tape I/O workload generator for verifying data integrity and measuring performance of storage.
  • iozone: Filesystem benchmark tool
  • hdparam: Command line interface to various kernel interfaces.
  • oiping Shows disk latency in the same way as ping shows network latency.

Network

  • iperf: Active measurements of the maximum achievable bandwidth on IP networks.
  • tracepath: Trace path to destination discovering MTU along this path.
  • traceroute: racks the route packets taken from an IP network on their way to a given host.
  • tcptraceroute: traceroute implementation using TCP packets.
  • paris-traceroute: Paris traceroute is a new version of the well-known network diagnosis and measurement tool. It addresses problems caused by load balancers with the initial implementation of traceroute.
  • dublin-traceroute: Dublin Traceroute is a NAT-aware multipath traceroute tool.
  • mtr: Combines the functionality of the 'traceroute' and 'ping' programs in a single network diagnostic tool.

GPU

HPC

  • OSU Micro-Benchmarks (OMB): Suite of benchmarks for measuring and evaluating the performance of MPI operations for point-to-point, multi-pair, and collective communications.
  • mpi-benchmarks: Intel(R) MPI Benchmarks provides a set of elementary benchmarks that conform to MPI-1, MPI-2, and MPI-3 standard.
  • FinanceBench: FinanceBench is aimed at those who work with financial code to see how certain code paths can be targeted for accelerators.
  • LAMMPS Benchmarks: Simple suite with Large-scale Atomic/Molecular Massively Parallel Simulator workloads
  • pyhpc-benchmark: Suite of benchmarks to test the sequential CPU and GPU performance of various computational backends with Python frontends

AI

HTTP

  • ab (apache benchmark): Load testing and benchmarking tool for HTTP servers.
  • cURL: Command line tool and library for transferring data with URLs.
  • wrk: Modern HTTP benchmarking tool capable of generating significant load when run on a single multi-core CPU.
  • os-benchmark : Simple tool to collect metrics about all object storage.
  • s3-benchmark: Measure S3's performance from any location
  • warp: S3 benchmarking tool
  • COSBench: Tool to measure the performance of Cloud Object Storage services.
  • Gosbench: Golang reimplementation of Cosbench. It is a distributed S3 performance benchmark tool with Prometheus exporter leveraging the official Golang AWS SDK

Database and messaging

  • redis-benchmark: Utility that simulates running commands done by N clients at the same time sending M total querie
  • memtier_benchmark: Command line utility developed for load generation and bechmarking NoSQL key-value databases.
  • rpc-perf: Tool for measuring the performance of RPC services and is primarily used to benchmark caching systems.
  • sysbench (oltp) : Scriptable multi-threaded benchmark tool based on LuaJIT.
  • sysbench TPCC: TPCC-like workload for sysbench 1.0.x.
  • YCSB: Framework and common set of workloads for evaluating the performance of different “key-value” and “cloud” serving stores
  • benchyou: benchyou is a benchmark tool for MySQL, similar to Sysbench.
  • Cassandra Stress: Benchmark and load-test a Cassandra cluster
  • HammerDB: Benchmarking and load testing software for the worlds most popular databases supporting Oracle Database, SQL Server, IBM Db2, MySQL, MariaDB and PostgreSQL.
  • mysqlslap: Emulate client load for a MySQL server and to report the timing of each stage.
  • pgbench: It runs the same sequence of SQL commands over and over and then calculates the average transaction rate.
  • OpenMessaging Benchmark Framework: Suite of tools that make it easy to benchmark distributed messaging systems in the cloud.
  • opensearch-benchmark: Macrobenchmarking framework for OpenSearch
  • Rally: Macrobenchmarking framework for Elasticsearch

Big data

  • HiBench: The bigdata micro benchmark suite.
  • YCSB: Framework and common set of workloads for evaluating the performance of different “key-value” and “cloud” serving stores.

Scheduler

Ecology and environment

  • Eco benchmark: Try to compare the energy consumption, for several languages.
  • Scaphandre: Metrology agent dedicated to electrical power consumption metrics.

Comment

Do not forget that any task/workload can be a benchmark, it always depends of your goal.