benchttp/cli is a command-line interface that runs benchmarks on HTTP endpoints.
Highly configurable, it can be used as a development tool at design time
as well as a CI step thanks to the testing suite.
- Visit https://github.com/benchttp/cli/releases and download the asset
benchttp_<os>_<architecture>matching your OS and CPU architecture. - Rename the downloaded asset to
benchttp, add it to yourPATH, and refresh your terminal if necessary - Run
benchttp versionto check it works properly.
benchttp run [options]In this section we dive into the many configuration options provided by the runner.
By default, the runner uses a default configuration that is valid for use without further tuning, except for url that must always be set.
You can override the default configuration by providing a configuration file (YAML or JSON) with the --configFile flag, or by passing flags to the run command (see below for the list of flags), or a mix of both.
The runner uses a default configuration that can be overridden by a configuration file and/or flags. To determine the final configuration of a benchmark and which options take predecence over the others, the runner follows this flow:
-
It starts with a default configuration
-
Then it tries to find a config file and overrides the defaults with the values set in it
- If flag
-configFileis set, it resolves its value as a path - Else, it tries to find a config file in the working directory, by priority order:
.benchttp.yml>.benchttp.yaml>.benchttp.json
The config file is optional: if none is found, this step is ignored. If a config file has an option
extends, it resolves config file recursively until the root is reached and overrides the values from parent to child. - If flag
-
Then it overrides the current config values with any value set via the CLI
-
Finally, it performs a validation on the resulting config (not before!). This allows composed configurations for better granularity.
With rare exceptions, any option can be set either via CLI flags or config file, and option names always match.
📄 A full config file example is available here (minus the testing suite, see below).
| CLI flag | File option | Description | Usage example |
|---|---|---|---|
-url |
request.url |
Target URL (Required) | -url http://localhost:8080/users?page=3 |
-method |
request.method |
HTTP Method | -method POST |
| - | request.queryParams |
Added query params to URL | - |
-header |
request.header |
Request headers | -header 'key0:val0' -header 'key1:val1' |
-body |
request.body |
Raw request body | -body 'raw:{"id":"abc"}' |
| CLI flag | File option | Description | Usage example |
|---|---|---|---|
-requests |
runner.requests |
Number of requests to run (-1 means infinite, stop on globalTimeout) | -requests 100 |
-concurrency |
runner.concurrency |
Maximum concurrent requests | -concurrency 10 |
-interval |
runner.interval |
Minimum duration between two non-concurrent requests | -interval 200ms |
-requestTimeout |
runner.requestTimeout |
Timeout for every single request | -requestTimeout 5s |
-globalTimeout |
runner.globalTimeout |
Timeout for the whole benchmark | -globalTimeout 30s |
Note: the expected format for durations is <int><unit>, with unit being any of ns, µs, ms, s, m, h.
| CLI flag | Description | Usage example |
|---|---|---|
-silent |
Remove convenience prints | -silent / -silent=false |
-configFile |
Path to benchttp config file | -configFile=path/to/benchttp.yml |
One of the nicest features is the ability to run a test suite on an endpoint's performances from the CLI using the regular command.
Once the test suite done, it exits the process with code 0 if successful or 1 if any test failed, which makes benchttp usable in a CI context, making sure your changes do not introduce perfomance regressions for instance:
For that matter, the test suite must be declared in a benchttp configuration file (there is currently no way to set these via cli options).
📄 Please refer to our Wiki for a fully detailed configuration including a test suite.