Build Benchmark

A not that impressive tool to benchmark compilation times of different devices. Yes, this benchmarks the devices, not the build. Though I guess you could use it to benchmark the build as well...

Usage

build-benchmark [OPTIONS] <path to solution file>

OPTIONS:
  -n|--number INT:      Number of compilations for first and second pass
  -c|--cooldown INT:    Cooldown in seconds between second and third pass
  -d|--duration INT:    Duration in seconds of third pass
  --property KEY=VALUE: MSBuild properties to use; can be specified multiple times

Requirements

You'll need the dotnet SDK installed. After all, we're building a solution here. Make sure that you've got the SDK that the solution you're building requires.

The tool looks at the DOTNET_HOST_PATH, DOTNET_ROOT and PATH environment variables to find dotnet. If it doesn't find dotnet, help the tool out by passing DOTNET_ROOT when invoking the tool.

Compile from scratch

To compile from scratch, use dotnet publish -c Release -o publish/ src/BuildBenchmark.Cli from the root of the repository. That's it! You'll find your single binary in publish, ready to be executed.

That's it, really.

Use the prebuilt binaries

You'll find prebuilt binaries for the most important platforms (win-x64, linux-x64, linux-arm64 and osx-arm64) in the releases.

For Linux and MacOS, you'll want to chmod +x $file the file before executing it. For MacOS, you'll want to remove it from quarantine using xattr -d com.apple.quarantine $file. The tool should be signed, but I don't know if that signing can expire - so if it doesn't work, try running codesign --force --deep -s - $file as well.

How it works

The benchmark is split in three distinct passes:

Continuous full-building
Continuous incremental building
A longer period of continuous incremental building

A complete restore and full-build is done prior to these passes to ensure that the disk cache is "warm". Between the second and third pass, there is a cooldown to ensure we're not starting the "stresstest" with high CPU temperatures.

The first pass is meant to capture raw compilation speed. The second pass is meant to show the speed of the inner loop, as most real world compilations are incremental. The third pass is meant to show how much performance is lost due to thermal throttling - most devices should warm up considerably during this stress test.