The general outline of the benchmark:
- The first 1000 lines of
data.txtare read and prepared during setup - For each benchmark iteration, parse each line in a loop and discard the result
data.txt consists of roughly 200 thousand messages. The benchmarks only read the first 1000 lines from this file.
Rust benchmarks use criterion.
$ cd rust && cargo bench
twitch-rs also has an optional SIMD implementation using SSE2 (only available on x86). It may be enabled by specifying the -F simd flag on cargo commands:
$ cd rust && cargo bench -F simd
twitch-rs also has support for a tag whitelist. Any tag which is not whitelisted will be ignored.
This can improve performance by up to 40%. In the results below, the version of the benchmark using
this feature is labelled cheating, which feels appropriate, because while it is fully parsing the tags
even if they're ignored, it isn't actually storing them in the parsed message.
C# benchmarks use BenchmarkDotNet.
$ cd dotnet && DOTNET_TieredPGO=1 dotnet run -c Release
Go benchmarks use the built-in benchmarking tool.
$ cd go && go test -bench=.
Benchmarks were run in WSL2 Ubuntu 22.04 on an AMD Ryzen 7950X
| library | language | time to parse 1000 lines |
|---|---|---|
| twitch | rustc 1.72.0-nightly (101fa903b 2023-06-04) | 593.24 µs ± 0.91 µs |
twitch (-F simd) |
rustc 1.72.0-nightly (101fa903b 2023-06-04) | 357.75 µs ± 1.07 µs |
twitch (-F simd + cheating) |
rustc 1.72.0-nightly (101fa903b 2023-06-04) | 195.51 µs ± 0.14 µs |
| twitch-irc | rustc 1.72.0-nightly (101fa903b 2023-06-04) | 2.2193 ms ± 2.58 µs |
| irc_rust | rustc 1.72.0-nightly (101fa903b 2023-06-04) | 969.86 µs ± 0.83 µs |
| justgrep | Go 1.20 | 1.395126 ms |
| go-twitch-irc | Go 1.20 | 3.75188 ms |
| minitwitch | .NET 8.0.100-preview.4.23260.5 | 883.4 µs ± 4.78 µs |
| minitwitch (+ AOT) | .NET 8.0.100-preview.4.23260.5 | 772.0 µs ± 5.27 µs |
Benchmarks were run with macOS 13.4 running on an M2 MacBook Air 16 GB
| library | language | time to parse 1000 lines |
|---|---|---|
| twitch | rustc 1.72.0-nightly (101fa903b 2023-06-04) | 1.0960 ms ± 3.16 µs |
twitch (-F simd) |
rustc 1.72.0-nightly (101fa903b 2023-06-04) | 604.93 µs ± 0.50 µs |
twitch (-F simd + cheating) |
rustc 1.72.0-nightly (101fa903b 2023-06-04) | 422.62 µs ± 1.00 µs |
| twitch-irc | rustc 1.72.0-nightly (101fa903b 2023-06-04) | 3.2780 ms ± 5.72 µs |
| irc_rust | rustc 1.72.0-nightly (101fa903b 2023-06-04) | 1.6897 ms ± 9.81 µs |
| justgrep | Go 1.20 | 1.707313 ms |
| go-twitch-irc | Go 1.20 | 4.714300 ms |
| *minitwitch | .NET 8.0.100-preview.4.23260.5 | 1.387 ms ± 0.0013 ms |
| *minitwitch (+ AOT) | .NET 8.0.100-preview.4.23260.5 | 1.263 ms ± 0.0119 ms |
NOTE: minitwitch was not run with DOTNET_TieredPGO=1, because the benchmark would segfault each time I attempted to run it.