Middleware benchmarks are still skewed (follow-up to #156)
jub0bs opened this issue · 0 comments
Unfortunately, the benchmarks are still skewed, even after #156. Sorry I didn't realise this at the time of the PR in question.
Problem
In a given benchmark, a FakeResponse
object gets re-used across iterations. At the start of each iteration, its header field (of type http.Header
) is cleared. However, the clear
function empties the map without reducing its capacity. Therefore, the cost (esp. in terms of heap allocations) of re-adding headers to the shared FakeResponse
object in subsequent iterations is amortised.
Of course, net/http
does not re-use response objects from one handler invocation to the next. Therefore, the benchmark results look better than they realistically should.
One solution
One way to fix the issue is to use a different FakeResponse
object for each iteration. You don't want to account for the initialisation of each FakeResponse
in the benchmark results, though. One possible approach consists in creating a collection of FakeResponse
s (one for each benchmark iteration) before starting the benchmark loop.
func BenchmarkWithout(b *testing.B) {
resps := makeFakeResponses(b.N)
req, _ := http.NewRequest(http.MethodGet, dummyEndpoint, nil)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
testHandler.ServeHTTP(resps[i], req)
}
}
func makeFakeResponses(n int) []*FakeResponse {
resps := make([]*FakeResponse, n)
for i := 0; i < n; i++ {
resps[i] = &FakeResponse{http.Header{}}
}
return resps
}
New benchmark results:
$ go test -benchmem -run '^$' -bench .
goos: darwin
goarch: amd64
pkg: github.com/rs/cors
cpu: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
BenchmarkWithout-8 269766921 20.93 ns/op 0 B/op 0 allocs/op
BenchmarkDefault-8 3944059 268.4 ns/op 352 B/op 1 allocs/op
BenchmarkAllowedOrigin-8 4456435 285.8 ns/op 352 B/op 1 allocs/op
BenchmarkPreflight-8 2806293 425.9 ns/op 352 B/op 1 allocs/op
BenchmarkPreflightHeader-8 2219377 517.5 ns/op 352 B/op 1 allocs/op