gonum/blas

benchmark stubs

Closed this issue · 2 comments

fhs commented

It looks like BenchmarkDgemmLgSmLg and BenchmarkDgemmLgLgSm are benchmarking the same thing. Also, BenchmarkDgemmHgHgSm should probably be called BenchmarkDgemmHgSmHg.

Given that there are 4*4*4*2*2 = 256 possible benchmarks for Dgemm, you probably don't want to autogenerate them all. That's just too many benchmarks for a single function.

In that case, you probably want to enumerate through the cases in a systematic way so that these kinds of mistakes don't creep in. It might also help to write each BenchmarkXxx strub within a single line, forming a kind of table:

var (
    Bench = testblas.DgemmBenchmark
    Lg = testbench.LargeMat
    ... etc.
)
func BenchmarkDgemmLgSmLg(b *testing.B) { Bench(Blas{}, Lg, Sm, Lg, NT, NT) }
func BenchmarkDgemmLgLgSm(b *testing.B) { Bench(Blas{}, Lg, Lg, Sm, NT, NT) }
... etc.

I agree on the table, but the enumerations are useful to determine the asymptotic behavior of Dgemm, which is probably the single most important function in the package. Maybe using the test.short flag would be appropriate, so that we can skip exhaustive benchmarks?

fhs commented

closed by above PR