klauspost/pgzip

SetConcurrency(?,?)

ayamzh opened this issue · 3 comments

If only one goroutine is used for the setting, what is the performance compare with gzip ; SetConcurrency(?,?), what is the optimal setting for the second parameter

There is little point then, except maybe hiding IO from output writes. Just use regular gzip.

func BenchmarkPGzip(b *testing.B) {
dat, _ := ioutil.ReadFile("testdata/test.json")
dat = append(dat, dat...)
dat = append(dat, dat...)
buf := new(bytes.Buffer)
w := NewWriter(buf)
b.ResetTimer()

for n := 0; n < b.N; n++ {
	w.Write(dat)
	w.Close()
	w.Reset(buf)
}
w.Close()

}

func BenchmarkGzip(b *testing.B) {
dat, _ := ioutil.ReadFile("testdata/test.json")
dat = append(dat, dat...)
dat = append(dat, dat...)
buf := new(bytes.Buffer)
w := gzip.NewWriter(buf)
b.ResetTimer()
for n := 0; n < b.N; n++ {
w.Write(dat)
w.Close()
w.Reset(buf)
}
w.Close()
}

cpu: Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz
BenchmarkMazhengPGzip-16 4728 255777 ns/op
BenchmarkMazhengGzip-16 6830 177580 ns/op
@klauspost the result is gzip is faster than pgzip,why

You should only use this if you are (de)compressing big amounts of data, say more than 1MB at the time, otherwise you will not see any benefit, and it will likely be faster to use the internal gzip library or this package.

Input too small.