ory/go-acc

[README.md] relationship between go-acc and Go 1.20 GOCOVERDIR feature

thediveo opened this issue · 3 comments

thediveo commented

For several years, go-acc has me served very well in my admittedly simple case of aggregating coverage data from root as well as a non-root test runs. A big thank-you for your work from a small fish.

Now, Go 1.20 is finally another interesting fish in the sea in this respect, with its new feature of aggregating multiple test and integration test runs. Aggregating multiple unit test runs can be achieved using go test ... -args -test.gocoverdir="GOCOVERDIR" (as "documented" here) .

How does this Go 1.20 feature relate to go-acc? Does it now the same or where are differences?

Do you think such a feature comparison to be useful to devs either using go-acc or thinking about using it?

aeneasr commented

Thank you :)

Very nice to see that Go finally delivered on this. If it turns out that Go does this correctly, I think we'll just archive this tool here.

mitar commented

There is also a fix in Go 1.22 which includes coverage in packages without tests: golang/go#24570

Also, you can run -coverpkg ./... instead of having to list all packages. So I think -coverpkg ./... + reporting coverage for code without tests does now the right thing.

And if one has to combine data from multiple runs (I combine both data from unit tests, integration tests, and e2e tests), there is now GOCOVERDIR and go tool covdata tooling.

  • 👍1
mitar commented

In fact, I think there are some recent issues with Go and coverage and this tool as well. I made a fork of testing repository and added go.mod to it.

Now, if I run with go version go1.22.1 linux/amd64:

$ go test -cover -covermode=atomic -count 1 ./...
ok  	github.com/mitar/accurate-test-coverage	0.003s	coverage: 0.0% of statements [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	0.005s	coverage: 100.0% of statements
ok  	github.com/mitar/accurate-test-coverage/pkg	0.005s	coverage: 50.0% of statements

And this returns the same:

$ go-acc ./... -- -count 1
ok  	github.com/mitar/accurate-test-coverage	0.003s	coverage: 0.0% of statements in github.com/mitar/accurate-test-coverage, github.com/mitar/accurate-test-coverage/otherpkg, github.com/mitar/accurate-test-coverage/pkg [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	0.003s	coverage: 100.0% of statements in github.com/mitar/accurate-test-coverage, github.com/mitar/accurate-test-coverage/otherpkg, github.com/mitar/accurate-test-coverage/pkg
ok  	github.com/mitar/accurate-test-coverage/pkg	0.003s	coverage: 50.0% of statements in github.com/mitar/accurate-test-coverage, github.com/mitar/accurate-test-coverage/otherpkg, github.com/mitar/accurate-test-coverage/pkg

But the correct values are:

$ go test -covermode=atomic -count 1 -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	0.003s	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	0.004s	coverage: 75.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	0.003s	coverage: 25.0% of statements in ./...

I think go-acc miscounts packages which have no tests. This was fixed in Go 1.22.

  • 👍1