golang/mock

Unable to generate mocks in reflect mode

nezorflame opened this issue · 13 comments

nezorflame commented

Actual behavior A clear and concise description of what the bug is.

My project uses Go modules, but due to the CI restrictions we also make use of the vendor folder.
So, our modules are synced by go mod tidy && go mod vendor command.

While using mockgen in a reflect mode, I'm unable to generate mocks. Can't pinpoint exactly when this became broken as this was working fine before, but it doesn't anymore.

The error message:

❯ go generate ./...

/Users/nezorflame/Go/src/github.com/nezorflame/mockgen-test - service - doc.go
prog.go:12:2: cannot find package "." in:
        /Users/nezorflame/Go/src/github.com/nezorflame/mockgen-test/vendor/github.com/golang/mock/mockgen/model
prog.go:12:2: cannot find package "." in:
        /Users/nezorflame/Go/src/github.com/nezorflame/mockgen-test/vendor/github.com/golang/mock/mockgen/model
prog.go:12:2: cannot find module providing package github.com/golang/mock/mockgen/model: working directory is not part of a module
prog.go:14:2: cannot find module providing package github.com/nezorflame/mockgen-test/internal/client: working directory is not part of a module
2020/10/22 19:41:04 Loading input failed: exit status 1
internal/service/doc.go:2: running "mockgen": exit status 1

Seems to be somewhat similar to the #423 but the proposed solution didn't work for me.

Expected behavior A clear and concise description of what you expected to
happen.

Mocks should've been generated fine.

To Reproduce Steps to reproduce the behavior

I've published a test project which can be used to reproduce the issue.

mockgen is called during the go:generate directive at internal/service/doc.go:

//go:generate echo $PWD - $GOPACKAGE - $GOFILE
//go:generate mockgen -package $GOPACKAGE -destination mocks_test.go github.com/nezorflame/mockgen-test/internal/client Producer

The interface to generate the mocks upon is pretty simple and is stored at internal/client/contract.go:

type Producer interface {
	Produce(ctx context.Context, objects []string) error
}

So, the steps are:

  1. Download the project
  2. (Optional) Generate the mocks without the vendor:
    a. Run go mod tidy && go mod vendor to clean up the dependencies
    b. Run go generate ./... once - it'll work fine, mocks_test.go file will be generated
    c. Delete mocks_test.go file to reproduce the error
  3. Run go generate ./...
  4. The error appears

It seems that the fact that vendor contains dependencies somehow breaks the ability of mockgen to generate the mocks, since the vendor-less mode works fine.

Additional Information

  • gomock mode (reflect or source): reflect
  • gomock version or git ref: v1.4.4
  • golang version: v1.5.3

go env output:

GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/nezorflame/Library/Caches/go-build"
GOENV="/Users/nezorflame/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="<my_gopath>/pkg/mod"
GONOPROXY="my.company.com"
GONOSUMDB="my.company.com"
GOOS="darwin"
GOPATH="<my_gopath>"
GOPRIVATE="my.company.com"
GOPROXY="https://proxy.golang.org,https://goproxy.io,https://gocenter.io,direct"
GOROOT="<my_goroot>"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="<my_goroot>/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/4p/q8s4q9s50qbdl5vllqg5fs000000gp/T/go-build294373709=/tmp/go-build -gno-record-gcc-switches -fno-common"
cvgw commented

The error message is due to github.com/golang/mock/mockgen/model not being vendored golang/go#41936

I would assume the reason you started seeing it at an indeterminate time has to do with them slowly making modules the default behavior in new versions of golang.

How to fix this is a bit more tricky.

The golang program generated by reflection mode depends on github.com/golang/mock/mockgen/model, but in most cases nothing else would depend on this, hence it not being included in the vendor folder.

There are probably some hacky work arounds like forcing the import with an _ "github.com/golang/mock/mockgen/model", but I'm not sure what the long term fix is.

cc @codyoss

  • 👍3
cvgw commented

As super hacky work around you can copy the directory github.com/golang/mock/mockgen/model into your vendor folder

codyoss commented

I think this is related to recent changes in Go tooling. Adding --build_flags=--mod=mod is one workaround. The other would be to include an empty import of that package so it gets pulled in import _ "github.com/golang/mock/mockgen/model".

  • 👍18
  • ❤️3
nezorflame commented

Thanks!
For now I'll use the proposed workaround with the empty import (and a comment to this issue), but it'd be nice to see this resolved.
Anything I could do to help further?

cvgw commented

Thanks!
For now I'll use the proposed workaround with the empty import (and a comment to this issue), but it'd be nice to see this resolved.
Anything I could do to help further?

I think what needs to happen now is some discussion about how to conceptually solve this problem; any input would be welcome and appreciated.

The basic overview (IIUC @codyoss can correct me) is that during reflection mode we dynamically generate some golang source code for a new main program. This main program, the reflection program, is the heart of what makes reflection mode work.

The source code is written out by mockgen and then mockgen executes some shell commands to compile and run the reflection program.

Problem we are hitting is in the dependency management; the reflection program requires github.com/golang/mock/mockgen/model but your code does not.

When the go toolchain attempts to compile the reflection program it sees your vendor directory, but github.com/golang/mock/mockgen/model is not contained and so it fails.

So we essentially have two different build environments that require different dependencies.

  • 👍2
valters commented

One of the problems is that the error message is so cryptic and unergonomic, it is impossible to figure out why this is happening. And the reason why (as comments above explain) is that that the mockgen/model directory never was vendored in the first place.

My workaround (strangely enough) was to remove the whole vendor directory, and this allows mockgen to work.

  • 👍4
codyoss commented

After looking more into this issue I am not sure there is too much we can do about this. For now I will treat this as a documentation issue and add some notes in the readme.

caskey commented

If documentation is the solution you could add a pointer to the readme in the error as this is a known problem and the error message printed is a dead end for the user.

  • 👍2
codyoss commented

@caskey That is a great idea, done!

estebane-frb commented

Hi there, any chance this has changed since last year? I'm new to GoMock but ran into this. Thanks for all you do for Go and the OSS community in general!

escalopa commented

Hey I also came across this issue today, Looks like installing the mockgen in GOPATH is not enough to make this work.

The ERROR I had

prog.go:12:2: missing go.sum entry for module providing package github.com/golang/mock/mockgen/model; to add:
        go mod download github.com/golang/mock
prog.go:12:2: missing go.sum entry for module providing package github.com/golang/mock/mockgen/model; to add:
        go mod download github.com/golang/mock
prog.go:14:2: no required module provides package github.com/escalopa/go-bank/db/sqlc: go.mod file not found in current directory or any parent directory; see 'go help modules'
prog.go:12:2: no required module provides package github.com/golang/mock/mockgen/model: go.mod file not found in current directory or any parent directory; see 'go help modules'
2022/11/27 00:08:24 Loading input failed: exit status 1

SOLUTION

Install the package using the following command

go get github.com/golang/mock/

The output should be like this

go: upgraded github.com/golang/mock v1.4.4 => v1.6.0

After so ran the mockgen command and everything worked. Notice Change command below to one that matches your project

mockgen -destination db/mock/store.go github.com/escalopa/go-bank/db/sqlc Store

As a result you should see the the following files have been updated

  • go.mod
  • go.sum

This is how to worked for me, Wish this might help 😄

  • 👍14
  • 🎉1
  • ❤️7
ryanrolds commented

Ran into this and resolve it by creating ./tools/tools.go with the contents of:

package tools

import (
	_ "github.com/golang/mock/mockgen/model"
)

Then I was able to:

go get github.com/golang/mock/mockgen/model
go mod vendor
go mod tidy
CodeNinjaUG commented

thnks it worked for me