syscall: go generate fails because golang.org/x/sys/windows/mkwinsyscall command isn't vendored

$ go version
go version devel +5d1378143b Thu Oct 8 00:28:09 2020 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/a/.cache/go-build"
GOENV="/home/a/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/a/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/a"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/a/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/a/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/a/go/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build066767711=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I tried to regenerate $GOROOT/syscall/zsyscall_windows.go file. I run go generate in syscall directory.

What did you expect to see?

I expected to see no errors. And have zsyscall_windows.go file updated.

What did you see instead?

$ go generate
cannot find package "." in:
        /home/a/go/src/golang.org/x/sys/windows/mkwinsyscall
syscall.go:29: running "go": exit status 1
$

and my zsyscall_windows.go does not change.

It appears golang.org/x/sys/windows/mkwinsyscall is broken now that we have to use modules.

We also need to update internal/syscall/windows and internal/syscall/windows/registry package. I also use mkwinsyscall in a couple of my own projects. I also suspect that some people use it too. So we need some easy to use solution.

@bcmills for suggestions. I am happy to use whatever approach you suggest.

Thank you.

Alex

Change https://golang.org/cl/260860 mentions this issue: all: use latest version of golang.org/x/sys/windows/mkwinsyscall

CC @jayconrod @matloob

I'm not able to reproduce this failure mode locally:

~/src/golang.org/x/sys/windows$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

~/src/golang.org/x/sys/windows$ go version
go version devel +5d1378143b Thu Oct 8 00:28:09 2020 +0000 linux/amd64

~/src/golang.org/x/sys/windows$ go generate

~/src/golang.org/x/sys/windows$

[Edit: I should have read more carefully. You are generating syscall, not golang.org/x/sys/windows.]

The error message in the output comes from the go/build package:

go/src/go/build/build.go

Line 764 in e92ce92

return p, fmt.Errorf("cannot find package %q in:\n\t%s", path, p.Dir)

That path is guarded by this condition:

go/src/go/build/build.go

Line 757 in e92ce92

if IsLocalImport(path) && !ctxt.isDir(p.Dir) {

The path "." certainly is a local import, but why would ctxt.isDir(p.Dir) be false?

Given the timing, there may be an interaction here with CL 253747.

Ah, now I understand. There are several issues at play here:

When your working directory is $GOROOT/src/syscall, you are working within the std module, and the dependency on golang.org/x/sys/windows/mkwinsyscall is not vendored, because it is not part of the cmd import graph (see #25922).
Because the std module has a vendor directory, the go command defaults to -mod=vendor instead of the usual -mod=readonly.
For some reason, the error message from go/build in this case is not helpful. (This may or may not be a bad interaction with the special case for standard-library vendoring.)

As an immediate workaround, GOFLAGS=-mod=readonly seems to work.

As a longer-term fix, we should either add the go:generate dependencies of the std module to std/vendor, or set -mod=readonly explicitly in the go:generate command.

I'll file a separate issue for the poor diagnostic. It seems to be closely related to #38748, but may or may not share a root cause.

Change https://golang.org/cl/261499 mentions this issue: internal/tools: add a dummy package that imports mkwinsyscall

As an immediate workaround, GOFLAGS=-mod=readonly seems to work.

I am not big module expert. So I used

GO111MODULE=off go generate

in https://go-review.googlesource.com/c/go/+/260860 (I am still waiting for reviewers). I hope it is OK for the moment.