Building with SDK v0.37.0 fails fetch-vendored task due to missing GOPROXY
markusboehme opened this issue · 4 comments
Platform I'm building on: Fedora 36 on x86_64
What I expected to happen: With the current develop branch after merging #3690, I expected cargo make -e BUILDSYS_VARIANT=metal-dev to succeed.
What actually happened: The fetch-vendored task fails with github.com/aws/aws-sdk-go/aws: GOPROXY list is not the empty string, but contains no entries and similar entries:
[cargo-make][1] INFO - Running Task: fetch-vendored
go: downloading github.com/containerd/containerd v1.6.26
go: downloading github.com/aws/aws-sdk-go v1.49.13
go: downloading github.com/awslabs/amazon-ecr-containerd-resolver v0.0.0-20231213164356-22a8023e9be3
go: downloading github.com/urfave/cli/v2 v2.27.1
go: downloading google.golang.org/grpc v1.60.1
go: downloading github.com/aws/aws-sdk-go v1.49.13
go: downloading github.com/urfave/cli/v2 v2.27.1
go: downloading github.com/awslabs/amazon-ecr-containerd-resolver v0.0.0-20231213164356-22a8023e9be3
go: downloading github.com/containerd/containerd v1.6.26
go: downloading google.golang.org/grpc v1.60.1
go: host-ctr/cmd/host-ctr imports
github.com/aws/aws-sdk-go/aws: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/aws/aws-sdk-go/aws/session: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/aws/aws-sdk-go/service/ecrpublic: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/awslabs/amazon-ecr-containerd-resolver/ecr: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd/cio: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd/containers: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd/contrib/seccomp: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd/errdefs: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd/log: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd/namespaces: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd/oci: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd/pkg/cri/server: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd/remotes/docker: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/containerd/containerd/runtime/v2/runc/options: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
github.com/urfave/cli/v2: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
k8s.io/cri-api/pkg/apis/runtime/v1 imports
google.golang.org/grpc: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
k8s.io/cri-api/pkg/apis/runtime/v1 imports
google.golang.org/grpc/codes: GOPROXY list is not the empty string, but contains no entries
go: host-ctr/cmd/host-ctr imports
k8s.io/cri-api/pkg/apis/runtime/v1 imports
google.golang.org/grpc/status: GOPROXY list is not the empty string, but contains no entries
[cargo-make][1] ERROR - Error while executing command, exit code: 1
How to reproduce the problem: Have no GO* environment variables set (ensure env | grep ^GO is empty), no go.env file is present, and then cargo make.
I haven't looked yet at what GOPROXY is set at during the build or who sets it (is not the empty string). I also don't know why it looks to affect the dependencies of host-ctr only. In the meantime, explicitly export GOPROXY=direct in the build environment does the trick, but wasn't needed before the update to SDK v0.37.0/Go v1.21.0.
SDK v0.36.1 has a default GOPROXY set:
$ docker run --rm public.ecr.aws/bottlerocket/bottlerocket-sdk-x86_64:v0.36.1 /usr/libexec/go/bin/go env | grep GOPROXY
GOPROXY="https://proxy.golang.org,direct"
SDK v0.37.0 does not:
$ docker run --rm public.ecr.aws/bottlerocket/bottlerocket-sdk-x86_64:v0.37.0 /usr/libexec/go/bin/go env | grep GOPROXY
GOPROXY=''
Wondering if the error message (GOPROXY list is not the empty string) is misleading or whether buildsys or twoliter perhaps misinterpret an empty GOPROXY.
From https://cs.opensource.google/go/go/+/7aa85e01376d840acc8bb931156d607a00b64a60
Various Linux distributions edit cmd/go/internal/cfg/cfg.go to change
the default settings of GOPROXY and GOSUMDB. Make it possible for
them to do this without editing the go command source code by
introducing GOROOT/go.env and moving those defaults there.
It would seem we need to package go.env during the SDK build then, because there is no such file in the resulting image.
This diff for bottlerocket-sdk includes the go.env file with the defaults, and GOPROXY is subsequently set to the old default:
diff --git a/Dockerfile b/Dockerfile
index 0c95d1a..5c3aae5 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1012,6 +1012,7 @@ COPY --chown=0:0 --from=sdk-go /home/builder/sdk-go/bin /usr/libexec/go/bin/
COPY --chown=0:0 --from=sdk-go /home/builder/sdk-go/lib /usr/libexec/go/lib/
COPY --chown=0:0 --from=sdk-go /home/builder/sdk-go/pkg /usr/libexec/go/pkg/
COPY --chown=0:0 --from=sdk-go /home/builder/sdk-go/src /usr/libexec/go/src/
+COPY --chown=0:0 --from=sdk-go /home/builder/sdk-go/go.env /usr/libexec/go/go.env
COPY --chown=0:0 --from=sdk-go \
/home/builder/sdk-go/licenses/ \
/usr/share/licenses/go/
I haven't run the full matrix of SDK builds yet to be able to actually test a build of Bottlerocket, but the output of go env in one of the SDK images looks right to me.