Unable to build binary due to missing go.sum

[F21]

I am using 0.7.0 installed via go get and Go 1.16.

My config file is:

exporters:
  - gomod: "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/newrelicexporter v0.21.0"

Upon executing the builder using opentelemetry-collector-builder --output-path=/tmp/dist --config=.otelcol-builder.yaml, I get the following error:

2021-03-05T15:50:35.630+1100    INFO    cmd/root.go:82  OpenTelemetry Collector distribution builder    {"version": "dev", "date": "unknown"}
2021-03-05T15:50:35.630+1100    INFO    cmd/root.go:98  Using config file       {"path": ".otelcol-builder.yaml"}
2021-03-05T15:50:35.633+1100    INFO    builder/main.go:80      Sources created {"path": "/tmp/dist"}
2021-03-05T15:50:35.666+1100    INFO    builder/main.go:94      Using go from PATH      {"Go executable": "/usr/local/go/bin/go"}
2021-03-05T15:50:35.666+1100    INFO    builder/main.go:97      Compiling
Error: failed to compile the OpenTelemetry Collector distribution: exit status 1. Output: "go: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/newrelicexporter@v0.21.0: missing go.sum entry; to add it:\n\tgo mod download github.com/open-telemetry/opentelemetry-collector-contrib/exporter/newrelicexporter\n"
Usage:
  opentelemetry-collector-builder [flags]

Flags:
      --config string            config file (default is $HOME/.otelcol-builder.yaml)
      --description string       A descriptive name for the OpenTelemetry Collector distribution (default "Custom OpenTelemetry Collector distribution")
      --go string                The Go binary to use during the compilation phase (default "/usr/bin/go")
  -h, --help                     help for opentelemetry-collector-builder
      --include-core             Whether the core components should be included in the distribution (default true)
      --module string            The Go module for the new distribution (default "github.com/jpkroehling/opentelemetry-collector-builder")
      --name string              The executable name for the OpenTelemetry Collector distribution (default "otelcol-custom")
      --otelcol-version string   Which version of OpenTelemetry Collector to use as base (default "0.20.0")
      --output-path string       Where to write the resulting files (default "/tmp/otelcol-distribution776575606")
      --version string           The version for the OpenTelemetry Collector distribution (default "1.0.0")

2021-03-05T15:50:35.702+1100    ERROR   cmd/root.go:77  failed to run   {"error": "failed to compile the OpenTelemetry Collector distribution: exit status 1. Output: \"go: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/newrelicexporter@v0.21.0: missing go.sum entry; to add it:\\n\\tgo mod download github.com/open-telemetry/opentelemetry-collector-contrib/exporter/newrelicexporter\\n\""}
github.com/open-telemetry/opentelemetry-collector-builder/cmd.Execute
        /c/Users/User/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-builder@v0.7.0/cmd/root.go:77
main.main
        /c/Users/User/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-builder@v0.7.0/main.go:22
runtime.main
        /usr/local/go/src/runtime/proc.go:225

From my understanding, the builder should take care of initializing the go.mod and go.sum files and user intervention is not required.

[F21]

This also happens when running test/test.sh using the latest master.

[F21]

This is because go build and other commands in Go 1.16 no longer modify go.mod and go.sum. See modules section here: https://golang.org/doc/go1.16#go-command
I downgraded to a Go 1.15 container to build and it works perfectly. I think a solution would be to have the builder run go mod tidy before executing go build.

[jpkrohling]

Or go get? Would you like to send a PR for this?

[F21]

I don't have enough bandwidth to dig into the code at the moment, but will try to revisit this if nobody gets to it in a couple of weeks.

[jpkrohling]

Alright. For reference, here's the relevant code:

opentelemetry-collector-builder/internal/builder/main.go

Lines 37 to 43 in f29b566

	func GenerateAndCompile(cfg Config) error {
	if err := Generate(cfg); err != nil {
	return err
	}
	return Compile(cfg)
	}

opentelemetry-collector-builder/internal/builder/main.go

Lines 84 to 106 in f29b566

	// Compile generates a binary from the sources based on the configuration
	func Compile(cfg Config) error {
	goBinary := cfg.Distribution.Go
	// first, we test to check if we have Go at all
	if _, err := exec.Command(goBinary, "env").CombinedOutput(); err != nil {
	path, err := exec.LookPath("go")
	if err != nil {
	return ErrGoNotFound
	}
	goBinary = path
	cfg.Logger.Info("Using go from PATH", "Go executable", path)
	}
	cfg.Logger.Info("Compiling")
	cmd := exec.Command(goBinary, "build", "-ldflags=-s -w", "-trimpath", "-o", cfg.Distribution.ExeName)
	cmd.Dir = cfg.Distribution.OutputPath
	if out, err := cmd.CombinedOutput(); err != nil {
	return fmt.Errorf("failed to compile the OpenTelemetry Collector distribution: %w. Output: %q", err, out)
	}
	cfg.Logger.Info("Compiled", "binary", fmt.Sprintf("%s/%s", cfg.Distribution.OutputPath, cfg.Distribution.ExeName))
	return nil
	}

[eroteme]

I've added a Pull request for this as I hit the same problem while trying to upgrade to otel v0.22.0

[fangyi-zhou]

Hi @jpkrohling, would you mind creating a new release of the binary? I think the latest version doesn't work under Go 1.16 due to this issue

[jpkrohling]

Version 0.23.0 should be ready in a few minutes.