runtime/debug: document BuildInfo.Main.Version == "(devel)"

Question

runtime/debug: document BuildInfo.Main.Version == "(devel)"

mark-rushakoff opened this issue 5 years ago · 26 comments

What version of Go are you using (`go version`)?

$ go version
go version devel +571d93e977 Thu Dec 13 15:08:48 2018 +0000 darwin/amd64

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/mr/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mr/go"
GOPROXY=""
GORACE=""
GOROOT="/Users/mr/gotip/src/github.com/golang/go"
GOTMPDIR=""
GOTOOLDIR="/Users/mr/gotip/src/github.com/golang/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/mr/gomod/debug-module-version-demo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ct/bl4_z3g51ks8239_r2k07v_40000gn/T/go-build638578617=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Repro case is https://github.com/mark-rushakoff/debug-module-version-demo. It's a module, and its main.go contents are a simple use of debug.ReadBuildInfo:

package main

import (
	"fmt"
	"runtime/debug"

	_ "rsc.io/quote"
)

func main() {
	bi, ok := debug.ReadBuildInfo()
	if !ok {
		panic("couldn't read build info")
	}

	fmt.Printf("%s version %s\n", bi.Path, bi.Main.Version)

	for _, d := range bi.Deps {
		fmt.Printf("\tbuilt with %s version %s\n", d.Path, d.Version)
	}
}

What did you expect to see?

I expected to see the first line print github.com/mark-rushakoff/debug-module-version-demo version v0.0.0-20181213... when checked out at an arbitrary commit, or github.com/mark-rushakoff/debug-module-version-demo version v0.0.1 when checked out at tag v0.0.1. I tried both go run . and go build . && ./debug-module-version-demo but both cases printed (devel).

What did you see instead?

github.com/mark-rushakoff/debug-module-version-demo version (devel)
	built with golang.org/x/text version v0.0.0-20170915032832-14c0d48ead0c
	built with rsc.io/quote version v1.5.2
	built with rsc.io/sampler version v1.3.0

Based on the behavior I've observed, it looks as though the main module returned by debug.ReadBuildInfo is hardcoded to (devel) for the main module, which I assume is intended behavior. If so, that's unfortunate for use cases like mycmd version to easily print the module version of the software being built; but it should be documented.

The current documentation at https://tip.golang.org/pkg/runtime/debug/#ReadBuildInfo does not mention (devel) in any way, nor does it mention any special behavior of the Main module.

/cc @hyangah since you're on git blame for src/runtime/debug/mod.go.

Answer 1 · 2018-12-13T20:55:52.000Z

We could perhaps detect the case where the main module happens to be a pristine checkout with a semantically-versioned tag, although that might be misleading if some other module has a cyclic requirement specifying a higher version than the tag.

Answer 2 · 2018-12-22T08:11:59.000Z

+1.

I thought being able to print the program’s main module’s version was the whole point of embedding build information into the binary.

Answer 3 · 2019-01-07T17:03:25.000Z

#26404 (comment) is some separate uncertainty about when and why (devel) shows up. There are some linked issues there that seem to paint a fuller picture, but I don't have time to go through them right now.

I think this issue is still fully separate from #26404, as this is just about documenting (devel).

Answer 4 · 2019-01-24T22:57:11.000Z

It looks like one way to get the version to show up, is to make a new module referring to the one you actually want to build; and then build the target module.

bash-3.2$ gotip mod init ignore
go: creating new go.mod: module ignore

bash-3.2$ gotip get github.com/mark-rushakoff/debug-module-version-demo
go: finding github.com/mark-rushakoff/debug-module-version-demo v0.0.1
go: downloading github.com/mark-rushakoff/debug-module-version-demo v0.0.1
go: extracting github.com/mark-rushakoff/debug-module-version-demo v0.0.1

bash-3.2$ gotip build !$
gotip build github.com/mark-rushakoff/debug-module-version-demo

bash-3.2$ ./debug-module-version-demo
github.com/mark-rushakoff/debug-module-version-demo version v0.0.1
	built with golang.org/x/text version v0.0.0-20170915032832-14c0d48ead0c
	built with rsc.io/quote version v1.5.2
	built with rsc.io/sampler version v1.3.0

Answer 5 · 2019-04-19T22:09:08.000Z

@bcmills

We could perhaps detect the case where the main module happens to be a pristine checkout with a semantically-versioned tag,

Or even just any of the current VCS state. For Caddy's builds, we use tag if clean, or if not clean: nearest tag, commit SHA, and build date (although I can understand why build date is a bad idea if seeking reproducible builds).

For now, @mark-rushakoff's workaround is quite handy.

+1 to this though, some version info on the main module would be immensely helpful.

Answer 6 · 2019-08-29T13:06:30.000Z

As observed in #33926, currently it's possible to end up building a binary with the same main module version but with different dependency versions. People who intend to use the main module version as the binary version should be careful.

Answer 7 · 2019-09-03T17:06:17.000Z

People who intend to use the main module version as the binary version should be careful.

Using the “main module” version as the binary version (in the sense of https://tip.golang.org/cmd/go/#hdr-The_main_module_and_the_build_list) should be fine as long as the module doesn't have filesystem-local replace directives.

Unfortunately, that is not the version that the Main field indicates today (see #33975).

So I agree that the BuildInfo.Main field itself is not really reliable as an indication of the overall configuration in use, even when that version is not (devel).

Answer 8 · 2019-09-13T18:43:21.000Z

Will this be a high priority for Go 1.14? It could really ease the burden of build-from-souce workflows. Like, a lot.

Answer 9 · 2019-09-16T13:36:15.000Z

@mholt, this is unlikely to happen in Go 1.14.

(We have a pretty full slate of issues already, and any time the git command is involved there are thorny security considerations that must be addressed with care.)

Answer 10 · 2019-09-16T19:23:29.000Z

Is using the git command really a requirement of this feature? Wouldn't it be possible to get the relevant metadata directly from .git/HEAD?

Answer 11 · 2019-09-16T19:36:06.000Z

@mback2k, .git/HEAD tells us the commit id of the HEAD revision. It does not tell us:

Whether the working directory is even in a Git repo, or perhaps in some other VCS nested within one.
Where to find the .git directory.
Whether the working tree contains ignored files or other differences from the commit indicated by .git/HEAD.

Answer 12 · 2019-09-16T21:41:27.000Z

The original point of this issue was simply documenting what it means when BuildInfo.Main.Version returns the string (devel). Whether it can be correctly set to a particular git sha seems to be a different topic in my opinion -- perhaps that should be another issue?

Answer 13 · 2019-09-16T21:55:31.000Z

@mark-rushakoff

The original point of this issue was simply documenting what it means when BuildInfo.Main.Version returns the string (devel)

BuildInfo.Main.Version always returns "(devel)".

I oppose documenting this, because I believe this is a bug: BuildInfo.Main.Version always returns "(devel)" even when the main module is versioned at HEAD.

Documenting that it always returns "(devel)" makes it appear as correct behavior and makes it hard or impossible to change later.

Let's fix the actual problem, not document the bug as expected behavior.

Answer 14 · 2019-09-18T18:17:07.000Z

@mholt, we can document the current behavior in a way that does not promise that behavior for all time.

For example, we can document that Buildinfo.Main.Version is (devel) when the go command does not know the version of the main module, without promising that it will never be changed to discover that version.

Answer 15 · 2019-09-19T19:56:57.000Z

nit:

Buildinfo.Main.Version is (devel) when the go command does not know the version of the main module, without promising that it will never be changed to discover that version.

The intention was that the BuildInfo.Main is the module that contains the main package, not the main module described in https://golang.org/cmd/go/#hdr-The_main_module_and_the_build_list
issue 33975.

I hope correcting the documentation error will help explaining this BuildInfo.Main.Version == "(devel)" case.

Answer 16 · 2019-09-19T20:05:18.000Z

Yeah, I think what we want is the version for the "main module" - the module that contains the main package.

Answer 17 · 2020-06-08T16:30:41.000Z

While the main module still shows up as "(devel)" for just go build, the full workaround from #29228 (comment) seems to be no longer necessary. Instead, you can just go get things outside of a current module:

$ GOBIN=$PWD GO111MODULE=on go get github.com/mark-rushakoff/debug-module-version-demo
$ ./debug-module-version-demo 
github.com/mark-rushakoff/debug-module-version-demo version v0.0.1
        built with golang.org/x/text version v0.0.0-20170915032832-14c0d48ead0c
        built with rsc.io/quote version v1.5.2
        built with rsc.io/sampler version v1.3.0

Answer 18 · 2020-08-07T19:59:30.000Z

How would one build a binary (go service in docker image for instance) from ci/docker from a tag to get said tag in the binary being built?
(the source tree is already there (from ci source checkout step) and being copied (dockerfile COPY . .), not downloaded)

Answer 19 · 2020-08-10T20:06:40.000Z

As far as I know, there's no good way to do this (and it appears this issue is somewhat codifying that we'll never know). I'd love to be able to access my current version as it would appear had I created some other module and used @<commit ID> with a tidy, such that I could use that version in other places.

For my docker builds, I pass something in via --build-arg=version=..., then include it by doing things like -ldflags="-X path.to/version.verson=${version}".

Answer 20 · 2020-08-10T20:22:08.000Z

I use the -ldflags="-X path.to/version.verson=${version}" too but I wanted the 2 versions to be in sync
for now I have a super ugly workaround which is to go get ...@tag but it's quite annoying specially on private repos to redownload something that is already there (it forces some very unsightly experimental docker support for ssh in docker etc)

putting it here in case it helps someone else:

DOCKER_BUILDKIT=1 docker build --ssh default ...

and dockerfile with

# syntax=docker/dockerfile:experimental
FROM golang as build
ARG VERSION
...
RUN git config --global url.ssh://git@somePrivateRepo/.insteadOf https://somePrivateRepo/
RUN mkdir -m 700 /root/.ssh && ssh-keyscan -4 -t rsa somePrivateRepo > /root/.ssh/known_hosts
RUN --mount=type=ssh go get -v -a -ldflags -s ... somePrivateRepo/pkg/path@${VERSION}

it would be so much easier for go build to have the option to check with git and/or some sort of --trust-me-the-version-is xxx flag

Answer 21 · 2021-01-06T09:56:15.000Z

Wouldn't adding a -modversion flag on go build solve this problem?
It is way more clear than -ldflags="-X main.version=${version}". Most (id not all) CI services include the tag and the commit hash as an environment variables, so it can be used easily and cleanly for building.

Is there a separate issue for this? I couldn't find it. I implemented it in the middle of writing this comment... here it is

Answer 22 · 2021-01-06T09:59:15.000Z

Change https://golang.org/cl/281912 mentions this issue: cmd/go: add -modversion flag setting main module version on debug.ReadBuildInfo

Answer 23 · 2021-01-06T14:32:55.000Z

@Fryuni The main module doesn't have a version. If it did, MVS would need to work differently in the case where another module requires a higher version of the main module. It may also be misleading if a binary appears to be built from a published version (authenticated by the sumdb) but was actually built from a local checkout with modifications.

Go 1.16 supports building and installing with go install pkg@version (#40276). When that command is used, there is no main module.

#37475 is also an accepted but unimplemented proposal to automatically stamp VCS information into binaries.

Answer 24 · 2022-03-21T21:52:53.000Z

Is this not supposed to be added in go1.18? The release notes seem to indicate such but the module version still reads as (devel) when building in the same directory as the .git folder in a repo with a tag.

Answer 25 · 2022-03-21T21:59:47.000Z

1.18 stamps vcs info but not as the version, that's #50603

Answer 26 · 2023-11-28T03:36:55.000Z

1.18 stamps vcs info but not as the version, that's #50603

I was wondering if there have been any updates on this since the last comment in March 2022; potentially in other issue's I may not be aware of?

What version of Go are you using (go version)?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?