extended forwards compatibility for Go
rsc opened this issue · 209 comments
Many people believe the go line in the go.mod file specifies which Go toolchain to use. This proposal would correct this widely held misunderstanding by making it reality. At the same time, the proposal would improve forward compatibility by making sure that old Go toolchains never try to build newer Go programs.
Define the “work module” as the one containing the directory where the go command is run. We sometimes call this the “main module”, but I am using “work module” in this document for clarity.
Updating the go line in the go.mod of the work module, or the go.work file in the current workspace, would change the minimum Go toolchain used to run go commands. A new toolchain line would provide finer-grained control over Go toolchain selection.
An environment variable GOTOOLCHAIN would control this new behavior. The default, GOTOOLCHAIN=auto, would use the information in go.mod. Setting GOTOOLCHAIN to something else would override the go.mod. GOTOOLCHAIN=local would force use of the locally installed toolchain, and other values would choose specific releases. For example, to test the package in the current directory with Go 1.17.2:
GOTOOLCHAIN=go1.17.2 go test
As part of this change, older Go toolchains would refuse to try to build newer Go code. The new system would arrange that normally this would not come up - the new toolchain would be used automatically. But if forced, such as when using GOTOOLCHAIN=local, the older Go toolchain would no longer assume it can build newer Go code. This in turn would make it safe to revisit and fix for loop scoping (discussion #56010).
See my talk on this topic at GopherCon for more background.
See the design document for details.
Updated link to design doc: https://go.dev/design/57001-gotoolchain.
I just want to note that the potential for confusion here is high. go build and go install should report whenever they invoke a different toolchain.
I'm not sure about that. What I hope will be a common mode of usage is that you install some update-aware Go toolchain on your machine and then from that point on just edit go or toolchain lines in your go.mod in various projects and never explicitly update the locally installed Go toolchain ever again. The Go toolchain becomes a detail managed inside go.mod just like all the other inputs to your build. In that mode of usage, this reporting would print on literally every go command that gets run, which is too noisy.
I think the combination of auto-upgrading and not allowing old toolchains to build new code will change the Go ecosystem to either no longer support multiple versions of Go, or to delay experimenting with new language features.
Where previously a Go module could target Go 1.21 but make use of 1.22 features in optional, build-tagged files, it can no longer do so. To be allowed to use Go 1.22 language features, the go.mod has to specify go 1.22, but this triggers auto-upgrading / build failures on Go 1.21. The module now only supports Go 1.22.
Auto-uprading makes this seem fine at first: anyone who needs Go 1.22 will get it for free, automatically. However, I don't think auto-upgrading can be assumed to be pervasive. For one, most people who hack on Go at least occasionally will be using GOTOOLCHAIN=local, pointed at a git checkout. Furthermore, I predict that some Linux distributions will patch their Go packages to disable the automatic upgrading, as it side-steps their packages and some are allergic to that. This'll lead to users who are left behind even more than they're now due to slow-moving distributions.
I personally have a few reservations about this proposal.
Firstly, I don't feel that the Go version in go.mod should be the "minimum" supported Go version for a module. The way I have understood it has been that it's a "feature unlocker". See https://twitter.com/_myitcv/status/1391819772992659461, https://twitter.com/marcosnils/status/1391819263325974530, #46201
It's very possible to enable a newer version of Go in go.mod to use things that are only available in that version, but gate those uses behind build tags, falling back to something else for older versions of Go. For example, //go:embed can be placed behind build tags such that it won't be used if the version of Go in use can't support it. The same applies for new exported stdlib types/functions. If go.mod enforced a minimum, that code couldn't be written (or at least be annoying to test).
This pattern seems common enough; x/tools and x/tools/gopls both set go 1.18, but in actuality test and support versions back to Go 1.16. It seems like this setup wouldn't be very tenable with this proposal implemented, as Go would automatically ignore that and download something else. Sure, maybe they could use GOTOOLCHAIN to work around that, but I think that'd be a major shift in how people generally get Go in CI. (It's also awkward to be able to go forward by changing Go on $PATH, but not backwards.)
Secondly, the automatic download/execution of binary releases of Go seems really surprising. I feel like it's going to be very awkward for Linux distributions to lose control of the toolchain in use without environment variables (especially if they patch Go). I do wonder how many distros might patch Go entirely to force GOTOOLCHAIN=local as the default. I believe there are also examples of corporate forks of Go (I've seen Microsoft's mentioned before), and those would also likely patch away this behavior becuase it'd be a bad thing for those to start bypassing the expected toolchain, especially without any sort of warning message that it's happening.
There are also systems where the binaries downloaded from golang.org won't be functional, e.g. Nix/NixOS (where the system layout is entirely different and outside binaries require a lot of work to use), musl-based distros like alpine (musl is not a first class libc for Go, as far as I know), and so on. Those distributors may also have to disable this download functionality to prevent breakage (be it to make things work consistently in the first place, or to just stop users from reporting the failures as distro bugs).
This part of the proposal is being compared to module dependency management itself. I strongly feel that modules work really, really well in comparison to other languages' package management scenarios (like node_modules, virtualenv, etc) in that it all happens automatically for you. But, I think the way this proposal does this for Go versions itself is going to be too surprising and likely to break.
(I mentioned some of this in #55092 (comment), but my thread seems to have been missed as all of the threads around it were replied to.)
I spoke to @ianlancetaylor about his concerns. For most commands, you can always run go version to see what version you are going to get in the next command. I think that will be enough for those commands (Ian does not completely agree but is willing to wait and see). The one exception is go install path@version.
For go install path@version, I think we probably have to do the module fetch and then potentially re-exec the go command based on the go line. I would be inclined to say we don't respect any toolchain line, just as we don't respect any replace lines. That is, today go install path@version is shorthand for:
mkdir /tmp/empty
cd /tmp/empty
go mod init empty
go get path@version
go install path
I think it should continue to be shorthand for that, which would mean ignoring the toolchain line in path/go.mod.
In the case where go install path@version needs to fetch and run a newer Go toolchain, I think it would be appropriate to print a message:
$ go install path@version
go: building path@version using go1.25.1
$
I think the combination of auto-upgrading and not allowing old toolchains to build new code will change the Go ecosystem to either no longer support multiple versions of Go, or to delay experimenting with new language features.
Where previously a Go module could target Go 1.21 but make use of 1.22 features in optional, build-tagged files, it can no longer do so. To be allowed to use Go 1.22 language features, the go.mod has to specify go 1.22, but this triggers auto-upgrading / build failures on Go 1.21. The module now only supports Go 1.22.
This is true. Older versions of the module will still support Go 1.21, of course. It's just the latest version of the module that only supports Go 1.22. I don't think it's a sure thing that this is a problem. The same happens today for dependencies, of course, and it seems to be fine.
I agree that making it easier to update to a new Go toolchain may well result in people updating more quickly.
Auto-upgrading makes this seem fine at first: anyone who needs Go 1.22 will get it for free, automatically. However, I don't think auto-upgrading can be assumed to be pervasive. For one, most people who hack on Go at least occasionally will be using GOTOOLCHAIN=local, pointed at a git checkout. Furthermore, I predict that some Linux distributions will patch their Go packages to disable the automatic upgrading, as it side-steps their packages and some are allergic to that. This'll lead to users who are left behind even more than they're now due to slow-moving distributions.
People who hack on Go will be using GOTOOLCHAIN=local (that will be the default in their builds), but they will also presumably be hacking on the latest Go version, which will not be too old for any code.
I agree that some Linux distributions are likely to patch Go to default to GOTOOLCHAIN=local. That seems fine too, as long as users can still go env -w GOTOOLCHAIN=auto.
@zikaeroh, Alpine should not have problems running standard Go distributions starting in Go 1.20. I am not sure about NixOS. The only thing it should need is a libc.so.6 for the dynamic linker to resolve. Or maybe we should build the distribution cmd/go with -tags netgo and then it wouldn't even need that. I wonder what that would break...
My (rudimentary) understanding is that libc.6.so is not located in the "typical" location on NixOS, so would not be found by a normal Go binary. It'd be at a long path like /nix/store/ikl21vjfq900ccbqg1xasp83kadw6q8y-glibc-2.32-46/lib/libc.so.6 as each package deterministically uses specific other packages. So, a flavor of the Go package that uses the precompiled binary releases of Go would use patchelf to fix this. Downloaded versions would not have that patching applied, and therefore would not work.
This pattern seems common enough; x/tools and x/tools/gopls both set go 1.18, but in actuality test and support versions back to Go 1.16. It seems like this setup wouldn't be very tenable with this proposal implemented, as Go would automatically ignore that and download something else. Sure, maybe they could use GOTOOLCHAIN to work around that, but I think that'd be a major shift in how people generally get Go in CI. (It's also awkward to be able to go forward by changing Go on $PATH, but not backwards.)
x/tools/gopls only tests that far back because they want to build with what's on people's machines. If what's on people's machines knew how to fetch a newer toolchain then we'd have stopped needing to support older versions long ago.
I think that one reason people are slow to update to new Go versions because it is too difficult. What's difficult is managing Go installations. This proposal removes that difficulty, which in turn should make it easier for people to keep up with newer versions.
Regarding Linux distributions setting GOTOOLCHAIN=local by default (which I think would be fine), I'm curious whether they apply similar rules to rustup or nvm. Does anyone know?
I think that one reason people are slow to update to new Go versions because it is too difficult. What's difficult is managing Go installations. This proposal removes that difficulty, which in turn should make it easier for people to keep up with newer versions.
I guess I'm confused; my impression was that the hardest problem in upgrading was not obtaining a new version of Go, but making sure that the Go code works for that new version of Go, which is the backwards compatibility proposal (not this one).
I would be very interested to know what proportion of Go users obtain Go via a package manager (versus golang.org). It seems to me like most package managers are going to set GOTOOLCHAIN=local (as noted by @dominikh and me above). But it also seems to me like most Linux users are getting Go via their distro's package manager and macOS users via brew (and I personally use scoop/chocolately/winget on Windows).
If all of those users are going to end up getting local as their default, is anyone going to use auto?
I'm curious whether they apply similar rules to rustup or nvm. Does anyone know?
On Arch, you can either install the rust package or rustup; both "provide" rust, but the actual packages are usually built in a clean chroot and automatically get the former, the latest rust compiler package.
Arch doesn't package nvm or any other node version manager; without the AUR, only the latest node/npm/yarn are provided. But, I think it's the case that many users may end up installing nvm (or its many alternatives), in which case the version in use is basically whatever anyone wants. The closest example I can think is projects which check in .node_version or use volta to pin a particular version for their project, but I've more often seen this for pinning a local dev setup and then more versions are checked in CI anyway.
We should describe what happens if we drop an existing port (per https://go.dev/wiki/PortingPolicy). Presumably the go command 1.N will see "go 1.N+1", try to download the binaries for 1.N+1, fail, and then give an error and stop the build.
For cases like NixOS I think we would have to expect users on that system to set GOTOOLCHAIN=local. And that in turn suggests that perhaps there should be a way to build a toolchain such that GOTOOLCHAIN=local is the default if not overridden by the environment variable.
This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group
I filed #57007 for building cmd/go without libc.so.6, which I think would make NixOS happy.
I agree that it should be possible to build a toolchain with GOTOOLCHAIN=local as the default, but I don't think most package managers should do this. For example I don't think it makes sense for user-installed package managers like Chocolatey or Homebrew to do this at all. They are not as pedantic about "we are the only way to install software on your machine!" as the operating system-installed package managers are, and they should not be going out of their way to break what will end up being a core feature of the Go experience.
I also think a fair number of Go developers still use the installers we provide, and those of course will have the GOTOOLCHAIN=auto default.
We should describe what happens if we drop an existing port (per https://go.dev/wiki/PortingPolicy). Presumably the go command 1.N will see "go 1.N+1", try to download the binaries for 1.N+1, fail, and then give an error and stop the build.
Yes, exactly. And we can give a good error.
I guess I'm confused; my impression was that the hardest problem in upgrading was not obtaining a new version of Go, but making sure that the Go code works for that new version of Go, which is the backwards compatibility proposal (not this one).
Go's backward compatibility is already very good. To the extent that it needs work, the backwards compatibility proposal will make it even better. That will leave actually getting the upgraded Go toolchain as the hardest problem. My experience maintaining other machines where I build Go programs but don't do Go development has been that I don't upgrade often at all because it is annoying to go download and unpack the right tar files. If all it took was editing a go.mod, I would do that far more often.
Another point that I forgot to make in the doc is that setting the Go toolchain in go.mod means that all developers working on a project will agree on the toolchain version, without other special arrangements. This was of course one of the big improvements of modules and moving out of GOPATH, for dependency versions. The same property can be provided for the Go toolchain version. This would also mean that if you are moving back and forth between two projects that have chosen different Go toolchain versions as their standard toolchain, you get the right one for that project automatically by virtue of just being in that project. You don't have to change your PATH each time you switch, or maintain a global symlink in $HOME/bin, or remember to type 'go1.18 build' in one place and 'go1.19 build' in the other, or any other kludge. It just does the right thing.
In a CI/CD environment, I don't think this feature would be useful. I would expect that job to be configured to use the correct Go version in the first place. Downloading a newer toolchain might not work anyway due to firewall restrictions. And as others have mentioned, using an older compiler with GOTOOLCHAIN=local should not fail, as it may mean the module only uses newer language features conditionally and is verifying compatibility. In addition, this could easily lead to a situation where a developer who attempts to test with an older go version on their local machine but forgets to set GOTOOLCHAIN=local (or doesn't know about it) will get different results than the build server.
On another note, while the go directive in go.mod controls language features like octal literals and generics, today it has no bearing on the standard library implementation. My expectation is that if I run go build with compiler version 1.X, then it will use standard library version 1.X. But with this change, that will not be the case if I use an older compiler, unless I set GOTOOLCHAIN=local.
Another point that I forgot to make in the doc is that setting the Go toolchain in go.mod means that all developers working on a project will agree on the toolchain version, without other special arrangements.
We build our own version of the toolchain and distribute it to our developers, rather than using what is on golang.org. So this would not address that problem for us, especially because it sounds like such a toolchain will default to GOTOOLCHAIN=local anyway. What would be more useful for us in this regard is a way to do an exact string match on the compiler's go version and fail if it is wrong. But this is independent of the go directive. (For example, we are currently using 1.18, but our go.mod file says 1.17 to prevent developers from using generics.) I see this is touched on with the new toolchain directive but it's not clear it would buy much in our particular use case.
Another use case is if I am writing a library and want to easily test it with multiple go versions on my local machine. It would be convenient if I could just do something like go test -go=1.17 ./... and have it test with the latest 1.17 release (regardless of what my "real" go version is). But the key is I'd want to specify on the command line, not in go.mod.
Another point that I forgot to make in the doc is that setting the Go toolchain in go.mod means that all developers working on a project will agree on the toolchain version, without other special arrangements.
I find this to conflict with the idea from the original proposal that it's a minimum version; if my project says "go 1.13" because that's the minimum, I very likely do not want to use that version of Go for development. A newer toolchain will be faster and behave better when called by tooling like gopls or other analyzers (regardless of the version of Go that gopls is compiled with). Or, I will definitely want to publish binaries using the absolute latest version of Go possible. For example, esbuild says "go 1.13", but the actual binaries published to npm are from Go 1.19. If 1.13 this were to be the expected development version, that wouldn't be very optimal.
An environment variable GOTOOLCHAIN would control this new behavior. The default, GOTOOLCHAIN=auto, would use the information in go.mod. Setting GOTOOLCHAIN to something else would override the go.mod. GOTOOLCHAIN=local would force use of the locally installed toolchain, and other values would choose specific releases. For example, to test the package in the current directory with Go 1.17.2:
GOTOOLCHAIN=go1.17.2 go test
To be honest I am fine with: go install github.org/dl/go1.17.2@latest.
If you have a module that says go 1.12 [...] Cloud Native Buildpacks will always build your code with Go 1.12, even if much newer releases of Go exist. [...] The GitHub Action setup-go [...] has the same problems that Cloud Native Buildpacks do.
I feel like this proposal also doesn't really address this problem. To me, it sounds like both of these features are fundamentally mis-designed. If I am publishing a library on GitHub, then I should be specifying a range of minor versions (e.g, 1.17+), and then it tests with the latest patch release of each of them as part of my merge gate. Since the go.mod file cannot be relied upon to denote the lower or the upper bound, it really has no bearing on this, except possibly as the default lower bound. Separately, if I am publishing an actual binary as a release artifact, then I should be specifying the Go version (for build reproducibility), but the go.mod should not be considered at all.
One final feature of treating the go version this way is that it would provide a way to fix for loop scoping, as discussed in discussion #56010. If we make that change, older Go toolchains must not assume that they can compile newer Go code successfully just because there are no compiler errors.
Existing versions of the Go compiler already do not fail just because the go directive is newer.
if my project says "go 1.13" because that's the minimum, I very likely do not want to use that version of Go for development. A newer toolchain will be faster and behave better when called by tooling like gopls or other analyzers (regardless of the version of Go that gopls is compiled with).
I don't think this is a concern here. This proposal does not say that newer Go versions should download an older Go toolchain. It says that older Go versions should download a newer Go toolchain. When a newer Go version sees an older "go" line in go.mod, it will emulate the language features of the older Go language (that is already true today).
don't think this is a concern here. This proposal does not say that newer Go versions should download an older Go toolchain.
I agree; that was my original interpretation of the proposal. It just seemed like the followups implied otherwise. (That is how pinning tends to work in other languages like node.)
Change https://go.dev/cl/450916 mentions this issue: cmd/go: draft of forward compatibility work
As others here have explained, this proposal would cause a lot of problems for CI/CD, for package management, for conditionally using newer features in older codebases with conditional compilation, ...
The version of go in go.mod is now a minimum, which is fine. Rather than a magic environment variable, I would add a new command, go upgrade, that can upgrade the go compiler if installed locally from the official packages, but gives a helpful message if one should upgrade using the package manager in stead.
x/tools/gopls only tests that far back because they want to build with what's on people's machines. If what's on people's machines knew how to fetch a newer toolchain then we'd have stopped needing to support older versions long ago.
FWIW I only mentioned x/tools as it's an example of one the Go team maintains that was at the top of my mind. I don't find this particular point to be convincing as while this may be true for gopls specifically as it's an executable tool that users need to run somehow, most other examples are libraries (which are of course going to be built with "what's on people's machines"). If everyone is bumping the go directive in go.mod to gain access to new features, they still may be supporting older versions of Go (especially if they are following Go's own supported version policy).
This also is not exclusive to language features; I recall a series of CLs updating the x repos for lazy loading (#36460), for example CL 316111, which was only possible by bumping the version directive. All of the CLs in that series said:
Note that this does not prevent users with earlier go versions from successfully building packages from this module.
Which to me very much supports the idea that go directive is not a minimum version at all, but a feature unlocker.
(I apparently forgot to actually submit this reply days ago, oops.)
This seems like a bad idea to me.
Other text file formats include a line identifying the tool version that created the file, and do not imply that older versions cannot process the file properly. So the current behavior is not without precedent. (Examples of these other formats aren't springing to mind, but it wasn't a novel concept when I first saw it in go.mod).
This change assumes that upgrading the toolchain will always be beneficial and regressions will never happen. That stance doesn't seem much different to me than that of a certain OS, where users can be forced to reboot for an update - with no possibility to defer or skip the update. With a language rather than an OS, this behavior may be less infuriating and less disruptive for most users, but I think it's still unacceptable.
If a change is to be made, I would instead deprecate the go keyword and replace it with two, such as recommended_go and minimum_go. The former would have the same behavior as the go keyword currently has, while the latter would cause the toolchain to exit with error. Neither of these keywords would cause silent upgrades, and the names should be less likely to mislead.
If I understand correctly, the minimum_go keyword's behavior should allow for the for loop scoping fix mentioned in the description, as it'd put a lower bound on the toolchain version in use.
@mark-pictor-csec I think you just described the actual proposal.
minimum_go is spelled 'go'. It sets the minimum Go toolchain version that must be used to compile the module, whether used directly (via git clone + cd into it + run go commands) or indirectly (as a module dependency).
recommended_go is spelled 'toolchain'. It only applies when code is used directly by cd'ing into the module. It is ignored when the module is used indirectly as a required dependency.
It is not true that the proposal "assumes that upgrading the toolchain will always be beneficial and regressions will never happen." In fact I have gone out of my way in the past to explain why that's false. See for example https://research.swtch.com/vgo-mvs. Instead, the proposal, like Go modules as a whole, puts the user in control of which toolchain they use. There are no gremlins going around updating the Go toolchain behind your back. You only get a new go toolchain when you explicitly make a change to your go.mod file to indicate that. This is the same behavior as other modules: you keep getting the one listed in go.mod, even if there are newer ones, until you explicitly upgrade.
Re: potential confusion
A few people, including @ianlancetaylor, mentioned potential confusion. That potential definitely exists (indeed, many people on this issue are confused about the details, which suggests I did not present this well enough). We have to make it easy for people to understand what toolchain they are using and why. The way to do that has not changed: run go version and it will tell you.
There is only one time when go version does not help you answer the question, and that is when you are using go install path@version, which does not use the local go.mod. In this case, if the go command does not use its bundled toolchain, we should print a message, like:
$ go install path@version
go: installing path@version using go1.25.1
$
That message combined with the ability to run go version should take care of understanding what is happening. It does not directly address understanding why. For that, we need to document the rules clearly. Perhaps go help version would be a good place to document that. I would write something like:
The Go distribution consists of a go command and a bundled Go toolchain, meaning the standard library as well as the compiler, assembler, and other tools. The go command can use that bundled Go toolchain as well as other versions that it downloads as needed. Running “go version” prints the version of the bundled Go toolchain that is being used, which depends on both the GOTOOLCHAIN environment variable and the go.mod file for the current work module.
These are the rules the go command follows to decide which Go toolchain to use. The rules are listed in priority order, from highest to lowest: the first matching rule wins.
- If GOTOOLCHAIN=local, then the go command uses its bundled toolchain.
- If GOTOOLCHAIN is set to a Go version, such as GOTOOLCHAIN=go1.23.4, then the go command always uses that specific version of Go.
- If GOTOOLCHAIN=auto, then the go command consults the work module's go.mod file.
- If GOTOOLCHAIN is unset, it defaults to auto in release builds and to local when building from Go's development branches.
- If the go.mod file contains a “toolchain” line, the go command uses the indicated Go toolchain. The line “toolchain local” means to use the go command's bundled toolchain. Otherwise the “toolchain” line must name a Go version, like “toolchain go1.23.4”, and the go command uses that specific version.
- If the go.mod file contains a “go” line, the toolchain decision depends on whether that line names a version of Go older or newer than the bundled toolchain. If the named Go version is newer than the bundled toolchain, the go command uses that specific newer version. Otherwise, the go command uses its bundled toolchain, emulating the old toolchain version as needed.
- Finally, if GOTOOLCHAIN is unset and the go.mod has no “toolchain” or “go” line, the go command uses its bundled toolchain.
Re: losing control over which Go toolchain is used
@zikaeroh raised a concern about whether Linux distributions would “lose control of the toolchain in use”, and @dominikh wrote that he thought “some Linux distributions will patch their Go packages to disable the automatic upgrading.” @ianlancetaylor said something similar to me directly. @zikaeroh seemed to imply that even Homebrew/Chocolatey/etc might apply such a patch. I of course cannot predict what these systems will do, but I would encourage them not to start deleting or disabling features in the software they package. I expect toolchain version selection to become a standard part of the Go developer workflow, and it would be a shame for these packagers to make it hard for Go users to get their work done. In the end, it's easy enough to override with go env -w GOTOOLCHAIN=auto (unless the code is removed entirely!), but it would still be best for Go to be the same out of the box for all users, no matter which box it comes out of.
I used this framing in my previous comment:
The Go distribution consists of a go command and a bundled Go toolchain, meaning the standard library as well as the compiler, assembler, and other tools. The go command can use that bundled Go toolchain as well as other versions that it downloads as needed.
I think that framing is useful when thinking about Linux distributions and other packagers too. Just as the go command can fetch specific code dependencies for use in a build, it would now be able to fetch specific toolchain dependencies for use in a build.
To address what might be the concerns, being able to fetch specific toolchain dependencies for a specific build, does not overwrite the existing bundled toolchain provided by the operating system. Nor does it change what version of Go you get when you create an empty directory, run go mod init, and start hacking: you get the bundled toolchain provided by the packager. But if a user explicitly indicates that they want to build source code that is too new for the bundled toolchain (for example, the go.mod explicitly says go 1.25 and the bundled toolchain is only go 1.21), then instead of a build failure, the go command fetches and uses an appropriate newer toolchain. Similarly, if a user explicitly indicates that they want a specific toolchain by setting GOTOOLCHAIN=go1.19.2 or writing toolchain go1.19.2 in go.mod, the go command will fetch and use that toolchain as requested.
Linux distributions packaging commands written in Go already have to decide how to handle those commands' module dependencies:
- One option is to trust in Go's high-fidelity, reproducible builds and let the go command fetch the dependencies directly. I would hope that systems that take this approach are also comfortable letting the go command fetch any toolchain dependency as well, since the toolchain fetches have the same high-fidelity, reproducible behavior as module dependency fetches.
- The other option is to insist on managing and providing all the source code that goes into the build. This is a completely understandable choice for a packager, and I assume they have built up mechanisms for adjusting the build environment to do that. For example perhaps they set GOPROXY=off to ensure there are no downloads and they supply an appropriate go.work or go.mod filled with replace statements to provide all the dependencies. Even packagers with those custom builds don't default GOPROXY to off for ordinary use by users, at least as far as I know. They simply run their own builds in a non-standard environment. I would hope that systems that take this approach would continue to customize their own build environment but not leak those details into the packaging of Go for end users. It is also worth pointing out that setting GOPROXY=off to disable module dependency fetches will also disable toolchain dependency fetches, since toolchains are treated as modules.
In either case, I don't think the decisions that distributions packaging Go commands make for their own builds should leak into their own packaging of Go itself. Setting a default GOTOOLCHAIN=local in a packaged Go distribution would make builds break when the go line is too new, and it would also make the go command ignore any toolchain line in the current work module's go.mod. Both of these would be confusing to Go users, who will expect the behavior explained in Go's own documentation as well as any new books about Go that are written.
As I've noted before, it wouldn't really make sense to me for a distribution to default GOTOOLCHAIN=local unless they are also defaulting GOPROXY=off and also disallowing packages like rustup and nvm, both of which do the same thing that this proposal would have the go command do: manage a collection of toolchains and run the version specified by the user's configuration for a specific build.
I am only speculating at the concerns of the Linux and other packagers. I would be happy to hear from them directly about any concerns and work with them to address those in a way that works for them and for Go. On the Linux side, perhaps @stapelberg has some thoughts? And if anyone working on Homebrew, Chocolatey, or others would be inclined to default GOTOOLCHAIN=local in those packagers, could you please get in touch? Commenting here is fine, or rsc@golang.org. Thanks.
minimum_go is spelled 'go'. It sets the minimum Go toolchain version that must be used to compile the module, whether used directly (via git clone + cd into it + run go commands) or indirectly (as a module dependency).
This is the part that I still fundamentally disagree with; there are many examples listed where this directive is being used to allow newer features in a module, without restricting it to compile with that version or higher, be it language features or go.mod/go.sum changes themselves.
If this proposal doesn't actually enforce this minimum and doesn't make it difficult to use typical CI tooling like setup-go, then that's fine; a new toolchain directive is certainly interesting. But my understanding based on phrases like:
for example, the go.mod explicitly says go 1.25 and the bundled toolchain is only go 1.21), then instead of a build failure, the go command fetches and uses an appropriate newer toolchain
Is that this minimum is being enforced, and I believe that to be undesirable in too many cases to make it feel like a benefit overall.
@zikaeroh seemed to imply that even Homebrew/Chocolatey/etc might apply such a patch. I of course cannot predict what these systems will do, but I would encourage them not to start deleting or disabling features in the software they package.
I'll retract this specific bit; these all use the binary releases of Go. E.g. homebrew, chocolatey, scoop, winget
It's the source-using packagers that I am mainly concerned about, i.e. basically all Linux distros, and Nix (which can be run on any distro, even macOS).
As I've noted before, it wouldn't really make sense to me for a distribution to default GOTOOLCHAIN=local unless they are also defaulting GOPROXY=off
Personally, I don't find this to be a good comparison; setting GOPROXY has effectively no impact on the build besides its performance. I never have to worry about the value of the proxy because no matter how I set it, I'm going to get the same result. The only concern would be where my traffic goes, which is a different discussion.1
The toolchain is another story; this value can significantly affect my compile by downloading a large toolchain and using it, without my interaction or knowledge. It's a behavior that no other language has, without user involvement to either change the toolchain or install a software which does this explicitly.
Footnotes
@zikaeroh, I hear you about the change around "minimum" semantics. I am writing replies for one topic at a time and haven't gotten to that one yet.
Personally, I don't find this to be a good comparison; setting GOPROXY has effectively no impact on the build besides its performance.
This is true of GOPROXY=direct, but I said GOPROXY=off. The former means just use direct connections to the original source hosts. The latter disables downloading code from anywhere on the internet.
I believe that there are distros like Debian (maybe Fedora too?) who insist on packaging the source of all Go code into GOPATH and (seemingly) bypass the module system entirely.
I can believe they do this for packaging commands written in Go as Debian packages. I'm happy for them to do whatever is appropriate for their use cases there. I'm concerned here instead with what defaults users get for their own Go development. My understanding is that the proxy remains enabled in that case.
This is true of GOPROXY=direct, but I said GOPROXY=off. The former means just use direct connections to the original source hosts. The latter disables downloading code from anywhere on the internet.
You're right, my mistake; my brain clearly wasn't functioning at this hour.
I can believe they do this for packaging commands written in Go as Debian packages. I'm happy for them to do whatever is appropriate for their use cases there. I'm concerned here instead with what defaults users get for their own Go development. My understanding is that the proxy remains enabled in that case.
Certainly, yes. I meant that to be an aside as to not make it the point of the comment.
@zikaeroh I am also confused about
The toolchain is another story; this value can significantly affect my compile by downloading a large toolchain and using it, without my interaction or knowledge.
What does "without my interaction or knowledge" mean? You only get a new toolchain if you explicitly ask for it by editing a go.mod or setting GOTOOLCHAIN. The only way this could happen without your knowledge is if you cd'ed into someone else's module after a git checkout and ran a go command. Understanding that when you work on someone else's project you may use a different toolchain is a mental model shift that we will have to make clear to users, but collaborating on a project is already a much closer relationship than most code use. I mentioned earlier how go install path@version may use a different toolchain, but it will inform you of that. If I'm misunderstanding you, can you give a specific sequence of commands that would trigger a toolchain fetch without your interaction or knowledge? Thanks.
It's a behavior that no other language has, without user involvement to either change the toolchain or install a software which does this explicitly.
This is a bit of a philosophical disagreement, I think. Go has never been held back by limiting itself to behaviors that no other language has. We wouldn't have a go command at all if we did that. We've always focused on the full Go environment not just a language compiler/interpreter.
For example, as a very loose analogy, Go has never aimed to ship /usr/bin/python but for a different language. It has always been more like python+pip+pytest+... instead. It's a full environment. We could just as easily say that no other language can download dependency modules without user involvement or installing software specifically for that task (pip, cargo, etc). We put that functionality in the main Go toolchain precisely because it is an critical part of a full environment. Managing the Go toolchain version as well is a little like adding +virtualenv to that list, although much of what virtualenv does is already taken care of by modules and static linking, so maybe it should be there already. Like I said, the analogy is only very loose. Perhaps a better analogy is that Go is more like rustc+cargo than just rustc, and with this proposal it becomes more like rustc+cargo+rustup. In any event, managing the toolchain version is another critical part of a full environment, and it doesn't bother me at all - in fact I think it is a good thing - for Go to include this functionality out of the box rather than expect people to discover and install a separate tool.
Re: whether its easy enough to manage multiple toolchain versions today
A few people have mentioned that they don't believe it needs to be more convenient to manage multiple Go toolchains. For example @mateusz834 wrote “To be honest I am fine with: go install github.org/dl/go1.17.2@latest”, and @beoran suggested perhaps we need a “go upgrade”. I don't believe those are easy enough.
There's a popular DevOps meme (in the old sense) about whether servers are treated like pets or cattle. There's a full writeup and history here but this is the key point:
In the old way of doing things, we treat our servers like pets, for example Bob the mail server. If Bob goes down, it’s all hands on deck. The CEO can’t get his email and it’s the end of the world. In the new way, servers are numbered, like cattle in a herd. For example, www001 to www100. When one server goes down, it’s taken out back, shot, and replaced on the line.
In the pets mode, people invest manual effort in maintaining each instance of the thing. In the cattle mode, at least the right way, people spend time on automating the maintenance and then it runs itself. This scales far better and it fundamentally changes what is possible in the system.
In the early days of GOPATH, every dependency was a pet. There wasn't even go get. You lovingly went out in search of a new dependency, brought it home, gave it a name, and got it settled in just the right place for its new life with in your file system. Introducing goinstall (which became go get) was a step away from that model: it took care of downloading the dependency, giving it its name (import path), and storing it in the right directory in your file system. Dependencies at that point were on their way to being cattle, but dependency versions were still pets. You had to go around to each dependency and adjust each to be exactly the version you wanted. And all builds on your system used the versions you had carefully selected, which were probably a different set from what other people had. Tools like godep and the line of tools that followed, eventually culminating in Go modules, moved versions from being pets to cattle. Now it's all automated, and no one has to go around making sure that the dependency versions on one particular machine match all the other machines being used for that project. And when you switch between two different projects, the right dependency versions get pulled into your builds automatically in each case. For example, as I write this, Kubernetes master is using github.com/emicklei/go-restful/v3@v3.9.0 while Istio master is using github.com/emicklei/go-restful/v3@v3.8.0. If I work in one project in the morning and the other in the afternoon, my go command automatically supplies the version appropriate to the project I'm working on at the moment. That even works if I'm running tests on both in different directories simultaneously.
We no longer have to think about which version of go-restful we have installed on each machine we use, but we still do have to think about which version of Go is installed. The Go toolchain is still a pet, one that requires ongoing maintenance on every machine where it is installed. It is great that we have golang.org/dl/go1.19.2, but that only gets me a command named go1.19.2. If I am working in two different projects that have two different standard Go toolchains they want developers to use, then I have to manage that myself, such as by changing my PATH when I switch from one to the other. And the idea of a “go upgrade” that upgraded “the” Go toolchain still assumes a single pet toolchain. This proposal is about automatic management of a herd of them instead.
As @zikaeroh pointed out, compatibility is another important problem for helping people upgrade to newer Go versions, and we are working on that separately (in #56986). But I can say that from my own experience that just the bother of needing to do the update is enough that many of my machines have very old Go versions installed. If I could just get the right one based on what my go.mod says, I'd be much happier. That's one less pet for me to take care of. This is also exactly why Cloud Native Buildpacks the setup-go action use the go line for toolchain selection.
I do appreciate that for some people, managing “the” Go toolchain on their machine is no big deal. For some people managing the specific checked out versions in GOPATH was also no big deal. These things tend not to be when they are small. But at scale it gets harder, and we are starting to see the scaling issues. Not everyone, but at least some people. In the discussion, for example, Josh mentioned that he uses a script that picks the Go version based on his go.mod. We should address the scaling issue for everyone, so that people don't have to build their own bespoke solutions. If people want to keep managing “the” Go toolchain on their machine (including forcing GOTOOLCHAIN=local if they are so inclined), that's entirely reasonable. But they shouldn't have to.
If I'm misunderstanding you, can you give a specific sequence of commands that would trigger a toolchain fetch without your interaction or knowledge? Thanks.
I often clone other peoples' projects or switch between projects of my own. If I open any of those codebases, I may have to download an entire Go distribution before being able to do anything. That to me is very surprising. Unlike a typical Go module, the Go distribution is some 150 MB. If everyone uses the toolchain directive to specify which version is intended for development, that feels like a lot of versions of Go to download (not everyone has incredible internet) and cache locally, when it seems like I have always been fine using the absolute latest version of Go (something that I believe #56986 is intending to improve even further in that it addresses the question "how can we keep code working as intended when compiling with new versions of Go?").
This is a bit of a philosophical disagreement, I think. Go has never been held back by limiting itself to behaviors that no other language has. We wouldn't have a go command at all if we did that. We've always focused on the full Go environment not just a language compiler/interpreter.
I figured you'd say that, and I agree that this isn't a major criteria on the whole. My philosophy is just that if something "surprises" a user, that surprise should be positive. Modules are "surprising", but in the sense of "wow, I didn't have to npm ci when I changed branches, it just works!", an incredibly refreshing surprise and a major selling point I use when I talk about Go. The same applies to the module proxy, to testing, benchmarking, fuzzing, profiling, and so on.
But I think that the surprise provided by this proposal is not positive. I feel like I'm going to be surprised if I have to wait for a 150 MB download to start coding, end up with a huge module cache because many versions of Go itself are sitting there, and the potential to have accidentally compiled my project with an old version of Go because toolchain says that's what should be used (but my CI or Dockerfile was configured to use the latest). These to me feel like places that will annoy people at best, and really trip up someone in deployment or publish at worst.
@rsc
What will happen when i try to go install sth-remote@latest and sth-remote go.mod has a toolchain directive?
Is it going to download a toolchain and use it?
What will happen when i try to go install sth-remote@latest and sth-remote go.mod has a toolchain directive?
Is it going to download a toolchain and use it?
No. It will respect the go line but not the toolchain line (see #57001 (comment) for rationale). So if you have a new enough go already, it will use that. Otherwise it will use a newer toolchain and print a message to tell you. Actually it will print before it uses it. If using meant also downloading, you'd have an indication what was slow and could ^C.
I also think it is probably OK to print "go: downloading go1.19.2" the same way we print about go downloading modules today, so you'd get a download print in all cases where the toolchain is not already cached.
@zikaeroh thanks for the clarifications in #57001 (comment). I expect that most projects will use go without toolchain, in which case you should not end up with many different Go versions, especially if your pet toolchain is the newest possible one. We are also working on cutting the release down. Go 1.20 will be more like 95MB instead of 150MB to download, and we have ways to keep chipping away. We might be able to get as low as 50MB. That's still large but it's not enormous, especially compared to modern disks.
For projects using Docker and CI there has always been the opportunity for disagreement between your machine and CI. I hope those would migrate to respecting the toolchain line in go.mod (the usual "go mod download" layer caching trick would still work), precisely so that the disagreement goes away. On those kinds of projects where everyone cares about the specific toolchain being used, if Docker/CI use the toolchain in the go.mod and so do you, then disagreement between your machine and Docker/CI is a thing of the past. That's exactly one of the motivations for this change.
@mark-pictor-csec I think you just described the actual proposal.
Yes and no. The functionality provided by keywords may be the same (and apologies, I seem to have glossed over the toolchain functionality) but the names are different. My take is that go and toolchain impart less meaning than alternatives using words like minimum or recommended. Deprecating go and adding a new keyword with the same behavior but a more descriptive name serves to clarify what the keyword does (and doesn't) do.
If I missed out on this discussion and was confronted with a go.mod containing toolchain, I suspect that I'd assume it had the same meaning as go. I think the chances are quite low that I'd land on the correct meaning of the keyword or that I'd understand that go had a new meaning. Those sorts of false assumptions seem much less likely if words like minimum or recommended are in play :)
minimum_go is spelled 'go'. It sets the minimum Go toolchain version that must be used to compile the module, whether used directly (via git clone + cd into it + run go commands) or indirectly (as a module dependency).
recommended_go is spelled 'toolchain'. It only applies when code is used directly by cd'ing into the module. It is ignored when the module is used indirectly as a required dependency.
It is not true that the proposal "assumes that upgrading the toolchain will always be beneficial and regressions will never happen." In fact I have gone out of my way in the past to explain why that's false. See for example https://research.swtch.com/vgo-mvs.
Instead, the proposal, like Go modules as a whole, puts the user in control of which toolchain they use. There are no gremlins going around updating the Go toolchain behind your back. You only get a new go toolchain when you explicitly make a change to your go.mod file to indicate that.
I would argue that this user control is not always true. Consider: you are using an older (but still supported) go version. You have a module foo, importing someone else's bar; since the last time you updated the dependency, bar has been updated to require the latest version of go. If you update foo to use the latest version of bar, the go keyword will enforce a minimum version, so your current toolchain won't even try to build. Rather than exiting with an error, it sounds to me like the toolchain would download and run a new version. I strongly prefer being confronted with an error, rather than the tool (go) deciding to download and run a new version of itself for me.
This presupposes that the user has not throughly vetted the changes in bar and does not realize a newer version is required. That's a failing on the user's part, but it seems to me that downloading and running a new toolchain exacerbates their mistake.
Another issue I have with automatic toolchain downloads is that this is another link in the security chain. Hopefully never a particularly weak link, but no matter how strong it does give attackers more area to investigate.
A few years ago, I would've said the weaknesses I can imagine would require nation-state resources to exploit, but some ransomware groups have become quite rich. Stealing or forging certs may be within their reach, in which case an MitM attack becomes plausible: intercept module downloads, updating the required go version to require a new toolchain, and then intercept traffic to whatever server is providing the toolchain and provide a tampered version. Not a novel attack, but considerably easier if the download is automated. With a person in the loop, they have the opportunity to notice problems, for example by verifying checksums in a way that the attacker is unable to MitM or by wondering why a new toolchain release has broken the normal update cadence.
I would argue that this user control is not always true. Consider: you are using an older (but still supported) go version. You have a module foo, importing someone else's bar; since the last time you updated the dependency, bar has been updated to require the latest version of go. If you update foo to use the latest version of bar, the go keyword will enforce a minimum version, so your current toolchain won't even try to build. Rather than exiting with an error, it sounds to me like the toolchain would download and run a new version. I strongly prefer being confronted with an error, rather than the tool (go) deciding to download and run a new version of itself for me.
What happens is something in the middle. A few releases back we made go get only adjust requirements, not also run builds. So when you run go get foo, it will update the version of foo, bar, and go in your go.mod. But it will not build or run any of them. To do that you have to run a command like go build or go test or go install. So you have the opportunity, between go get and the next go command, to run git diff and see what has changed in the go.mod. And while I haven't thought it through completely, it also seems like it is probably okay for go get foo to print a message about updating the go line, so that you don't even have to reach for git diff.
Another issue I have with automatic toolchain downloads is that this is another link in the security chain. ...
The toolchain downloads would be protected in exactly the same way as the module downloads. Personally I trust the computer to check checksums more than I trust myself to do it. If an attacker can substitute a module download without your noticing, then they might as well just insert code into the packages you are building and running. Being able to tamper with the toolchain as well doesn't substantially change what is possible.
Re: go.mod go line as a minimum version or something else
The design doc did not call enough attention to the implications of changing the go line to a minimum Go version, bringing it into line with all the other versions in go.mod that are minimum versions.
There are two questions: can we change it, and should we change it?
Can we change it? @zikaeroh pointed out a tweet about the go version being a “feature unlocker”. That's a reasonable description of what it is today, but not many people actually understand that. We hear from Go users all the time that they are confused about or misunderstand what it means and what it does and doesn't do. And obviously the people who reused it for version selection in Cloud Buildpacks and the setup-go GitHub Action seem to have misunderstood it as well. So I don't think we are locked in to the current semantics. If we can identify better semantics and document them clearly (and even better if they align with what people expect when they see first see that line), then we have plenty of room to get ourselves there. This is something we can change.
Should we change it? This is the more difficult question. I think we should, precisely because people are so often confused about what it means. If I see require 5.99.0 in a Perl script or edition = "2024" in a Cargo.toml, I know that older Perl or Cargo are simply going to reject that code. They're not going to try to compile it and hope for the best, like Go does. In fact I can't think of any other language that ignores this kind of version requirement. So why does Go? That's not rhetorical. It's worth understanding, in the Chesterton fence sense.
The original implementation of the go line (in CL 125940) did specify a minimum requirement. When we started enforcing that in the compiler, so that go 1.X modules didn't accidentally use features from Go 1.(X+1), @ianlancetaylor pointed out that people might get go 1.X in their module because that's what they were running, unnecessarily cutting off Go 1.(X-1) users from the module even if it would happen to build just fine with Go 1.(X-1). The most relevant comment is #28221 (comment) (by me), specifically:
For a dependency asking for Go 1.(N+1) or later, when the current go version is Go 1.N, one option is to refuse to build. Another option is to try compiling as Go 1.N and see if the build succeeds. If we never redefine the meaning of existing syntax, then it should be safe to conclude from a successful build that everything is OK. And if the build fails, then it can give the error and say 'by the way, this was supposed to use a newer version of Go, so maybe that's the problem.' @ianlancetaylor has been advocating for this behavior, which would make as much code build as possible, even if people accidentally set the go version line too new. It sounds like this should work - in part because we've all agreed not to redefine the meaning of any existing syntax - so we should probably do it and only roll it back if there is a major problem.
That was in 2018. The costs and benefits of treating the go line as a strict minimum have shifted since then.
On the cost side, the only way in 2018 to implement the go line as a strict minimum was to refuse to build. That's a huge cost, and it would have forced lots more toolchain maintenance onto users. Most machines have a single toolchain installed globally for the entire machine, so updating it is a whole process and could break builds in unrelated projects. Or maybe you don't control the installed toolchain at all. Updating it is impossible. Or maybe it's just a big headache to deal with (see my earlier comment). Whatever the details, there's a high cost associated with having to switch toolchains, one that adopting “strict minimum” semantics would make users pay unnecessarily (at least in the cases where the listed version was not the true minimum version).
The proposal we are discussing here eliminates most of that cost. There would no longer be a single global toolchain for the machine, just like there's no longer a single global installed copy of any particular dependency package. The go command would manage multiple toolchains and select the right one for a given build the same way it does for modules. Now that its much easier to change toolchains, switching cost is no longer a reason to avoid strict minimum semantics.
On the benefits side, four years of not having strict minimum semantics have made clearer at least four benefits we are missing.
-
The first benefit is that the strict minimum semantics would be easier to understand. It's what users expect. That's why the tweet mentioned earlier said:
@golang community. Reminder that the
godirective in go.mod is a feature enabler for your current module and NOT a "minimum version of Go supported" indicator for your module's clients.We wouldn't have to write things like that, to tell people that their expectations are wrong, if the go line did what those people expect. (There would of course need to be clear documentation and communication about this meaning changing.)
-
The second benefit is that the strict minimum semantics provide the useful ability to say “you can't compile this code except with this version of Go or later.” The reported error can be made very clear, in contrast to the cryptic compile errors we get today (the design doc talks about these). If your program uses a new standard library package or function, it's much better for all your users if you can just write
go 1.20and rely on the toolchain to tell users they need to update, instead of making them first read errors about failed imports or unknown symbols and then mention the version mismatch. -
The third benefit is that the strict minimum semantics let us make changes to what programs mean and signal very clearly to older Go toolchains that they should not try to compile these programs. When we did the
//go:embedchanges, we had to require writingimport _ "embed"to use//go:embedon a string variable to break older toolchain builds. Otherwise, even though you have to writego 1.16(or later) to use//go:embed, Go 1.15 toolchains would ignore thego 1.16, compile the code, miss the//go:embedcomment, and write out a program without the embedded value. We made the experience worse (requiring the dummy import) because we didn't have strict minimum semantics.The same kind of thing can happen if you have written a package that only works properly with a bug fix included in a newer release. The best you can do to stop people from building with the older release is to insert an import or symbol reference to something unrelated that is new in that release. And there's nothing you can do if the bug is fixed in a point release. Later today we will release Go 1.19.4 with a bug that will keep you from using atomic.Pointer[T] to implement an atomic linked list. (Sorry. Wasn't caught in time and the train has left the station.) The compiler error if you try is inscrutable. That will get fixed in Go 1.19.5. With semantic minimums, if your code uses an atomic linked lists and therefore only works with Go 1.19.5 or later, you write
go 1.19.5in your go.mod and move on. Your users never see the weird errors that arise from Go 1.19.4.With things as they are today, there is nothing you can do to prevent those older builds and the confusion they will cause. This specific case of the atomic linked list is unlikely to arise, but similar things do happen. And there are bugs that don't result in compiler errors too, which is even worse for your users. Strict minimum semantics let you be sure no one is building your module with an incompatible toolchain and then decoding cryptic compiler errors or (worse) chasing down strange execution bugs that have already been fixed.
-
The fourth benefit is that strict minimum semantics let us fix semantic problems in the exceptional cases when that is appropriate. This will not arise often: perhaps just once. But that one is still compelling: we cannot safely fix loop scoping semantics unless we have some clear signal to older Go toolchains not to blindly compile code that depends on the newer semantics for correct execution. The most obvious, simplest, clearest such signal is to write
go 1.99(or whatever the right version is) in thego.modfile. If we don't have strict minimum semantics, I don't see how we fix loop scoping. I hope that loop scoping is the only time we will ever need this feature, but who knows? Perhaps another will arise after another 10 years of experience with Go. The quoted text above said "in part because we've all agreed not to redefine the meaning of any existing syntax". We've realized that was a mistake, at least in the exceptional case of loop scoping.
So compared to when we rolled back strict minimum semantics in 2018, we've found a way to remove most of the costs we were concerned about, and we now have a clearer understanding of which benefits we are missing out on. On balance, it seems to me that rolling strict minimum semantics forward again is a win.
There will be an effect on the ecosystem. We avoided strict minimum because we thought it would be too hard for users to upgrade. If we move to strict minimum because it is now easier for them to upgrade, yes, the effect will be quicker upgrades. I hope that most users won't think much about it at all. They'll just run 'go get newdependency', see the message about having a new Go version in this project, just like having a new indirect dependency, and continue on with their work.
@dominikh's comment was very insightful and is worth quoting in full:
I think the combination of auto-upgrading and not allowing old toolchains to build new code will change the Go ecosystem to either no longer support multiple versions of Go, or to delay experimenting with new language features.
Where previously a Go module could target Go 1.21 but make use of 1.22 features in optional, build-tagged files, it can no longer do so. To be allowed to use Go 1.22 language features, the go.mod has to specify go 1.22, but this triggers auto-upgrading / build failures on Go 1.21. The module now only supports Go 1.22.
Auto-uprading makes this seem fine at first: anyone who needs Go 1.22 will get it for free, automatically. However, I don't think auto-upgrading can be assumed to be pervasive. For one, most people who hack on Go at least occasionally will be using GOTOOLCHAIN=local, pointed at a git checkout. Furthermore, I predict that some Linux distributions will patch their Go packages to disable the automatic upgrading, as it side-steps their packages and some are allergic to that. This'll lead to users who are left behind even more than they're now due to slow-moving distributions.
Taking these paragraphs in reverse order, I think we can and should assume that Go toolchain management can be depended on and assumed to be pervasive. As I replied at length above, I think GOTOOLCHAIN=auto is perfectly compatible with Linux packager policies, and if packagers disagree then I'm happy to engage with them to understand that better and try to address their concerns. I expect that management of toolchain versions will become just as critical a part of Go as management of dependency versions, so we do need to make sure it's available to all our users. It is also important to emphasize (from above) that this proposal is not “auto-upgrading”, because there is no single pet toolchain to upgrade. (This is not like Chrome overwriting itself.)
It is true that a Go module will not be able to support Go 1.21 while also making limited use of Go 1.22 features using build tags. A different way to view this is that a Go module will not have to worry about that kind of complication anymore. If they want to use a Go 1.22 feature, they can easily use Go 1.22, confident that - due to the combination of this proposal and #56986 - users who want the latest version of the module can use Go 1.22 with that project instead of being “stuck” on Go 1.21. Yes, the module now only supports Go 1.22, just like the current Kubernetes only supports github.com/emicklei/go-restful/v3@v3.9.0. Users who want the latest of the module use the updated dependencies it requires.
I don't think it's strictly accurate to say the Go ecosystem wouldn't support multiple versions of Go. That's like saying the Go ecosystem doesn't support multiple versions of go-restful or go-yaml. Different modules will make different decisions about whether to start using the latest Go version. I do think it is accurate to say that it will not be as important for popular Go ecosystem packages to keep their latest module versions working with the last two Go toolchain versions, and that perhaps as a result more will choose to move forward more quickly than in the past. That's fine with me. The main reason these popular packages needed to support older Go versions was for people with environments where it was too costly or impossible to move to a newer Go version. This proposal and #56986 aim to remove those costs and impossibilities, which will make it less important for these packages to support older Go versions. I am sure better things can be done with the time currently dedicated to that support.
Overall, the comment is spot-on. We are explicitly aiming to remove barriers to updating to the newest Go version. That should in turn let the entire Go ecosystem adopt newer versions more quickly, which should be a win for everyone.
One final feature of treating the go version this way is that it would provide a way to fix for loop scoping, as discussed in discussion #56010. If we make that change, older Go toolchains must not assume that they can compile newer Go code successfully just because there are no compiler errors.
Existing versions of the Go compiler already do not fail just because the go directive is newer.
Sorry for the delay. It took me a while to realize what you were getting at. You are absolutely right that existing Go versions do not reject code with newer go lines. That's why we can't do the for loop scoping immediately. Instead, the best plan I have is to land the new go line interpretation in Go 1.X and then wait to land loop scoping until Go 1.(X+1). When Go 1.(X+1) comes out, Go 1.(X-1) and earlier will be officially unsupported, and Go 1.X will know enough to refuse to build Go 1.(X+1) code. So no supported Go version will miscompile new loop code.
We have to make it easy for people to understand what toolchain they are using and why. The way to do that has not changed: run go version and it will tell you.
@rsc If go version now reports the toolchain version that would be used for the current module, is there now no way to tell from the command line what the bundled Go version is?
Quick point while I contemplate if it's worth engaging on this topic as the Arch maintainer for the go package.
One option is to trust in Go's high-fidelity, reproducible builds and let the go command fetch the dependencies directly. I would hope that systems that take this approach are also comfortable letting the go command fetch any toolchain dependency as well, since the toolchain fetches have the same high-fidelity, reproducible behavior as module dependency fetches.
It's a very big difference between downloading sources files defined in the go.mod files and fetching binary files files from some remote location. We are all very aware of the trusting trust attack and moving the reproducible builds requirements from the downstream distributor (Linux distributions) to the upstream (Google) is not trivial.
So how is Google going to provide Reproducible Builds for the downloaded toolchains?
@willfaught GOTOOLCHAIN=local go version would do it.
@Foxboron, regarding "Reproducible Builds", by that do you mean https://reproducible-builds.org/? And if so what is involved in "providing" one? As of Go 1.21 we expect our toolchains will be fully reproducible even when cross-compiling. (That is, if you build a Mac toolchain on Windows, Linux, and Mac, you get the same bits out in all cases.) I would be delighted to have a non-Google project reproducing our builds in some way.
regarding "Reproducible Builds", by that do you mean https://reproducible-builds.org/?
Yes. I have been working on this project since 2017 for Arch Linux.
And if so what is involved in "providing" one?
If this gets implemented we would be downloading binary toolchains, right? I want to reproduce the binaries distributed by Google.
Just checking out the source and building versions won't necessarily be enough, so there needs to be some attestation or SBOMs published to support the distribution of the binaries.
I'm not saying this can't be done. I'm just trying to point how the bar between the "reproducible builds" Go already facilitates with source code is very different from what you would need to ensure for binary builds.
I would be delighted to have a non-Google project reproducing our builds in some way.
I'm not sure if "our builds" is the distributed binaries from Google? But Arch has been publishing verifiable builds of the Go compiler for 2 or 3 years now.
It is true that a Go module will not be able to support Go 1.21 while also making limited use of Go 1.22 features using build tags.
For a real-world example of where this change would be a problem, take the github.com/aws/aws-lambda-go library. It supports all the way back to Go 1.13. But recently, it started using generics in build-tagged files, which means that go.mod has to say go1.18. With this new policy, it would have to say go1.13 instead, meaning that it could not leverage generics at all (or any new language features), even for the majority case where people are using 1.18/1.19, without dropping support for 1.13-1.17 entirely. In short, while the Go authors only support the latest two minor versions, the rest of the Go ecosystem has been more forgiving.
That is also why I am quite leery of this statement:
Instead, the best plan I have is to land the new go line interpretation in Go 1.X and then wait to land loop scoping until Go 1.(X+1).
As an example, one of our teams tends to end up on officially unsupported versions because qualifying new releases is a large effort and doing it every six months is not feasible. It would be better if an explicit failure could be induced in older compilers (e.g., by adding a new directive to go.mod) instead of just assuming no one would use one.
I am generally in favor of this goals of this proposal because I think it does solve some real problems. But I also think it creates some new problems that need more thought. @rittneje Makes good points above. Mine are similar. Repeating some of the same quotes here for completeness...
It is true that a Go module will not be able to support Go 1.21 while also making limited use of Go 1.22 features using build tags. A different way to view this is that a Go module will not have to worry about that kind of complication anymore.
I do think it is accurate to say that it will not be as important for popular Go ecosystem packages to keep their latest module versions working with the last two Go toolchain versions, and that perhaps as a result more will choose to move forward more quickly than in the past. That's fine with me. The main reason these popular packages needed to support older Go versions was for people with environments where it was too costly or impossible to move to a newer Go version. This proposal and #56986 aim to remove those costs and impossibilities, which will make it less important for these packages to support older Go versions.
When Go 1.14 changed the implementation of runtime timers it broke a project I owned at the time. I reported that in #38860. As a result that project had to stay with Go 1.13 for the time being. I was able to contribute a fix for #38860 that made it into Go 1.16, but by that time Go 1.13 was already out of support for a release cycle. If Go already had the proposed behavior and the ecosystem was moving forward on Go versions faster than it did back then we may have been unable to upgrade any dependencies that required Go 1.14 or Go 1.15 (transitively). That would have left us stuck between a rock and a hard place if any of our dependencies had security updates that also bumped the Go version, perhaps for unrelated reasons.
But also, the fix for Go #38860 introduced #44343 which has caused many other projects problems and remains unresolved today. There may be projects pinned to Go 1.15 now for that reason that are in a similarly bad situation that my old project was before.
Perhaps it is useful to consider how would we have navigated the runtime timer rewrite and resulting issues that came up as a result if this proposal and #56986 were in place from the beginning?
Also, would this proposal (if accepted) be binding on other implementations of Go (e.g. TinyGo, gccgo)?
- If yes, to what extent? Do they have to implement toolchain updating or is failing builds if any dependency requires a newer version of Go acceptable?
- If no, does that diminish any of the suggested benefits?
I think this proposal can be divided into two pieces.
- Should the
godirective be redefined as the minimum supported version?
The go directive has always represented the maximum supported version (essentially). For this reason, newer compilers must run in "compatibility mode", disabling newer language features that could constitute a breaking change. If a minimum version requirement is needed, that should be a separate directive. This preserves the existing semantics and use cases. For example, this means that compilers older than 1.17 should fail, but those newer than 1.18 must run in "1.18 compatibility mode".
go 1.18
require-go ">=1.17.0"
(This hypothetical directive is for example purposes only.)
- Should the go toolchain automatically upgrade itself?
In a CI/CD pipeline, I would expect people to use docker images as per current best practices. It would be surprising and undesirable if I pulled, say, golang:1.13 and then it decided to download 1.17. If 1.17 is desired, I should have pulled that image in the first place. Thus in this context, automatic upgrades are counterproductive.
If I am coordinating with other developers on an executable (as opposed to a library), then I want to be able to force all developers to use a specific version, regardless of whether their toolchain is older or newer. But this proposal does not address that, as only older compilers will be affected. Here again I think the hypothetical require-go directive helps. For example, this will reject 1.18.7 and 1.18.9.
go 1.18
require-go "==1.18.8"
While it could be convenient for the Go toolchain to automatically run a different toolchain automatically as per go.mod, I feel this does not get to the heart of the problem. We have developers that switch between two projects using different versions of Go. Because both binaries are simply named "go", they are forced to play games with PATH (or maybe symlinks, I haven't tried) to switch between the two. It would be great if the toolchain offered first-class support for this situation. But that exists independently of any automated installation, since I should be able to easily switch between custom installations too.
For a real-world example of where this change would be a problem, take the github.com/aws/aws-lambda-go library. It supports all the way back to Go 1.13. But recently, it started using generics in build-tagged files, which means that go.mod has to say go1.18. With this new policy, it would have to say go1.13 instead, meaning that it could not leverage generics at all (or any new language features), even for the majority case where people are using 1.18/1.19, without dropping support for 1.13-1.17 entirely. In short, while the Go authors only support the latest two minor versions, the rest of the Go ecosystem has been more forgiving.
Some of the ecosystem, perhaps - I don't think I'd say "the rest". We only test golang.org/x/... on the last two releases, for example. (You can see our build+test dashboard at build.golang.org; scroll down for the x repos.) Lots of the ecosystem depends on x/sys or x/net. My guess is that very little of the ecosystem still builds on Go 1.13. aws-lambda-go seems like an exception.
It's an interesting question though why it's important to support back to Go 1.13. Go only issues security fixes and critical stability fixes for the previous two releases. Anyone using Go 1.13 through Go 1.17 may well be hitting serious bugs that we found and fixed in later versions. The goal of this work is to make it easier for people to stay up to date. All other things being equal, we'd keep unsupported versions working as well as they do today, but if the benefits I outlined above, including making it easier for people to stay up to date, come at a cost to unsupported versions, that cost is not going to weigh heavily on the balance sheet.
And of course, making it easier to manage multiple toolchain versions and to upgrade should help anyone stuck on Go 1.13 move forward.
Instead, the best plan I have is to land the new go line interpretation in Go 1.X and then wait to land loop scoping until Go 1.(X+1).
As an example, one of our teams tends to end up on officially unsupported versions because qualifying new releases is a large effort and doing it every six months is not feasible. It would be better if an explicit failure could be induced in older compilers (e.g., by adding a new directive to go.mod) instead of just assuming no one would use one.
For exactly the same kind of backwards compatibility, unrecognized lines in dependency go.mod files are ignored. So we've painted ourselves into a bit of a corner where the best we can do is prepare K versions that recognize that they don't understand the new code and then issue the K+1'th version supporting the new code. The only thing we really have control over is the choice of K. In the quoted text I was assuming K=1. Maybe we should do K=2, but I wouldn't want to do more than that and I'm not even sure K=2 is worth it.
I completely sympathize about being on unsupported releases. Kubernetes was stuck on unsupported releases and #56986 is an attempt to address the problems that were keeping them there, along with I assume other large projects. If there are other kinds of reasons for being stuck on old releases, I'd love to hear about them (maybe file a separate issue and just link it here, or email me rsc@golang.org) so we can think about how to address those. That said, part of being on an unsupported release is having to do some things for yourself since we're not supporting them anymore. That can include backporting security fixes or in this case being sure to avoid pulling in code that is too new. One way would be to set up a test based on
go list -f '{{if .Module}}{{.Module.GoVersion}} {{.ImportPath}}{{end}}' all
and another way would be to apply some variant of the diff below to the old go command when the time comes.
diff --git a/src/cmd/go/internal/work/exec.go b/src/cmd/go/internal/work/exec.go
index d6fa847be04..077e0fac808 100644
--- a/src/cmd/go/internal/work/exec.go
+++ b/src/cmd/go/internal/work/exec.go
@@ -525,6 +525,10 @@ func (b *Builder) build(ctx context.Context, a *Action) (err error) {
return errors.New("binary-only packages are no longer supported")
}
+ if p.Module != nil && !allowedVersion(p.Module.GoVersion) {
+ return errors.New("module requires Go " + p.Module.GoVersion)
+ }
+
if err := b.Mkdir(a.Objdir); err != nil {
return err
}
I'm always happy to suggest paths forward for things like this, but one key part of "unsupported" is that we don't hold back new work due to unsupported versions. As I noted earlier, we'd rather put our effort into fixing the reasons people can't keep up, so if you see more we can do on that front, please let me know.
@ChrisHines, thanks for the real-world examples and details. Those are always incredibly helpful.
When Go 1.14 changed the implementation of runtime timers it broke a project I owned at the time. I reported that in #38860. As a result that project had to stay with Go 1.13 for the time being. I was able to contribute a fix for #38860 that made it into Go 1.16, but by that time Go 1.13 was already out of support for a release cycle. If Go already had the proposed behavior and the ecosystem was moving forward on Go versions faster than it did back then we may have been unable to upgrade any dependencies that required Go 1.14 or Go 1.15 (transitively). That would have left us stuck between a rock and a hard place if any of our dependencies had security updates that also bumped the Go version, perhaps for unrelated reasons.
But also, the fix for Go #38860 introduced #44343 which has caused many other projects problems and remains unresolved today. There may be projects pinned to Go 1.15 now for that reason that are in a similarly bad situation that my old project was before.
Perhaps it is useful to consider how would we have navigated the runtime timer rewrite and resulting issues that came up as a result if this proposal and #56986 were in place from the beginning?
Thanks for your work on these issues. Timers are always a tricky balance between efficiency and latency, and Windows timers especially so, unfortunately. If the ecosystem does start moving forward at something more like the actual Go support policy, then I expect that would create pressure to land fixes for these kinds of problems more quickly, instead of letting them linger. Overall that would be a good thing for Go.
A very brief skim of your CL 232298 looks like it would have been possible - not trivial, but not too bad - to flag that code with a GODEBUG as well. Perhaps we still should until we have a more complete fix. That way at least people can update with a //go:debug line or an older go line and get the other important fixes. Another example that lingered too long was asyncpreempt. We did introduce a GODEBUG, but we should probably have defaulted it to off quickly after the reports started coming in, to let people update.
I hope that #56986 will be part of a shift in our focus and response to these kinds of incompatibilities. In day-to-day development we now have a fairly strong default of "if the build breaks, roll it back first and debug later", so that we unbreak everyone else working on unrelated things. I am trying to cultivate a similar strong default of "if a program breaks, add a GODEBUG first" too. It can default to the new behavior in new releases, but it needs to default to the old behavior in old releases, so that updating the toolchain (without updating the go line) is as much a non-event as possible. Another related change would be establishing a similar pattern in vet, so that updating the toolchain (but not the go line) and running go test is as much a non-event as possible too. I filed #57063 for that. There may be more we will discover. We are only at the start of exploring what this kind of emphasis on practical compatibility based on the go line means.
Also, would this proposal (if accepted) be binding on other implementations of Go (e.g. TinyGo, gccgo)?
If yes, to what extent? Do they have to implement toolchain updating or is failing builds if any dependency requires a newer version of Go acceptable?
If no, does that diminish any of the suggested benefits?
The important semantic change for these systems is not trying to build code with a newer version than they know how to provide semantics for. They need that to provide the benefits enumerated in #57001 (comment). Without it, they will try to compile new code with old compilers and may or may not get working programs out the other side, even if the compilation succeeds. So knowing when not to run a build is necessary to keep providing a working Go implementation.
Managing multiple toolchains is not strictly necessary. It helps remove some of the pain of not building newer code with older toolchains, but it's not the same level of requirement. If gccgo or TinyGo chose not to implement that part, it would be like they hard-coded GOTOOLCHAIN=local with no possible override in the environment or a go.mod file. That would not be ideal, but there are already other ways in which both of them provide a different Go experience; adding this one would not be a huge deal. That said, one reason the toolchain line includes the go prefix is to allow the future possibility of toolchain gccgo1.2.3 or toolchain tinygo1.2.3.
The reason they use older releases is they are using Go on an embedded system and thus have to do extensive testing to verify the new version. As that is a lot of effort, it cannot happen every six months, so inevitably the version lags. I don't think there's anything to be done on this front, unless you would be willing to change the policy to support the last 3 releases instead of just the last 2.
It's an interesting question though why it's important to support back to Go 1.13.
Regardless of the particular version, it is very common for libraries to support versions that the Go authors do not, in order to not prescribe anything on their users. (For the same reason, for better or for worse some libraries I have used keep themselves on older versions of their module dependencies.) And even if they did choose to support only the officially supported releases, this proposal still causes problems. With this change, no library would have been able to use generics at all until after 1.19 got released, or else they would have dropped support for 1.17 while it was still officially supported. The same will be true of every new language feature that gets released.
I'm also quite frankly confused. While the issue title purports to extend forwards compatibility, it actually does the complete opposite, and then masks it behind auto-upgrades. Based on all the discussion, the goal is to break forwards compatibility explicitly (i.e., in a way that the old toolchain will notice and complain about). To that end, I stand by my earlier statement that a new go.mod directive is the best path forward, as it easily enables people to say what they actually mean, without invalidating existing use cases.
Should the go directive be redefined as the minimum supported version?
The go directive has always represented the maximum supported version (essentially). For this reason, newer compilers must run in "compatibility mode", disabling newer language features that could constitute a breaking change.
This is oversimplifying the current state of the world. The go directive is really neither a minimum nor a maximum. If the go line says go 1.15, you can certainly build it with newer Go versions, and that will remain true. And you can use standard library features from versions newer than Go 1.15, provided they are properly build tagged. It's only language features that are capped at the go line. I do see what you're saying, but the situation is not as clear-cut as you make it out. It's very confusing, and preserving the status quo means preserving that confusion.
I am not a fan of 'require-go >=1.17.0' because when you combine those from multiple modules you end up with version-sat problems. Avoiding those was one of the design requirements of Go modules.
Should the go toolchain automatically upgrade itself?
In a CI/CD pipeline, I would expect people to use docker images as per current best practices. It would be surprising and undesirable if I pulled, say, golang:1.13 and then it decided to download 1.17. If 1.17 is desired, I should have pulled that image in the first place. Thus in this context, automatic upgrades are counterproductive.
I'm not convinced that is true. Why should there be two places where the target Go toolchain is recorded (both the go.mod and the Dockerfile)? The go.mod is required; the Dockerfile is not. Therefore it should be in the go.mod, and the Dockerfile should use the go.mod to decide. Perhaps someone will write a lightweight Dockerfile that does exactly that. Or perhaps it is enough to use the go mod download caching trick. All that said, I do understand that some users will want to proceed more cautiously in CI/CD, and that's one of the cases where I expect GOTOOLCHAIN=local to be the most useful.
If I am coordinating with other developers on an executable (as opposed to a library), then I want to be able to force all developers to use a specific version, regardless of whether their toolchain is older or newer. But this proposal does not address that, as only older compilers will be affected.
This proposal does address that. If you write toolchain go1.23.4 in the go.mod, then that's what the builds will use, unless the user overrides it by setting GOTOOLCHAIN in their environment. The toolchain line applies regardless of whether the local toolchain is older or newer than that line. If it's not exactly the same, a different toolchain is used.
The line that only affects older compilers is the go line. The expectation is that the go line will be the main one people use, especially since it is respected in dependencies and has a clear way to merge different requirements (take the maximum listed version, just like MVS on module versions). But if you are coordinating with other developers on a closed system, then you want the toolchain line, to make sure you are all in sync.
The precise rules are in the quoted text in #57001 (comment).
While it could be convenient for the Go toolchain to automatically run a different toolchain automatically as per go.mod, I feel this does not get to the heart of the problem. We have developers that switch between two projects using different versions of Go. Because both binaries are simply named "go", they are forced to play games with PATH (or maybe symlinks, I haven't tried) to switch between the two. It would be great if the toolchain offered first-class support for this situation. But that exists independently of any automated installation, since I should be able to easily switch between custom installations too.
Up until the last sentence, what you are describing is exactly a problem that this proposal does address. In the last sentence, the part about custom toolchains is a fair point. As I mentioned earlier, I do envision that you might be able to use toolchain tinygo1.2.3. It would be interesting to consider what we'd need to do to allow non-standard toolchain sources. I'm not opposed to that, but there are security aspects to work out. It is probably better to handle the standard case before we design the non-standard one. The syntax is there.
I'm not convinced that is true. Why should there be two places where the target Go toolchain is recorded (both the go.mod and the Dockerfile)? The go.mod is required; the Dockerfile is not.
If my CI/CD is for a library, then I would have to have some knowledge of the list of versions I want to test with. Then I would pull the image for each of them and test. By the time that I am running inside the container, I have already chosen the version, and so I would never expect it to then decide to download another version.
If my CI/CD is for an application, then I would have know the specific Go version somehow. Again, by the time that I am running inside the container, I have already chosen the version, and so I would never expect it to then decide to download another version.
I cannot think of a reason that I would ever have my CI/CD first pull the image of some arbitrary Go version, and then have it automatically pull a different version. That would be extremely slow and wasteful. Even if I did want some piece of info in go.mod to drive the version selection, I would first just parse it with grep or something, not download an entire toolchain.
The toolchain line applies regardless of whether the local toolchain is older or newer than that line.
Ah okay. When I first read the proposal I thought it only applied if the local toolchain was older.
I am not a fan of 'require-go >=1.17.0' because when you combine those from multiple modules you end up with version-sat problems.
Given what you said about the toolchain directive, I think the hypothetical directive could be simplified to something like require-go 1.17.0 and strictly be a lower bound. In practice I don't think people would use both directives together very often, but in theory you could.
It's very confusing, and preserving the status quo means preserving that confusion.
I think as long as the current use cases still work somehow, that would be nice. For example, maybe we say that if there is no require-go directive, it is implied to be the same as the go directive. But if you are in a situation like these libraries, then you can override that with an explicit require-go directive.
For example, this can use Go 1.20 language features, and will fail if compiled with < 1.20 (presumed majority case).
go 1.20
This can use Go 1.20 language features (conditionally), and will fail if compiled with < 1.18.
go 1.20
require-go 1.18Also, one thing I want to point out is that patch versions cannot just be ordered lexicographically. For example, if I say that the module requires Go 1.18.9+ because of some bugfix, it is unclear whether it should allow 1.19.0. This is an existing problem with modules, but will now be exacerbated.
It would be interesting to consider what we'd need to do to allow non-standard toolchain sources.
One thing I was pondering is whether the toolchain would should work as follows when the toolchain directive is set.
- If the current toolchain is the same version, then just use it.
- Else if there is an executable in the
PATHwith the same name, then use that. (This would need to account for .exe suffix on Windows.) - Else consult the module cache.
- Else prompt the user (or something).
This gives us a relatively simple mechanism to use the correct Go version by setting up a properly named symlink on developer machines.
@rittneje, I like the idea of looking in the path for the toolchain before fetching it as a module. That handles custom toolchains very nicely. Thanks.
On the point about allowing old Go toolchains to build a module with build-tagged newer code, I wonder if we really need a separate line for that. Perhaps it would be enough to make the go line a strict minimum as in the proposal but then also have the compiler treat //go:build go1.55 as enabling Go 1.55 features in that source file. The tag will be necessary anyway to stop older toolchains from building it.
So the idiom for supporting Go 1.X and later but optionally having Go 1.Y features would just be to write 'go 1.X' in the go.mod and '//go:build go1.Y' in the files that use the Go 1.Y features. That should be fairly clear.
The only wrinkle is that, supposing this lands in Go 1.21, to keep compatibility with unsupported versions, instead of "write 'go1.X'", the rule is "write 'go1.X' when X >= 20 and otherwise write 'go 1.20'." Since this only affects people trying to keep up with unsupported versions prior to Go 1.20 and it's not too complicated anyway, it seems an acceptable wrinkle.
Perhaps it is useful to consider how would we have navigated the runtime timer rewrite and resulting issues that came >> up as a result if this proposal and #56986 were in place from the beginning?
A very brief skim of your CL 232298 looks like it would have been possible - not trivial, but not too bad - to flag that code with a GODEBUG as well.
OK. I understand your response to mean that, assuming we had a GODEBUG setting for timer behaviors, the project I described would have been able to add something like //go:debug timers=1.13 in its main package and build with a Go 1.14 or 1.15 toolchain. Then when the the timers fix landed in Go 1.16 we could remove the //go:debug line. Meanwhile we would not be blocked from upgrading dependencies with go.mod files containing go 1.14 or go 1.15 directives that this proposal would otherwise prevent.
That seems like an overall better outcome from a project developer's perspective, as long as the necessary GODEBUG settings are available. In particular the project would have benefited from other improvements in Go 1.14 and 1.15 sooner and the concern about dependencies potentially requiring one of those versions would be reduced. 👍
On the point about allowing old Go toolchains to build a module with build-tagged newer code, I wonder if we really need a separate line for that. Perhaps it would be enough to make the go line a strict minimum as in the proposal but then also have the compiler treat //go:build go1.55 as enabling Go 1.55 features in that source file. The tag will be necessary anyway to stop older toolchains from building it.
Just to be clear, let's say this change around go.mod gets introduced in 1.21, and the loop scoping change gets introduced in 1.22. If my go.mod says go 1.21, then:
- files with no
//go:build go1.Xline, or with a//go:build go1.Xline where X ≤ 21 will use the old loop semantics regardless of the toolchain version - files with a
//go:build go1.Xline where X ≥ 22 will use the new loop semantics regardless of the toolchain version
In other words, it is possible for one module to use both loop semantics simultaneously across different files. Is that correct?
Assuming so, I like this approach because it also allows a module author to incrementally switch to 1.22 semantics file-by-file instead of it being all-or-nothing. Thus it will ease the migration. And once they are done, they can just change go.mod to say go 1.22 and remove all the redundant //go:build flags.
There is one other situation I wanted to make sure gets considered. On rare occasion, we have to manually patch the toolchain. When we do this, we also tack on some suffix to the real version that gets baked in (e.g., "1.18.9-1"). It would be helpful if this were officially supported as far as this proposal goes, in that such a version should not cause it to barf when checking against the go directive in go.mod, for instance. I'm not sure how exactly this feature will be implemented, but if it will be utilizing the same version string, it should chop off any such suffix when it goes to do comparisons. But it should not do this when checking the toolchain directive as that should be an exact match against the whole version.
@ChrisHines, yes exactly right. That kind of experience is supposed to become the standard one as part of the #56986 work.
@rittneje, yes exactly. I like that about the loop transition too.
There is one other situation I wanted to make sure gets considered. On rare occasion, we have to manually patch the toolchain. When we do this, we also tack on some suffix to the real version that gets baked in (e.g., "1.18.9-1"). It would be helpful if this were officially supported as far as this proposal goes, in that such a version should not cause it to barf when checking against the go directive in go.mod, for instance. I'm not sure how exactly this feature will be implemented, but if it will be utilizing the same version string, it should chop off any such suffix when it goes to do comparisons. But it should not do this when checking the toolchain directive as that should be an exact match against the whole version.
I am not sure about this one. If you want to use a patched toolchain, it seems like it would make more sense to use the toolchain line to select that, not the go line. I would prefer to keep the go line about the semantics. So for example if you want to test Go 1.25rc2, you would use
go 1.25
toolchain go1.25rc2
or even
go 1.23
toolchain go1.25rc2
not
go 1.25rc2
We do have a fair amount of flexibility in what the go line can hold today (at least in dependencies), but it's unclear how to do the min/max operation on custom strings like 'go 1.18.9-1'. I'd rather keep it to standard Go versions (excluding both custom versions by others and beta and rc versions from Go itself).
@rsc Sorry, I was unclear what I meant about patched toolchains. For example if go.mod says go 1.18.8 and my patched toolchain reports its version as 1.18.8-1, that should satisfy the min version requirement. But if go.mod says go 1.18.9, then it should not satisfy the minimum version requirement. I just wanted to make sure the toolchain would not barf attempting to parse its own version in order to do those comparisons.
I agree that I would not want to put the patched version itself as the go directive, but it should be supported for the toolchain directive.
For go install path@version, I think we probably have to do the module fetch and then potentially re-exec the go command based on the go line. I would be inclined to say we don't respect any toolchain line, just as we don't respect any replace lines.
@rsc I don't understand why go install path@version would ignore the toolchain in go.mod. If I understand correctly, if I download the source for a module, then run go build, it uses the go.mod's toolchain. If I run go install in that same directory, does it also ignore the go.mod's toolchain, like go install path@version would?
I would expect go install path@version to use the go.mod's toolchain if GOTOOLCHAIN=auto. That way, we get reproducible bits that are actually (presumably) tested and guaranteed to work, just like if we installed a binary command via Homebrew.
As I've noted before, it wouldn't really make sense to me for a distribution to default GOTOOLCHAIN=local unless they are also defaulting GOPROXY=off and also disallowing packages like rustup and nvm, both of which do the same thing that this proposal would have the go command do: manage a collection of toolchains and run the version specified by the user's configuration for a specific build.
Managing the Go toolchain version as well is a little like adding +virtualenv to that list, although much of what virtualenv does is already taken care of by modules and static linking, so maybe it should be there already. Like I said, the analogy is only very loose. Perhaps a better analogy is that Go is more like rustc+cargo than just rustc, and with this proposal it becomes more like rustc+cargo+rustup. In any event, managing the toolchain version is another critical part of a full environment, and it doesn't bother me at all - in fact I think it is a good thing - for Go to include this functionality out of the box rather than expect people to discover and install a separate tool.
I wasn't sold on the go tool dynamically downloading toolchains, until I read these points. Environment managers like virtualenv and nvm are useful and even necessary for working with other languages, and I can now see the value in building this functionality into the toolchain, much like dependency management is. I like how this proposal goes one step further than nvm, and makes it happen transparently and automatically. There are catches, as others have pointed out, but I think long-term it'll be better than the status quo.
However, it seems to me that we can still get that functionality without the explicit toolchain directive in go.mod. The go.mod's toolchain can be inferred from the go version directive, assuming it's required to be up-to-date, and then downloaded if needed. Kubernetes or whatever large project can still have reproducible builds that way.
The toolchain directive in go.mod seems to be more about backward compatibility, and perhaps belongs instead in the back compat proposal, where it would be clearer to demonstrate how it would work together with GODEBUG. I, for one, don't agree with the GODEBUG approach, and I would prefer that back compat concerns not be mixed in with forward compat concerns, if it can be helped.
Rather than exiting with an error, it sounds to me like the toolchain would download and run a new version. I strongly prefer being confronted with an error, rather than the tool (go) deciding to download and run a new version of itself for me.
@mark-pictor-csec Perhaps before it gets to that point, the go tool should determine that the go vX directive in go.mod is incorrect, and report an error about that, similar to how a main module's dep versions must all accurately reflect the maximum version used by the module and all its dependencies. For example, if your main module declares go 1.19, and a dependency module declares go 1.25, go build shouldn't try to fetch go 1.25, it should report an error that the go.mod's go version is out of date.
@golang community. Reminder that the go directive in go.mod is a feature enabler for your current module and NOT a "minimum version of Go supported" indicator for your module's clients.
What's the difference? Can someone give an example? If I understand correctly, you can currently compile a go 1.X module with a go 1.X+N version, but assuming 1.X is a minimum version doesn't cause a problem in that case, since go 1.X and 1.X+N can both be used (and even go 1.X-K, apparently).
The line that only affects older compilers is the go line. The expectation is that the go line will be the main one people use, especially since it is respected in dependencies and has a clear way to merge different requirements (take the maximum listed version, just like MVS on module versions). But if you are coordinating with other developers on a closed system, then you want the toolchain line, to make sure you are all in sync.
@rsc Do you envision that most main modules will have a toolchain directive, and most library modules will not? Or should most main modules also omit a toolchain directive unless they really need to?
Since this proposal would make the go directive a minimum Go version, perhaps go tidy should minimize the Go version. For example, if I write an idiomatic Hello World module today, go.mod should say go 1 (or perhaps more accurately go 1.0.0), not go 1.19. Likewise, if I write a new module today that doesn't use any features past, say, 1.12, then go.mod should declare 1.12, not 1.19.
This would also help older, unsupported toolchains to remain useful.
What's the difference? Can someone give an example?
Today, a module whose go.mod says go 1.18 can still be compiled by older toolchains, and the module can leverage 1.18 language features (generics) when compiled by a 1.18+ toolchain. Meanwhile, if go.mod says go 1.17, it cannot leverage 1.18 language features at all, even when compiled by a 1.18+ toolchain.
In other words, there really is no concept of "minimum version" today, All the go directive really does is act as a gate for newer toolchains to enable newer language features.
That said, one reason the toolchain line includes the go prefix is to allow the future possibility of toolchain gccgo1.2.3 or toolchain tinygo1.2.3.
It seems to me that the go tool shouldn't know about other toolchains. If someone wants to build a package with tinygo, then they should do tinygo build, not go build.
I am only speculating at the concerns of the Linux and other packagers. I would be happy to hear from them directly about any concerns and work with them to address those in a way that works for them and for Go. On the Linux side, perhaps @stapelberg has some thoughts?
Note that I retired from Debian in 2019. @anthonyfok has uploaded Go 1.16, 1.17, 1.18 and 1.19 to Debian (thank you!), so perhaps he has thoughts to share, too.
The Go compiler is included in Debian primarily so that Debian can build programs written in Go. You can find details about how Debian creates a build environment and invokes the go tool in lib/Debian/Debhelper/Buildsystem/golang.pm, if you’re curious.
As you suspected, Debian sets GOPROXY=off, and in fact, all Debian packages are built without network access, so even if we wanted to, we can’t access the internet. I would expect Debian to set GOTOOLCHAIN=local for Debian package builds.
While the Go compiler is primarily included for building Debian packages, plenty of users install the golang-go package for their own usage. Debian’s “popularity contest” infrastructure shows 5450 installs of golang-go: https://qa.debian.org/popcon.php?package=golang-defaults.
Given that Go releases more frequently than Debian, I would expect most users to install Go themselves, but as I said, a non-negligible number just installs (possibly old versions of) Go from Debian.
I will note that Debian has a history of preventing programs to call home in any capacity. About 10 years ago, Debian had a custom patch for the golang-go package which asked users at package installation time (!) whether they want to report to the goinstall dashboard. This mindset goes pretty far: I have seen folks patch out the <style> tag out of HTML documentation files so that browsers wouldn’t “call home” by requesting the CSS file.
Hence, I would not at all be surprised if there was a discussion in Debian about preventing Go from installing newer toolchain versions, and I can see multiple angles that would likely come up: circumventing the system packages, downloading large volumes of data from the internet without warning, generally surprising behavior.
Personally, I think end users would actually appreciate builds to just work, but I would be curious what @anthonyfok thinks. (Though, if Debian developers feel strongly enough, they may chose to overrule even the package maintainer, so ultimately it’s not up to any single human.)
I hope this helps, but do let us know if you had other questions/concerns/ideas you wanted to cover.
@rsc I don't understand why go install path@version would ignore the toolchain in go.mod.
See #57001 (comment). The toolchain line, like the replace line, is for developers working in the repo. People downloading and running a module should be able to substitute their preferred local toolchain (provided it is new enough) rather than being forced to use a specific one chosen by the authors.
Consider running go install program@latest but the program hasn't been updated since Go 1.15, and I am running Go 1.19 locally. The fact that I'm using Go 1.19 locally is a strong indication that I want to use the Go 1.19 runtime, with all the up-to-date crypto and http security fixes, not Go 1.15.
I wasn't sold on the go tool dynamically downloading toolchains, until I read these points.
Glad to hear it. ☺
However, it seems to me that we can still get that functionality without the explicit toolchain directive in go.mod. The go.mod's toolchain can be inferred from the go version directive, assuming it's required to be up-to-date, and then downloaded if needed. Kubernetes or whatever large project can still have reproducible builds that way.
The toolchain directive in go.mod seems to be more about backward compatibility, and perhaps belongs instead in the back compat proposal, where it would be clearer to demonstrate how it would work together with GODEBUG. I, for one, don't agree with the GODEBUG approach, and I would prefer that back compat concerns not be mixed in with forward compat concerns, if it can be helped.
I expect that most users will in fact not use the toolchain directive. It's there for larger projects that need to worry more about consistency. (See answer at end too.)
@mark-pictor-csec Perhaps before it gets to that point, the go tool should determine that the go vX directive in go.mod is incorrect, and report an error about that, similar to how a main module's dep versions must all accurately reflect the maximum version used by the module and all its dependencies. For example, if your main module declares go 1.19, and a dependency module declares go 1.25, go build shouldn't try to fetch go 1.25, it should report an error that the go.mod's go version is out of date.
Yes, that's right. If your go.mod says go 1.19 but you have a go 1.25 dependency, that's an error just like if the same thing happened with module versions. The only way that happens is if you edit the dependency into the go.mod file yourself. If you did go get dep@version and dep@version said go 1.25, then that go get command would update the go.mod file to say 'go 1.25' and tell you it had done so. The next go command would look for Go 1.25 if needed. You have the time in between those two commands to undo, sanity check, etc.
@rsc Do you envision that most main modules will have a toolchain directive, and most library modules will not? Or should most main modules also omit a toolchain directive unless they really need to?
Yes, that sounds plausible, roughly like the split on when replace directives make sense or when go.work files do.
Since this proposal would make the go directive a minimum Go version, perhaps go tidy should minimize the Go version. For example, if I write an idiomatic Hello World module today, go.mod should say go 1 (or perhaps more accurately go 1.0.0), not go 1.19. Likewise, if I write a new module today that doesn't use any features past, say, 1.12, then go.mod should declare 1.12, not 1.19.
This would also help older, unsupported toolchains to remain useful.
Tidy does not know enough to understand what the minimum Go version is. Even if it had perfect knowledge about what's in each release (it doesn't today), it can't possibly know which bug fixes your program needs to work correctly. Just like with module versions, you should write down the one you want to use and expect your users to use, and other people can update as needed.
It seems to me that the go tool shouldn't know about other toolchains. If someone wants to build a package with tinygo, then they should do tinygo build, not go build.
From my point of view, I'd rather just have a project where I put toolchain gccgo5 in my go.mod file and then run all the usual go commands built into my fingers as I run go test, go vet, go build, go install, and so on, instead of having to remember to type a different leading word depending on which directory I am in. That seems like a much smoother experience.
Thanks for the (ex-)Debian perspective, @stapelberg. It's been a very long time since I heard anyone mention the goinstall dashboard! Those were simpler times.
See #57001 (comment). The toolchain line, like the replace line, is for developers working in the repo. People downloading and running a module should be able to substitute their preferred local toolchain (provided it is new enough) rather than being forced to use a specific one chosen by the authors.
@rsc If I understand correctly, if you run go build in a module with a toolchain directive, then it will always use that toolchain. I assume the same is true for go install; please correct me if I'm wrong. It seems to me to be consistent for go install path@version to be equivalent to git-cloning and go-installing. (Isn't that what the go tool basically does anyway?) If the user wants to use the local toolchain, they can override the behavior with GOTOOLCHAIN=local go install path@version. Using the local toolchain by default would basically be saying that in that one case, GOTOOLCHAIN defaults to local, but everywhere else, it defaults to auto. It's better for GOTOOLCHAIN to work consistently.
Consider running go install program@latest but the program hasn't been updated since Go 1.15, and I am running Go 1.19 locally. The fact that I'm using Go 1.19 locally is a strong indication that I want to use the Go 1.19 runtime, with all the up-to-date crypto and http security fixes, not Go 1.15.
@rsc I think perhaps the opposite is the case. 1.15 was the latest (or perhaps only) version that the author tested against. Using a later toolchain introduces a risk that something will break or otherwise go wrong. For all we know, using 1.19 would break the code, like, say, if SHA1 were broken, or path lookup behavior changed, etc. There may be bugs introduced by later versions that haven't even been uncovered yet that the code might experience. The safest path is to use the exact version that the author himself tested against and certified. If the author wants users to take advantage of the latest Go improvements, then they need to bump the module language version themselves and re-test. Users can override the toolchain with GOTOOLCHAIN=local if they want to roll the dice.
It's the same consideration as dependency versions. If your module M depends on module A, and A depends on B v1.0, and you want M to use B v1.5, then you can do that, but A hasn't been vetted by A's author against B v1.5; it's up to you to roll the dice and see how it goes. Or you could just stick to B v1.0 and inherit all the existing stability benefits of A.
Put another way: using the local toolchain by default makes the installed version a pet, not cattle. You can't delete the installed copy, reinstall it, and have the same bits, because you might have upgraded your local toolchain in the meantime. Maybe features changed, or something is now broken. There's no reproducibility.
It seems to me that it's important for go install path@version to produce reproducible bits. Otherwise, you're still better off installing Go-based programs with package managers like Homebrew.
I expect that most users will in fact not use the toolchain directive. It's there for larger projects that need to worry more about consistency.
@rsc I don't follow why that wouldn't be a concern for everyone, for packages/modules of all sizes. Even small libraries that have multiple collaborators would want everyone to use the same exact toolchains, all else being equal. If I remember correctly, I think it was you that pointed out that the toolchain directive would ensure that CI built the right bits.
From my point of view, I'd rather just have a project where I put toolchain gccgo5 in my go.mod file and then run all the usual go commands built into my fingers as I run go test, go vet, go build, go install, and so on, instead of having to remember to type a different leading word depending on which directory I am in. That seems like a much smoother experience.
@rsc I guess it depends on how it works, but assuming toolchain gccgo5 just translates go build into gccgo5 build, I think it would be confusing for go tool users to run go build and get weird, non-standard output from some other toolchain that it re-exec'ed. Or does toolchain just refer to a compiler specifically, and not the overall interface of the go tool (with build, test, etc. subcommands)?
@rsc If I understand correctly, if you run go build in a module with a toolchain directive, then it will always use that toolchain. I assume the same is true for go install; please correct me if I'm wrong. It seems to me to be consistent for go install path@version to be equivalent to git-cloning and go-installing. (Isn't that what the go tool basically does anyway?)
No, that's not true today. If you want to git clone + cd + go install, you can, but that's not what the go command does.
There are some directives that only apply in the work module but not in dependency modules, most notably replace and exclude. When you run go install path@version, it is as though you made an empty work module and then ran
go get path@version
go install path
That is, path ends up being a dependency, not the work module. In particular, replace directives are ignored. One reason is that they often refer to local state that doesn't make sense when others are building the program. To me toolchain seems similar to replace and should also be ignored.
@rsc I think perhaps the opposite is the case. 1.15 was the latest (or perhaps only) version that the author tested against. ...
Maybe if we were starting from scratch, but we're not. Today if you use Go 1.20 to go install path@version you get it built with Go 1.20. If the go.mod says toolchain go1.13 then Go 1.20 will ignore that and still use itself. Supposing we land this in Go 1.21, it would be strange for Go 1.21 to have this discontinuity where it uses an older version than Go 1.20 does.
It seems to me that it's important for go install path@version to produce reproducible bits.
This has never been the case. At the very least the local file system paths are stored in the file-line tables used when printing stack traces. It might be interesting to consider move toward that, such as by using -trimpath in that mode, but that's not the topic of this bug, and I would still insist on never using an older toolchain than the user has.
@rsc I don't follow why that wouldn't be a concern for everyone, for packages/modules of all sizes. Even small libraries that have multiple collaborators would want everyone to use the same exact toolchains, all else being equal.
It's not equal, though. Small libraries will be imported by lots of other modules using a variety of toolchains. They can't unilaterally say "we only care about building with this one toolchain". In contrast, most large projects stand on their own - they are not dependencies of others. So they really can limit their concerns to a single toolchain.
@rsc I guess it depends on how it works, but assuming toolchain gccgo5 just translates go build into gccgo5 build, I think it would be confusing for go tool users to run go build and get weird, non-standard output from some other toolchain that it re-exec'ed.
I am assuming here that 'toolchain gccgo5' ships with an appropriate copy of the go command. (Gccgo does not have its own reimplementation of that.) If not, then yes people will not like using that, and they don't need to write that line. But the same is true if the compiler behaves in a weird, non-standard way. In any event, the go command is tied to the specific toolchain being used: it knows how to invoke the tools in that toolchain, not other ones. If you write 'toolchain go1.30beta1' you definitely want the Go 1.30 beta1 go command, not an older one. An older one almost certainly won't work. And it also won't have any new go command features being introduced in Go 1.30, which you'd also want.
If you write 'toolchain go1.30beta1' you definitely want the Go 1.30 beta1 go command, not an older one. An older one almost certainly won't work. And it also won't have any new go command features being introduced in Go 1.30, which you'd also want.
This made me think about future changes to the go command more critically. Say a future version of the go command adds a new command line flag "-foo". Now suppose a project documents to use that flag when installing/building it, go install -foo proj@latest. If someone with an older version of the go command follows the instructions what would happen? Would the older go command accept the unrecognized flag tentatively hoping that the version specified in proj/go.mod would know what to do with it? Or would the older go command simply error with "flag provided but not defined: -foo" causing confusion and frustration?
@ChrisHines, in general go blah will need to check the go.mod version and perhaps run a new go command without parsing command-line options, since maybe blah doesn't even exist as a subcommand in the original go toolchain.
The specific case ofgo install -foo proj@latest is already a special case in various ways, and it is special here too. Today the rule is that all the install targets named on the command line are from the same version of the same module. In that case, it may be that we should check for @ in the final argument, resolve that module's go.mod, and potentially run a new go command, all without parsing the rest of the command line. That would let future flags be used as you described.
Regarding the toolchain line in go.mod versus the GOTOOLCHAIN environment variable, proposal #57179 contemplates letting the work module provide a go.env file similar to the user's environment file (what go env -w updates). If we did that, then I think we'd just say to put a GOTOOLCHAIN= line in the go.env and not introduce the toolchain line in go.mod.
@rsc You havw made the pets and cattle comparison above, which I think is interesting. To that I answer that treating servers like cattle doesn't actually work well, especially with mixed services as each had their own specific requirements. What we have is more like a zoo. Each service needs a specific habitat and care, even though you have more than just one instance of the same.
Likewise, go 1.13 is a different animal than go 1.31 will be and each if the two requires their own care to be used. This is especially so for low level functionality like runtime or unsafe, which can be used for building GUI applications, mobile applications, games, simulations, embedded software, etc.
Especially for embedded applications, often security fixes are less important than stability, and continuous support for low level runtime/unsafe functionality. This is why I have argued for a long term support version of Go before, but this was declined. Nevertheless, people in certain software categories do need an LTS version, so they just take a given version of the Go compiler and validate that through rigorous testing to get their LTS, ignoring any security fixes. For this kind of software, automatic upgrades of the compiler are wholly unwanted.
I think the focus of Go development has been on web based/back end software, which is not bad in itself, and for that, security is of utmost importance. For back end software, automatic compiler upgrades would indeed help improve the security. But I consider that Go should be a general use programming language. And for anything other than servers, what is required is precise control of the compiler used. I feel this proposal overlooks and ignores the general use case of Go.
Maybe if we were starting from scratch, but we're not. Today if you use Go 1.20 to go install path@version you get it built with Go 1.20. If the go.mod says toolchain go1.13 then Go 1.20 will ignore that and still use itself. Supposing we land this in Go 1.21, it would be strange for Go 1.21 to have this discontinuity where it uses an older version than Go 1.20 does.
@rsc How do you envision enabling reproducible, bit-for-bit identical builds for main packages? If I understand correctly, with this approach, if a package manager like Homebrew wants to do that for a Go-based Homebrew package, it would still need to use a third-party Go environment manager like gimme to do it. It seems to me that's the kind of problem this kind of approach could solve.
Perhaps this boils down to using MVS for language versions as well as module versions. It seems to me that the reasons for using it for module versions apply equally to language versions. Why not use it for both?
Discontinuities have been rolled out before, like the transition from GOPATH to modules. It's probably possible to do it somewhat gracefully with a transition period and a feature flag, like modules.
Thinking about the auto-upgrade behavior, I really think that should be split to a separate proposal. That keeps this proposal just focused on forwards compatibility - in particular how to explicitly break it by redefining the go directive. It is already the case that some module may fail to compile with an older toolchain (e.g., by referencing some newer standard library function).
That said, I think that having the toolchain automatically download and execute some binary without the user's direct consent is a potential security issue. This is exacerbated by the the need for checksum verification. I know this can be addressed this by pulling the checksum from GOPROXY like go install does, but that just shifts the problem to the module proxy itself being compromised/poisoned. Another options would be to record the toolchain checksum in go.sum, but that could be a problem across GOOS/GOARCH combinations, as well as for pre-installed or custom toolchains.
With regards to a GOTOOLCHAIN environment variable and a hypothetical go.env file, I don't think this is an equivalent solution. With an environment variable, I think it is likely people will forget they set it, leading to much confusion as they switch between projects. As for go.env, I worry it will not be committed to source control (as it may for example contain the credentials for GOPROXY), which means it is not a suitable replacement for the proposed toolchain directive, which must be committed in order to enforce consistency across developers.
However, if a modules lacks a toolchain directive (e.g. a library that supports multiple versions), then I do see value in the environment variable and (uncommitted) go.env file. Thus I would see the GOTOOLCHAIN environment variable as specifying the toolchain to use only if the module did not explicitly specify one via the toolchain directive.
@rsc How do you envision enabling reproducible, bit-for-bit identical builds for main packages? If I understand correctly, with this approach, if a package manager like Homebrew wants to do that for a Go-based Homebrew package, it would still need to use a third-party Go environment manager like gimme to do it. It seems to me that's the kind of problem this kind of approach could solve.
Starting in Go 1.21, a specific Go toolchain running 'go install -trimpath package@version' will give a bit-for-bit identical build of a package that does not use cgo (including not using package net), either because there is no cgo use or because CGO_ENABLED=0 is set. If this proposal is adopted, it would also work to use GOTOOLCHAIN=specific-version go install -trimpath package@version.
Suppose we use a relatively new Go toolchain to build an old Go program. Concerns such as avoiding known bugs and incompatibilities in older Go toolchains (including the older toolchain not even supporting the platform the new toolchain is being run on) argue in favor of the new toolchain using itself even if the old program says go 1.15 in its go.mod file. Bit-for-bit reproducibility argues in favor of using the older toolchain. To me, and I think to most users, the concerns in favor of using the new toolchain outweigh the bit-for-bit reproducibility of using the older toolchain.
To summarize the current state of the proposal, there are a few ideas here:
-
Starting in Go 1.N, the
goline of any module becomes the minimum allowed version of Go that can build that module. This is the fundamental change that ensures forward compatibility, because now older versions of Go will know that they are incapable of compiling newer Go code. Without this change, it is impossible to make corrections to problems like loop scoping (discussion #56010), and it is also impossible to declare that a program needs to be built with a new enough version of Go to avoid a semantic bug fixed in a point release. -
If a file has a
//go:buildexpression that implies a newer version of Go, newer Go features present in that version (generics, better for loops, literals with underscores, and so on) can be used in that file. Previously those features could only be "unlocked" by thegoline. Now they can be unlocked by thegoline for the whole module but also on a per-file basis using//go:buildlines. The//go:buildline would be required regardless, to hide the file from older Go versions. Inferring the newer Go version from the line does not add any additional cost to users. This too is safe for forward compatibility, since older versions of Go will not attempt to build the file. -
Because (1) would otherwise cause build breakages for users of older Go toolchains trying to compile newer code, the go command becomes more like Node's nvm, Python's virtualenv, or Rust's rustup, where it manages a set of installed toolchains (including one bundled with it as a default). When the
gocommand finds that its bundled toolchain is too old to build the code it is asked to build, it automatically selects the minimal supported Go version available (either locally or downloaded) to satisfy the requirement in thegoline. If it can't do that, it stops the build. The toolchain management avoids unnecessary build breakages. -
As a convenience and for debugging, the GOTOOLCHAIN variable overrides the usual selection algorithm and forces the use of a specific toolchain version.
Concerns were raised about potential confusion, answered here. It will always be possible to run go version to find out what version of Go is being run in a given directory. The special case go install path@version, which operates outside any given workspace, will report when it switches to a Go toolchain than the one bundled with the go command being run.
Concerns were raised about "losing control" over the go toolchain version, answered here. To the extent that Linux distributions or other use cases want to force builds to use of the local Go toolchain (the one bundled with the go command being run), they can set GOTOOLCHAIN=local, and the go line will be ignored for toolchain selection. If the local toolchain is too old for the command being build, that will be a build error and they can update their build scripts.
Concerns were raised about the idea of managing multiple toolchains, answered here with analogies to nvm, virtualenv, and rustup.
Concerns were raised about managing multiple toolchains not being an actual problem, answered here.
Concerns were raised about the change in the go line to a minimum version, answered here and then also here adding the //go:build unlock mechanism (thanks very much to @rittneje for the discussion that led to that refinement).
Are there any other concerns with adopting this proposal? Thanks everyone for a productive discussion.
I appreciate the addition of //go:build to enable language features; that is the main problem I had with the original proposal. I think without this, the ecosystem would be much more likely to lag (not move forward), given many project's tendencies to try and maintain the same support window as Go itself. Using //go:build makes language feature unlocking congruent with the use of new APIs behind build tags.
Though, I don't really know how this is supposed to work with a breaking change like loop semantics, since there isn't a compiler error involved to tell you that you need to do something to get the new behavior. I can totally see myself introducing bugs because I don't realize the project I'm working on doesn't have the new semantics yet. There are even still big projects like Docker which don't have go.mod at all, so are locked to Go 1.13 (?) semantics. (I'm personally of the opinion that the loop behavior could be changed wholesale ignoring compat and things would be better off, but that's another thread.)
I am still not super happy with the idea that the Go tool will be downloading toolchains and using them instead of what I have installed, but I am at least glad there have been some updates related to this, namely #57007 (make the Go toolchain not depend on libc) and #57179 (effectively, allow distributions to set GOTOOLCHAIN without patching). The decreasing download size is also a help, though I still think that it's still a pretty big download if your internet isn't so good (or is metered). I guess the AWS and Azure modules are some 100+ MB already (ouch!), so, what's another 90 MB.
- If a file has a
//go:buildexpression that implies a newer version of Go, newer Go features present in that version (generics, better for loops, literals with underscores, and so on) can be used in that file. Previously those features could only be "unlocked" by thegoline. Now they can be unlocked by thegoline for the whole module but also on a per-file basis using//go:buildlines. The//go:buildline would be required regardless, to hide the file from older Go versions. Inferring the newer Go version from the line does not add any additional cost to users. This too is safe for forward compatibility, since older versions of Go will not attempt to build the file.
Do I have access to Go 1.55 features if I write:
//go:build linux && go1.55?//go:build ignore && go1.55?//go:build go1.54 || (ignore && go1.55)?
Do I have access to Go 1.55 features if I write:
//go:build linux && go1.55?
Yes.
//go:build ignore && go1.55?
Yes.
//go:build go1.54 || (ignore && go1.55)?
No, only go1.54.
The specific algorithm is to ignore the non-go build tags and interpret && as max and || as min. Feel free to try https://go.dev/play/p/cdclo6wg-kF by adding more tests to the table.
Starting in Go 1.21, a specific Go toolchain running 'go install -trimpath package@version' will give a bit-for-bit identical build of a package that does not use cgo (including not using package net), either because there is no cgo use or because CGO_ENABLED=0 is set. If this proposal is adopted, it would also work to use GOTOOLCHAIN=specific-version go install -trimpath package@version.
@rsc Makes sense; that could work. However, they have to have a specific Go version in mind, and I imagine most Homebrew package maintainers would prefer to use the latest tested bits, not the bits produced by the latest version of Go. To do that, they would have to look at the Go version in go.mod, then set GOTOOLCHAIN to that manually. It seems like we could facilitate that with a GOTOOLCHAIN mode, like GOTOOLCHAIN=minimal or GOTOOLCHAIN=match.
It seems to me that defaulting to using the latest toolchain version (or a hard-coded toolchain version) effectively puts the testing burden on Homebrew package maintainers (or any builder), rather than on Go package authors, since the toolchain version used to build is unlikely to be the one used to test.
You listed benefits to using a later Go toolchain, but there can also be detriments, like the loss of support for an architecture or OS, the demotion of a port to third-party status (or a port otherwise falling "behind"), ports developing new/unique defects, changed behavior (e.g. SHA1), etc. Basically, the reason why we use MVS for modules, which is why I asked why not use MVS for toolchain versions too.
Personally, I would rather use GOTOOLCHAIN=minimal for go install commands (and have it be used for my installed Go-based Homebrew packages) to ensure stability and correctness.
- Starting in Go 1.N, the
goline of any module becomes the minimum allowed version of Go that can build that module. This is the fundamental change that ensures forward compatibility, because now older versions of Go will know that they are incapable of compiling newer Go code. Without this change, it is impossible to make corrections to problems like loop scoping (discussion redefining for loop variable semantics #56010), and it is also impossible to declare that a program needs to be built with a new enough version of Go to avoid a semantic bug fixed in a point release.
-
In the original proposal, the
goline in go.mod accepts point release version strings (e.g.go 1.30.1). Is it still true with this adjustment? Based on the requirement stated in the last statement, I guess that's true. That will be useful if we want to utilize this to help users easily address security issues in stdlib/toolchain fixed in a point release - users just need to update thegoline in their work module go.mod or go.work. -
When
go mod initis called when my defaultgois 1.30.1, will it writego 1.30orgo 1.30.1? -
The original proposal was more about the
goline in the work module and it did not change the semantic of thegoline in the dependency (Effect in dependencies). Is this part going to be changed, too?
- ... When the
gocommand finds that its bundled toolchain is too old to build the code it is asked to build, it automatically selects the minimal supported Go version available (either locally or downloaded) to satisfy the requirement in thegoline. If it can't do that, it stops the build. The toolchain management avoids unnecessary build breakages.
- Why should it be the "minimal" supported Go version available instead of the "latest" among the available versions (either locally or downloaded)? When
go 1.30in go.mod but a satisfying version is not locally available, I guess the go command will prefer go 1.30.1 to go1.30 for download. Similarly, if I have bothgo1.30andgo1.30.1locally available, shouldn't it prefer go1.30.1? If I havego1.30andgo1.34locally, shouldn't we prefergo1.34even when the work module still hasgo 1.30?
Feature request: would be nice if go offers commands that help maintaining locally/remotely available go versions.
- Listing locally available versions
- Listing the latest supported versions & prereleases (beta/RCs) for the current os/arch
- Manually installing/uninstalling selected versions
- Set the default version. For example, a user may want to install/upgrade the local
goversion once or twice a year from go.dev/dl or initially rely on the default version packaged in the distro. But then utilize this feature to easily pick a more recent version as the default.
I am still trying to wrap my mind around how some of the mechanics work, as well as some of the possible resulting human / ecosystem behavior, so sorry if some or all of this is off base.
Avoiding old releases running code they don’t understand
So we've painted ourselves into a bit of a corner where the best we can do is prepare K versions that recognize that they don't understand the new code and then issue the K+1'th version supporting the new code. The only thing we really have control over is the choice of K.
There might be a way for us to unpaint our way out of that corner by making sure old releases outside of that K window still reject the newer go.mod. There might be a few different ways to do it, including I filed a separate proposal #57631 yesterday with one approach that might work, which is to always use three digits for the go version in the go.mod file, even for the first release of a major release.
Why is a particular Go version needed?
go version helps with what version is in use, but determining why a particular Go version is needed for a given module might be a challenge in non-trivial cases. Would go mod graph be updated to include a go module, and/or would there be a variation of go mod why or similar that includes go version information, or …?
‘go get’ updating the ‘go’ line in go.mod
In a comment above (with a somewhat similar comment here), Russ wrote:
it also seems like it is probably okay for
go get footo print a message about updating the go line, so that you don't even have to reach for git diff.
If go get and presumably go mod tidy update the go line in my go.mod based on my dependencies, I'm not quite sure how that works.
If I want Go 1.20 semantics in my module, but I have a dependency that uses Go 1.25 semantics, does that mean that my module is forced into Go 1.25 semantics?
Minor: should this sentence in the proposal also mention the go line?
On the other hand, if a change is needed, the go command would edit the toolchain line or add a new one, set to the latest Go 1.M patch release Go 1.M.P.
Ratchet effect
For modules, there is by design a general “ratchet” effect that tends to nudge versions upwards. That effect is more pronounced for larger projects, with larger dependency graphs and greater odds of the same dependency appearing in multiple pieces of the graph. With this change, rather than having something like logrus that might appear in, say, 10% of your dependencies, this would be the “go” module that appears in 100% of your dependencies, so it seems the “ratchet” effect for the go version would be stronger than what has been seen so far with modules in the wild under the current system. (Presumably the go version selection rides on top of module graph pruning? That helps cut down the graph, but the graphs can still be big and include dependencies that the main module maintainer might consider unimportant).
This comment has some overlap with how this proposal might affect the choices people might make under what @dominikh described. Russ replied to that in a couple spots including here. That’s a reasonable discussion, including people’s behavior will be heavily influenced by tending to choose more convenient or lower cost options. In contrast, the ratchet effect is less about choosing easier options, and instead it increases the frequency of “you are getting upgraded unless you work against it”.
Manual intervention
There’s an analogy that can made between toolchain and replace directives, but it seems replace is more powerful, including because it sounds like toolchain cannot be forced below what one of the modules in the build states it requires for a go version (even if your build doesn’t actually need that go version). The proposal says “The Go toolchain will refuse to build a dependency that needs newer Go semantics than the current toolchain”. In contrast, require allows one to force as low as desired (and then it’s up to the human to be sure the override is a valid thing to do). As I understand it, the power of replace in the design of modules was part of the balancing act for handling less common cases that still inevitably occur, including under an overall module system that had fewer knobs than prior package management systems, and I wonder if toolchain should be roughly as powerful.
Medium/larger projects
If the manual intervention toolset specifically for go versions is less powerful, and that's coupled with a more powerful ratchet effect, might that be some degree of problem at least for medium and large projects?
One can use also replace to try to resolve go version issues by changing the versions of dependencies, but that might be inappropriate in some circumstances, or might result in more work for medium/large projects compared to the current situation. toolchain helps, but also seems to be of limited use for a library in an open environment. (Forking is always an option too, but if that is needed more often than now, that’s also more work).
Newer toolchains better at being an older Go than older toolchains?
In addition, under this proposal and #56986, it is probably fair to say that something like the go1.25.9 toolchain will be a better version of Go 1.24 than a 1.24 toolchain, but I’m not sure if that would always be true for a go1.X.0 toolchain, or at least, it is fairly classic behavior for some more conservative people to wait for 1-2 patch releases of almost anything important after a new major release to let any new bugs shake out. Aspirationally of course, the Go project wants a .0 release to be as stable as possible, and the much heavier use of GODEBUG via #56986, the (new?) continual rolling out of passing dev builds to Google’s production use, and the new shifts in timing of the Go project dev cycles would all likely help with that… but the worldwide Go ecosystem is a superset of Google’s use of Go, and I wonder if people will end up on the .0 release more often than they desire due to some of the dynamics described above, and as a result hit problems that could have been avoided under a slightly different scheme…
In any event, all of this might be desirable, but partly just trying to understand the proposal and people’s current predictions of resulting ecosystem behavior.
@willfaught I think we are just going to disagree about this detail. I will note only that the proposed behavior is as close as possible to the current behavior except for updating just enough to avoid a "this version of Go is too old to build your new code" error. That is:
- If you use Go 1.19 today and go install something with a go.mod that says 'go 1.12', you use Go 1.19 to do it.
- If this lands by Go 1.30 and you use Go 1.30 to go install something with a go.mod that says 'go 1.25', it will use itself (Go 1.30) to do it. Same as always.
- The only time the behavior changes is if you use Go 1.30 to go install something with a go.mod that says 'go 1.40'. When GOTOOLCHAIN=local you get a build error (cannot build Go 1.40 code). But by default instead it gets a newer stable version of Go and uses that, to avoid the build error.
Replying to @hyangah
- In the original proposal, the
goline in go.mod accepts point release version strings (e.g.go 1.30.1). Is it still true with this adjustment? Based on the requirement stated in the last statement, I guess that's true. That will be useful if we want to utilize this to help users easily address security issues in stdlib/toolchain fixed in a point release - users just need to update thegoline in their work module go.mod or go.work.
Yes.
- When
go mod initis called when my defaultgois 1.30.1, will it writego 1.30orgo 1.30.1?
I think it should write go 1.30.1.
- The original proposal was more about the
goline in the work module and it did not change the semantic of thegoline in the dependency (Effect in dependencies). Is this part going to be changed, too?
The text does change the semantics of the go line in the dependency, in the sense that (as in the example) if you're using Go 1.27 and the go line in the dependency says go 1.28, it will refuse to build. In both the work module and the dependencies, the go line is now the minimum allowed version of Go to use to build the code. Just like with require lines, if the go command is doing its job, the version listed in the work module should always be >= the version listed in a dependency. The part about //go:build unlocking newer Go version features will apply in dependencies as well.
- Why should it be the "minimal" supported Go version available instead of the "latest" among the available versions (either locally or downloaded)? When
go 1.30in go.mod but a satisfying version is not locally available, I guess the go command will prefer go 1.30.1 to go1.30 for download. Similarly, if I have bothgo1.30andgo1.30.1locally available, shouldn't it prefer go1.30.1? If I havego1.30andgo1.34locally, shouldn't we prefergo1.34even when the work module still hasgo 1.30?
We need to work out what happens for local toolchains. For remote toolchains if the two stable Go versions are Go 1.40 and Go 1.41 and the go.mod says Go 1.30, then all things being equal it seems safer from a compatibility standpoint to take the latest point release of Go 1.40 (closer to Go 1.30) than to use Go 1.41. Same reasoning as for minimal version selection elsewhere.
Feature request: would be nice if
gooffers commands that help maintaining locally/remotely available go versions.
- Listing locally available versions
- Listing the latest supported versions & prereleases (beta/RCs) for the current os/arch
- Manually installing/uninstalling selected versions
- Set the default version. For example, a user may want to install/upgrade the local
goversion once or twice a year from go.dev/dl or initially rely on the default version packaged in the distro. But then utilize this feature to easily pick a more recent version as the default.
I think we should consider this but in a separate proposal. The local toolchains for this proposal are whatever is in $PATH. We can make it easier to put them there but that can be separated from the rest of this.
Thanks!
Replying to @thepudds's comment
So we've painted ourselves into a bit of a corner where the best we can do is prepare K versions that recognize that they don't understand the new code and then issue the K+1'th version supporting the new code. The only thing we really have control over is the choice of K.
There might be a way for us to unpaint our way out of that corner by making sure old releases outside of that K window still reject the newer go.mod. There might be a few different ways to do it, including I filed a separate proposal #57631 yesterday with one approach that might work, which is to always use three digits for the go version in the go.mod file, even for the first release of a major release.
Unfortunately that won't work for rejecting dependency modules, where the rejection is most important. Unsupported versions are unsupported, though, so I'm not too worried. I saw your followup on 57631 about changing the go line to have two fields or something like that, and while we could do that, it feels a bit like inventing a whole new for loop syntax just to fix the scoping bug: yes strictly speaking it avoids one problem, but it causes so much churn that I think it becomes a whole new problem.
go versionhelps with what version is in use, but determining why a particular Go version is needed for a given module might be a challenge in non-trivial cases. Wouldgo mod graphbe updated to include agomodule, and/or would there be a variation ofgo mod whyor similar that includes go version information, or …?
Yes, including the go line in go mod graph makes sense. Thanks for the suggestion. The idea is to make go as a module path refer to the Go line, so you can go get go@latest and so on. Making it work with go mod graph would be a natural extension.
If
go getand presumablygo mod tidyupdate the go line in my go.mod based on my dependencies, I'm not quite sure how that works.If I want Go 1.20 semantics in my module, but I have a dependency that uses Go 1.25 semantics, does that mean that my module is forced into Go 1.25 semantics?
You're right, this got mangled during the discussion. I will go back and figure out what the right update is. I think the toolchain line needs to be >= the dependency go lines, but the go line itself does not. I haven't implemented this part of the proposal yet, and I tend not to fully understand the details until I do. I hope to do more of it this coming week. Thank you!
For modules, there is by design a general “ratchet” effect that tends to nudge versions upwards. ... In contrast, the ratchet effect is less about choosing easier options, and instead it increases the frequency of “you are getting upgraded unless you work against it”.
I think it remains to be seen. It might be that maintainers of popular libraries will decide to (or be pressured to) ensure that they declare an older minimum Go version so that people on those older Go versions can still use their libraries. Or it might be that the ecosystem stops thinking about updating to a newer Go version as being a heavyweight operation. I don't know which it will be; either seems okay to me.
There’s an analogy that can made between
toolchainandreplacedirectives, but it seemsreplaceis more powerful, including because it sounds liketoolchaincannot be forced below what one of the modules in the build states it requires for a go version (even if your build doesn’t actually need that go version). The proposal says “The Go toolchain will refuse to build a dependency that needs newer Go semantics than the current toolchain”. In contrast,requireallows one to force as low as desired (and then it’s up to the human to be sure the override is a valid thing to do). As I understand it, the power ofreplacein the design of modules was part of the balancing act for handling less common cases that still inevitably occur, including under an overall module system that had fewer knobs than prior package management systems, and I wonder iftoolchainshould be roughly as powerful.
I don't believe it should be. If code is using Go 1.30 loop semantics and you force the use of Go 1.20, it's just going to fail. Perhaps there should be an override option on the command line for debugging and testing purposes, but I wouldn't want to put that in the go.mod file where it might be used on a regular basis with unfortunate results.
If the manual intervention toolset specifically for go versions is less powerful, and that's coupled with a more powerful ratchet effect, might that be some degree of problem at least for medium and large projects?
We don't know. I don't think it has to be a problem (see two answers up).
In addition, under this proposal and #56986, it is probably fair to say that something like the go1.25.9 toolchain will be a better version of Go 1.24 than a 1.24 toolchain, but I’m not sure if that would always be true for a go1.X.0 toolchain, or at least, it is fairly classic behavior for some more conservative people to wait for 1-2 patch releases of almost anything important after a new major release to let any new bugs shake out. Aspirationally of course, the Go project wants a .0 release to be as stable as possible, and the much heavier use of GODEBUG via #56986, the (new?) continual rolling out of passing dev builds to Google’s production use, and the new shifts in timing of the Go project dev cycles would all likely help with that… but the worldwide Go ecosystem is a superset of Google’s use of Go, and I wonder if people will end up on the .0 release more often than they desire due to some of the dynamics described above, and as a result hit problems that could have been avoided under a slightly different scheme…
This is an important point. I do think the ecosystem will continue to exert some pressure to keep popular things working with older Go versions, and assuming there's a choice, the points you raise are the main reason for using the lower major version of the two stable ones when possible. It means if you do need to use a newer toolchain you get the "most baked" one that you can, not the most recent one.
Thanks for the thoughtful analysis.
Hi @rsc
If go get and presumably go mod tidy update the go line in my go.mod based on my dependencies, I'm not quite sure how that works.
If I want Go 1.20 semantics in my module, but I have a dependency that uses Go 1.25 semantics, does that mean that my module is forced into Go 1.25 semantics?
You're right, this got mangled during the discussion. I will go back and figure out what the right update is. I think the toolchain line needs to be >= the dependency go lines, but the go line itself does not. I haven't implemented this part of the proposal yet, and I tend not to fully understand the details until I do. I hope to do more of it this coming week. Thank you!
If the go line in my module is not updated by a go get foo or go mod tidy to be >= the dependency go lines, it might then be harder to see what toolchain version a module really needs? It would be nice for that information to be easily visible in the go.mod, such as when deciding whether or not to adopt a library dependency in an open system, or when looking at the VCS history of your own go.mod, and so on. (Perhaps pkg.go.dev could help some, at least for open source modules).
Messages output during a go get or go mod can be helpful, but also can be lost when there are many such messages.
If go.mod doesn't tell you the answer and instead go version is the recommended way to see what version of the toolchain your module needs, it seems you might in some cases execute the thing you just downloaded in order to see what you just downloaded. (As I understand it, the cmd/go you invoke from your shell would be delegating things like go help and go version to the appropriate version of cmd/go).
The go line and toolchain line under this proposal have various responsibilities, and those responsibilities have different scopes (e.g., some are specific to the module, some apply to the build). For example, what Go semantics apply to this module, what is the minimum version of tools needed for a build that includes this module, is there an override version of the tools, and so on.
This is only a half-formed thought — would it make sense to juggle those responsibilities slightly? For example, would it work to have the go line indicate the Go semantics for that module, but then always have a toolchain line, where it is the minimum toolchain required by that module, and the toolchain line is updated by go get and go mod tidy if needed based on the go lines and toolchain lines of all dependencies? If someone needs an override version of the tools, given it is an error to request something lower than what cmd/go would naturally write during a go mod tidy, perhaps it is sufficient for the human to increase the toolchain line (via edit in a text editor or via cmd/go) if they need an override for a higher version of tools, and then for cmd/go to respect that higher value during subsequent operations like go get and go mod tidy? I'm not sure if that works, and I'm also not sure of the interplay with the environment variables.
Only a quarter-formed thought is you alternatively could lean into the similarities with require and actually use a require directive, but that is probably not a good approach given it would likely not be exactly the same as require (e.g., versions with a leading v or no v, can it then be managed via replace and exclude, and so on).
Separately, in a couple of places above, I wrote things like "for a build that includes a module"... but I'm curious what is the precise way to say "build" in the context of this proposal? Is it the module graph that would be required to satisfy a go test all when executed in the work module?
Thanks for the detailed response. Once there is some time to work through the details, it will likely be clear how to best resolve... 😅
There are three potentially different Go versions in play:
- GoSemantics: The specific Go version semantics requested by this module.
(This can be overridden on a per-file basis using //go:build tags.) - GoMinToolchain: The minimum toolchain version this module needs for a successful build.
- GoToolchain: The specific Go toolchain to use for the build.
It must be that GoSemantics ≤ GoMinToolchain ≤ GoToolchain, and GoMinToolchain = max(GoSemantics, every dependency module's GoSemantics).
Often, we want GoToolchain > GoSemantics: this happens when you are building old code with a new toolchain, which is what #56986 aims to make even easier. So there are at least two versions we need to express. In the same contexts, often GoToolchain > GoMinToolchain, such as when you are working on code that you want to allow to build with older Go even though you yourself are using a newer Go.
Wanting to support those inequalities implies that the GoToolchain must be specified separately from GoSemantics and GoMinToolchain. In the original proposal, that's the role of the toolchain line. Since the go.env proposal as accepted did not end up including a per-module go.env, we don't have the option of a GOTOOLCHAIN= line in a per-module go.env, so the go.mod toolchain line remains part of this proposal.
The current discussion of the proposal has converged on requiring GoMinToolchain = GoSemantics by making the go line set both, rather than allowing them to be different by having a third kind of go.mod line. By the definition of GoMinToolchain, the only way they can be unequal is if the work module's GoSemantics < a dependency module's GoSemantics, meaning we have a Go module written for an older Go version that requires a dependency written for a newer Go version.
The main benefit of merging GoMinToolchain with GoSemantics is simplicity. There is one less configuration knob, making the system overall easier to understand and predict. In this scheme, the go line follows MVS the same way that dependency versions do.
The downside would be if developers really need to have a module requiring old Go semantics be able to add or update to a dependency requiring new Go semantics, without first updating the Go semantics of the parent module. I think this ends up being a self-fulfilling prophecy. If we make it possible, it will happen, and if we make it impossible it will not happen. In the impossible case, I think it will both not happen and not be a big deal, because developers will internalize the rule that you don't start using new Go semantics if you are trying to support older Go versions and won't bump their go lines unnecessarily. We go through a transition like this in every release: developers understand that using a new library function in their module will require their users to update to a newer Go version. The same will become true of bumping the go line, and it will just be added to that same mental model. (This is also why I like using //go:build to unlock Go semantics in newer files, since it reuses the existing mental model of those files.)
With that clearer understanding of the design space, I am leaning toward at least starting out with the go line setting both GoMinToolchain and GoSemantics. As others have noted, setting GoSemantics = GoMinToolchain serves as a kind of "ratchet" that moves the line forward, but it's the same kind of ratchet as adopting strings.Cut (introduced in Go 1.18) or generics in your package, and that hasn't been a problem. We can revisit the forced equivalence of GoMinToolchain and GoSemantics and "unsimplify" if we accumulate evidence that it's not working. On the other hand if we start with them split we will not be able to try simplifying in the future.
It is also interesting that over on #56986 there was concern about nothing pushing go lines forward in the overall ecosystem, with the result that users might be stuck on old Go semantics and in particular depending on old //go:debug settings without realizing it. The ratchet behavior being introduced here would address that concern, giving a clear positive benefit to offset the hypothetical downside we have been discussing.
Hi Russ, does this mean that if loop variable semantics are redefined (#56010) in for example Go 1.25, then in my module if I update to use a version of a single dependency that has adopted Go 1.25 semantics, it would mean my module is necessarily adopting the redefined loop variable semantics at that point?
For example, I might need a bug fix or perhaps a security fix within a dependency, but the version of the dependency with the fix might have moved its go line to 1.25.
Is the theory that #56010 is expected to break ~0 code, or that open source projects won't adopt it too quickly, or otherwise be the case that individual projects can adopt the change after what they consider sufficient testing and so on?
One thing to think about might be something official or some semi-official statement from the Go project about expected norms for an open source project in terms of how quickly new Go semantics are adopted (that is, how quickly a project's go line is updated), especially for public modules consumed as libraries, tools that appear within go.mod files, and so on. That might help it not be too fast, as well as help reduce debate on various open source issue trackers about when to update.
Separately, do the //go:build lines as proposed allow people to return to earlier semantics? It seems to say above that the //go:build lines allow adopting new semantics on a file-by-file basis, but as far I understood, they seem to be about opt-in for newer features as "unlockers", and are not a way to say "I'm not ready yet". For example, from above: "If a file has a //go:build expression that implies a newer version of Go, newer Go features present in that version (generics, better for loops, literals with underscores, and so on) can be used in that file.".
I might need a bug fix or perhaps a security fix within a dependency, but the version of the dependency with the fix might have moved its go line to 1.25.
I hope this will not be the case. This kind of thing can already happen, like if you need a bug fix but the security patch also somehow includes a new use of strings.Cut or generics, but I assume well-maintained libraries don't do that.
The for loop fix is the biggest question mark in all this, but yes the theory is that #56010 should break essentially 0 code (especially based on C# experience and our own code inspection).
A statement about expected norms in docs sounds good.
Separately, do the //go:build lines as proposed allow people to return to earlier semantics?
Good question, one I've been thinking about for a few days off and on. I think yes, if the file says //go:build go1.15 then that's an assertion it expects Go 1.15 semantics.
Are there any remaining concerns about accepting this proposal?
Wanting to support those inequalities implies that the GoToolchain must be specified separately from GoSemantics and GoMinToolchain. In the original proposal, that's the role of the toolchain line. Since the go.env proposal as accepted did not end up including a per-module go.env, we don't have the option of a GOTOOLCHAIN= line in a per-module go.env, so the go.mod toolchain line remains part of this proposal.
@rsc I haven't seen the rationale for why the toolchain directive in go.mod hasn't been deferred until it's decided whether to enable go.env per module. Enabling go.env per module would make the toolchain directive in go.mod redundant.
@willfaught Over in #57179 we decided against enabling go.env per module. See the concerns in #57179 (comment) . There is no current proposal for go.env per module, and it seems hard to do.
Meta:
Are there any remaining concerns about accepting this proposal?
I want to signal that I've read it all and and have no more concerns, but that particular form of asking for feedback made me stop to think the best way to do that.
If I put a 👍 on the comment asking the above question, will it be interpreted as a thumbs up on the proposal or a thumbs up that I have more concerns? If I did have more concerns I should post a new comment containing the concerns, so maybe it's clear enough. On the other hand, maybe I'm still writing out my concerns. The reader doesn't know when an emoji vote was made. But if I don't have any concerns I don't want to add a comment because we want to avoid +1 comment clutter. Putting a 👎 on the comment asking the above question (to signal "no objections") doesn't seem right. If I do nothing then maybe I'm happy with the proposal and maybe I just haven't caught up yet, you don't know.
Github doesn't offer a lot of choices for emoji voting. Maybe 😄 works here, I'll go with that, but had the question had been:
Have all concerns about this proposal been addressed?
Then I think 👍, ❤️, 😄, or 🎉 would all have been good choices to signal satisfaction, and posting a comment with additional concerns clearly communicates the opposite.
Good metapoints @ChrisHines thanks. I will adopt that phrasing.
Over in #57179 we decided against enabling go.env per module. See the concerns in #57179 (comment).
Thanks, but I was aware of those details.
There is no current proposal for go.env per module, and it seems hard to do.
There's a difference between there being no current proposal for it (because it was recently decided to make it a separate issue), and there will never be a proposal for it. The way I read it, @rsc seemed to indicate the issue was that per-module go.env files require more thought to flesh out, not that the idea is a bad one, and will never be adopted:
Thanks @hyangah. I started answering these questions but there are too many significant problems with the per-module go.env that don't arise with the $GOROOT/go.env. Let's keep this proposal scoped to just the $GOROOT/go.env.
If the Go team is resolved that setting GOTOOLCHAIN in a module's go.env will never be allowed, then it makes sense to me to keep the toolchain directive as part of this proposal. However, if it hasn't, then it seems to me that the toolchain directive should be separated into its own proposal, like per-module go.env files, until the go.env question is answered. The rest of this proposal could be adopted as-is, and the toolchain directive added later.
I don't see per-module go.env happening any time soon - there are too many problems with it! - and we need some way to set the toolchain per-module, so keeping the toolchain line seems like the way forward.