proposal: cmd/go: provide a portable and simple way to run SDK-like Go tools
eliasnaur opened this issue ยท 8 comments
The Gio library contains a module (gioui.org/ui) for creating GUI programs in Go. For desktop platforms, running a Gio program is as easy as any other Go program, provided you have a few common system libraries and a C compiler installed:
$ go run gioui.org/apps/hello
Running Gio programs on Android, iOS and in the browser requires packaging several support files and meta data, so the Gio project includes the gogio
tool in the `gioui.org/cmd module to automate the tedious work.
Note that the gogio
tool is not general purpose: it is intrinsically bound to the gioui.org/ui module. Similar examples are the gobind
and gomobile
tools from the Gomobile project.
This issue is about providing an easy and portable way to run such support tools.
In #33468 I described a possible solution in terms of go run
because go run
is very close to what I want. This is the simple and portable one-liner for creating an Android app for a Gio program:
$ go run gioui.org/cmd/gogio -target android gioui.org/apps/gophers
Unfortunately, according to #33468 (comment), #25416 and #25416 (comment), it is almost accidental that go run
can run the above one-liner.
go install
@bcmills brings up the usual way to use Go tools: go install
and setting up $PATH to run them:
You could give separate instructions for cmd.exe and for Unix-like shells. Or assume that they have their PATH configured appropriately (perhaps by reference to some other document) and tell everyone:
go install gioui.org/cmd/gogio
gio -target android gioui.org/apps/gophers
However, setting up PATH is not portable and not nearly as simple as go run
, in particular for Windows users.
Case in point: go env
can now set environment variables for the Go tool because (#30411)
Setting environment variables for go command configuration
is too difficult and system-specific.
go build
#25416 (comment) suggests
$ go build -o /some/dir/gogio gioui.org/cmd/gogio
$ /some/dir/gogio ...
which doesn't depend on the environment, but still more awkward than just go run
.
(Some) Solutions
- Let
go run
be the way to conveniently run Go commands without fiddling with the environment. #33468 is about speeding upgo run
so that it is (nearly) as fast as running ago install
'ed binary.go run
is also the only way to get reproducible builds; see #33518 (comment). - Set up the user's environment when installing Go so that
go install
'ed programs are guaranteed to be in PATH. - As a variant to the above, provide a portable and simple way to run a
go install
'ed binary that doesn't depend on the environment.
In #33468 (comment), @bcmills brings up an interesting point:
When working within a module in module mode,
go run
should produce a reproducible result, not always upgrade to the latest version.
I agree that within a module the result should be reproducible which is the reason go run
records the version of the command in a current module's go.mod.
However, that means that within a module, go run
ning gio
is superior to running a previously go install
'ed version, just because the tool version is reproducible with go run
, but not with the binary that happens to be in GOBIN.
So go install
is not always the correct choice, even if we assume PATH is correctly set up.
As a variant to the above, provide a portable and simple way to run a go install'ed binary that doesn't depend on the environment.
It is possible to get the path to the installed binary in a portable way, with something like:
go list -f {{.Target}} some/main/pkg
Note that this will still resolve the version and do extra work outside a module, so it's not instantaneous.
It is possible to get the path to the installed binary in a portable way, with something like:
go list -f {{.Target}} some/main/pkg
Note that this will still resolve the version and do extra work outside a module, so it's not instantaneous.
Thank you. My current thinking is that the speed of go run
outside a module is not that important; most actual uses will be from inside a module.
Unfortunately, according to #33468 (comment), #25416 and #25416 (comment), it is almost accidental that go run can run the above one-liner.
It seems like your use cases for running this tool would always be within a module, though -- it's only relevant when developing programs using your giou.org/ui module. And for that, go run (or go build blah && ./blah
, for caching the executable) seem just right.
ISTM that #31173 would solve this reasonably, particularly if combined with an idea from @ianthehat of being able to pass a -modfile
argument to the go
command to ask it to use a different go.mod
file .
A random list of points to add to the conversation, followed by a question on how we should move forward:
- #30515 (comment) represents the most recent discussion about this from a golang-tools perspective
- in the golang-tools discussions, it was generally agreed/observed that the "within a module" use case is one part of the problem/solution space: being able to install/run "globally" (i.e. "outside of a module") is also a problem that needs solving. Indeed @ianthehat sort to define this more precisely:
The concrete use cases people have suggested so far are:
- go code generators (often from go generate lines) which must be versioned at possibly per invocation granularity
- code analysis tools (the kinds of things we are trying to merge into gopls, an also gopls itself of course) which may need to be versioned per project
- System wide installs of binaries, for the kinds of instructions people put on their wiki
- per previous threads with @bcmills, @rsc and others, it's not clear to me that these changes have to be made to
cmd/go
. @rsc made the point (somewhere I can't now recall) that trying to load too much intocmd/go
effectively has costs associated with it, costs that aren't necessarily outweighed by the cost of putting this functionality in a different command. I have definitely come to appreciate this point more, although the attraction of having everything incmd/go
remains - if the changes are not made as part of
cmd/go
, then the command within which the functionality sits must ultimately be distributed with Go (a lagodoc
in the old days): the overhead of installing a command to install/run other commands is too high. This obviously doesn't preclude developing something inx/tools
as a starting point gobin
is an experiment that covers the cases of "within a module" and "outside a module". It has been largely successful in acting as a hybrid run/install command that sits atopcmd/go
.gobin
respectsreplace
directives (and could respectexclude
directives, this is simply not implemented)gobin
seeks to optimise the cost of running main packages. It is faster thango run
, but currently nowhere near as fast as running something fromPATH
. However it does obviate the need for settingPATH
which is a big win. I'm sure there is more scope for optimisation here, particularly if we can start to build on assumptions about the pristineness of the module cache (arguably if the user messes with the module cache it's their own problem)- one conclusion we (read @myitcv, @rogpeppe and @mvdan) have (almost) reached with
gobin
to date is that in the "within a module" scenario, polluting the main module with tool dependencies is not good. myitcv/gobin#81 proposes the use of nested modules for this reason - whatever and wherever the solution lies:
- it must be able to support further
replace
andexclude
directives, and build flags/constraints - it must be possible to cross compile
- version information for all modules (including the main package's module) must be available in the resulting binary
- it must be able to support further
Next steps
This issue has been rumbling in the background for some time now and it seems, to my mind, worthy of tackling given the plethora of problems/issues/common stumbling blocks it would help to avoid (arguments along similar lines to why go env -w
is preferable to people setting environment variables). Continuing some sort of gobin
-like experiment in x/tools
feels like a good next step, regardless of whether that work is adopted into cmd/go
or not.
@rsc, @bcmills, @ianthehat - what do you see as the next steps? Any thoughts on how best to proceed from here?