Go modules Dev Dependencies
bcomnes opened this issue · 5 comments
Go didn't have a concept of devDependencies like node.js does for a long time. They are sort of addressing this problem in a very go-esque way it sounds.
-
Go modules has something called a tools.go file: golang/go#25922 (comment)
-
Details on go module support for vendoring golang/go#27618
-
Experimental proposal thread: myitcv/gobin#44
-
gex: a tool to help manage devDeps: https://github.com/izumin5210/gex
-
https://github.com/srvc/wraperr: a project using tools.go
-
https://github.com/izumin5210/grapi: another project using tools.go pattern
-
https://github.com/oxequa/realize A go task runner
-
golang/go#29494 Another example of someone asking the same issue
-
golang/go#27653: issue addressing some of the complications with the current solutions, canns the idea of devDeps beyond what's supported.
-
Go get outside of a module scope for global deps: golang/go#24250
-
Retool: https://github.com/twitchtv/retool vendoring go bins
-
https://github.com/myitcv/gobin go bin, a proposed solution
-
https://github.com/myitcv/gobin/wiki/FAQ gobin fan
-
go 1.12 modules https://tip.golang.org/doc/go1.12#modules
-
Go tools.go https://github.com/go-modules-by-example/index/blob/master/010_tools/README.md
-
https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
-
https://godoc.org/golang.org/x/tools/cmd/stringer a tool often used as an example
Another solution: https://github.com/nordicdyno/golang-tools
Both tools.go and golang-tools have problems, due to being third party systems.
They rely on make + bash + awk to parse Go code. That's not particularly maintainable.
Nevermind the complexity of setting up a Go project for contribution on native Windows COMSPEC (Command Prompt or PowerShell) environments. I may love the WSL effort, but I don't like having requiring anyone to install WSL when working on my projects. I'm not writing polyglot scripts with tofrodos hacks. Please don't suggest batsh, or worse, Python, as a shim.
You might as well just install the tools in a Dockerfile, and always work in Docker. These workflows are heavyweight, compared to regular, built-in go mod
support. One of Go's best qualities is that the built-in functionality is usually very comprehensive and very cross-platform, not requiring external systems or any specific shell interpreter.
You can't simply reuse the go mod path when determining the right path as far as go get
is concerned. With these hacks, It's difficult to distinguish between binaries that require explicit ellipses (...
) during installation (for multiple binaries), versus ordinary top-level artifacts.
go get -u
is highly recommended for any x items from the Go extension collection, that may break across Go versions. But may or may not be desired for other items. Some developers may prefer to assume good Go semver and automatically float tool versions. Other developers know just how poorly real projects implement semver, and will prefer to pin tool versions down to a VCS ref, just in case of any remote possibility of breaking changes. So handling even just the -u
flag appropriately per tool, can be challenging from the perspective of the existing go.mod
spec.
Another problem with the third party systems is that each change in developer tools would require two different file changes: One to go.mod
and another to tools.go
. Three if you include the checksum file. That's a code smell. Caused by the Go language's resistance to implementing NPM/RubyGems pre-existing ability to denote development dependencies, particularly support for making sure that executable artifacts get pulled in correctly into $GOPATH/bin.
The most reliable system I've come up with so far is a POSIX sh script, like accio
. It sets a POSIX shebang, enables -e. And then it runs a series of go get [-u]
commands. With git checkout
commands interspersed to enforce pinning. Finally, a go mod tidy
command to fix the go.mod
file. Super nasty, twice as slow as it should be (due to double-checking out the source code). And requires the user to have a POSIX shell available. And you have to remember to use LF endings. For COMSPEC users, they have to copy and paste the go get
lines into their terminal session, in order to prevent the installations from occuring in the wrong file system. Executable permission is added for convenience, but sh accio
is the recommended invocation syntax, to try to widen support for non-UNIX hosts. No support for tools housed in non-git version control repositories. Poor support for x collection URL's. Honestly, the pinning process is so arduous to manage external to go mod, that I'd rather just float all of my tools to latest and take the risk of breakages and malware.
You could in theory automate these things with an insane go:generate setup, or write yet another third party tool. (Which inevitably breaks as the Go language releases new versions.) I might have a try at using gotpl
to simplify this. But pinning the gotpl version itself creates a chicken and egg problem. Same if using Mage. Maybe the Go template lib, without actually invoking the CLI helper? I dunno. All of these suggestions probably pollute the local file tree with redundant copies between any vendor caches versus the larger $GOPATH/src
tree. Unfortunately, Go has stretched the meaning of the path in go get <path>
to the limit, impossible to reuse this path without significant handling of different cases.
Not. Fun.
We want the normal go mod
system to actually implement tool management as a first party system, in order to reliably resolve all of the edge cases without requiring massive extra components scrambling to solve half of them.
Yeah me to. Looks like go run @Version is in though, probably could work something out with that
I am automating the low level go get
, git checkout
commands with a more pure Go system, accio
. No need for flakey shell scripts, or makefiles, or awk, or other such nonsense. Just the Go programming language, and git
.
You may now use an accio.yaml configuration file to pin your Go developer tools which go mod
has not deigned to track. It's so crude, but for my projects it's a kind of local max for convenience and reproducible builds.
That's cool I'll try it out next time I'm on a go project.