bingo
go get
like, simple CLI that allows automated versioning of Go package level binaries (e.g required as dev tools by your project!) built on top of Go Modules, allowing reproducible dev environments.
Features
From our experience all repositories and projects require some tools and binaries to be present on the machine to be able to perform various development operations like building, formatting, releasing or static analysis. For smooth development all such tools should be pinned to a certain version and bounded to the code commits there were meant to be used against.
Go modules does not aim to solve this problem, and even if they will do at some point it will not be on the package level, which makes it impossible to e.g pin minor version X.Y.0
of package module1/cmd/abc
and version X.Z.0
of module1/cmd/def
.
At the end bingo
, has following features:
- It allows maintaining separate, hidden, nested Go modules for Go buildable packages you need without obfuscating your own module or worrying with tool's cross dependencies!
- Package level versioning, which allows versioning different (or the same!) package multiple times from a single module in different versions.
- Works also for non-Go projects. It only requires the tools to be written in Go.
- No need to install
bingo
in order to use pinned tools. This avoids the "chicken & egg" problem. You only needgo build
. - Easy upgrade, downgrade, addition, and removal of the needed binary's version, with no risk of dependency conflicts.
- NOTE: Tools are often not following semantic versioning, so
bingo
allows to pin by commit ID.
- NOTE: Tools are often not following semantic versioning, so
- Immutable binary names. This creates a reliable way for users and CIs to use expected version of the binaries, reinstalling on-demand only if needed.
- Works with all buildable Go projects, including pre Go modules and complex projects with complex directives like
replace
,retract
orexclude
statements. (e.g Prometheus) - Optional, automatic integration with Makefiles.
You can read full a story behind bingo
in this blog post.
Requirements
- Go 1.17+
- Linux or MacOS (Want Windows support? Helps us out)
- All tools that you wish to "pin" have to be built in Go (they don't need to use Go modules at all).
Installing
In your repository (does not need to be a Go project)
go install github.com/bwplotka/bingo@latest
For go version before 1.17 use
go get github.com/bwplotka/bingo
instead.
Recommended: Ideally you want to pin bingo
tool to the single version too (inception!). Do it via:
bingo get -l github.com/bwplotka/bingo
Usage
go get
but for binaries!
The key idea is that you can manage your tools similar to your Go dependencies via go get
:
bingo get [<package or binary>[@version1 or none,version2,version3...]]
For example:
bingo get github.com/fatih/faillint
bingo get github.com/fatih/faillint@latest
bingo get github.com/fatih/faillint@v1.5.0
bingo get github.com/fatih/faillint@v1.1.0,v1.5.0
After this, make sure to commit .bingo
directory in git repository, so the tools will stay versioned! Once pinned, anyone can install correct version of the tool with correct dependencies by either doing:
bingo get <tool>
For example bingo get faillint
... or without bingo
:
go build -mod=mod -modfile .bingo/<tool>.mod -o=$GOBIN/<tool>-<version>
For example go build -mod=mod -modfile .bingo/faillint.mod -o=$GOBIN/faillint-v1.5.0
bingo
allows to easily maintain a separate, nested Go Module for each binary. By default, it will keep it .bingo/<tool>.mod
This allows to correctly pin the binary without polluting the main go module or other's tool module.
Using Installed Tools
bingo get
builds pinned tool or tools in your $GOBIN
path. Binaries have a name following <provided-tool-name>-<version>
pattern. So after installation you can do:
- From shell:
${GOBIN}/<tool>-<version> <args>
For example: ${GOBIN}/faillint-v1.5.0
While it's not the easiest for humans to read or type, it's essential to ensure your scripts use pinned version instead of some non-deterministic "latest version".
NOTE: If you use
-l
option, bingo creates symlink to . Use it with care as it's easy to have side effects by having another binary with same name e.g on CI.
bingo
does not have run
command (for a reason), it provides useful helper variables for script or adhoc use:
NOTE: Below helpers makes it super easy to install or use pinned binaries without even installing
bingo
(it will use justgo build
!) 💖
- From shell:
source .bingo/variables.env
${<PROVIDED_TOOL_NAME>} <args>
- From Makefile:
include .bingo/Variables.mk
run:
$(<PROVIDED_TOOL_NAME>) <args>
Real life examples!
Let's show a few, real, sometimes novel examples showcasing bingo
capabilities:
-
golangci-lint
is all-in-one lint framework. It's important to pin it on CI so CI runs are reproducible no matter what new linters are added, removed or changed in new release. Let's pin it tov1.35.2
and use path recommended by https://golangci-lint.run/usage/install/#install-from-source doc:github.com/golangci/golangci-lint/cmd/golangci-lint
(funny enough they discouragego get
exactly because of the lack of pinning featuresbingo
have!)bingo get github.com/golangci/golangci-lint/cmd/golangci-lint@v1.35.2
This will pin to that commit and install
${GOBIN}/golangci-lint-v1.35.2
-
It's very common in Go world to use
goimports
, populargofmt
replacement which formats Go code including imports. However, not many know that it's breaking compatibility a lot between versions (there are no releases). If you want to assert certain formatting of the Go code in the CI etc your only option is to pingoimports
version. You can do it viabingo get
:bingo get golang.org/x/tools/cmd/goimports@latest
This will install (at the time of writing) latest binary:
${GOBIN}/goimports-v0.0.0-20210112230658-8b4aab62c064
-
You rather like older formatting? No issue, let's downgrade. Since
goimports
was already installed you can reference it by justgoimports
. Let's pick the commit we want e.ge64124511800702a4d8d79e04cf6f1af32e7bef2
:bingo get goimports@e64124511800702a4d8d79e04cf6f1af32e7bef2
This will pin to that commit and install
${GOBIN}/goimports-v0.0.0-20200519204825-e64124511800
-
Installing (and pinning) multiple versions:
bingo get goimports@e64124511800702a4d8d79e04cf6f1af32e7bef2,v0.0.0-20200601175630-2caf76543d99,af9456bb636557bdc2b14301a9d48500fdecc053
This will pin and install three versions of goimports. Very useful to compatibility testing.
-
Updating to the current latest:
bingo get goimports@latest
This will find the latest module version, pin and install it.
-
Listing binaries you have pinned:
bingo list
-
Unpinning
goimports
totally from the project:bingo get goimports@none
PS:
go get
also allows@none
suffix! Did you know? I didn't (:* -
Installing all tools:
bingo get
-
Bonus: Have you ever dreamed to pin command from bigger project like...
thanos
? I was. Can you even install it using Go tooling? Let's try:go get github.com/thanos-io/thanos/cmd/thanos@v0.17.2 # Output: go: cannot use path@version syntax in GOPATH mode
Ups you cannot use this in non-Go project at all... (: Let's create setup go mod and retry:
go mod init _ # Output: go: creating new go.mod: module go get github.com/thanos-io/thanos/cmd/thanos@v0.17.2 # go get github.com/thanos-io/thanos/cmd/thanos@v0.17.2 # go: downloading github.com/thanos-io/thanos v0.17.2 # go: found github.com/thanos-io/thanos/cmd/thanos in github.com/thanos-io/thanos v0.17.2 # go get: github.com/thanos-io/thanos@v0.17.2 requires # github.com/cortexproject/cortex@v1.5.1-0.20201111110551-ba512881b076 requires # github.com/thanos-io/thanos@v0.13.1-0.20201030101306-47f9a225cc52 requires # github.com/cortexproject/cortex@v1.4.1-0.20201030080541-83ad6df2abea requires # github.com/thanos-io/thanos@v0.13.1-0.20201019130456-f41940581d9a requires # github.com/cortexproject/cortex@v1.3.1-0.20200923145333-8587ea61fe17 requires # github.com/thanos-io/thanos@v0.13.1-0.20200807203500-9b578afb4763 requires # github.com/cortexproject/cortex@v1.2.1-0.20200805064754-d8edc95e2c91 requires # github.com/thanos-io/thanos@v0.13.1-0.20200731083140-69b87607decf requires # github.com/cortexproject/cortex@v0.6.1-0.20200228110116-92ab6cbe0995 requires # github.com/prometheus/alertmanager@v0.19.0 requires # github.com/prometheus/prometheus@v0.0.0-20190818123050-43acd0e2e93f requires # k8s.io/client-go@v12.0.0+incompatible: reading https://proxy.golang.org/k8s.io/client-go/@v/v12.0.0+incompatible.mod: 410 Gone # server response: not found: k8s.io/client-go@v12.0.0+incompatible: invalid version: +incompatible suffix not allowed: module contains a go.mod file, so semantic import versioning is required
The reasoning is complex but TL;DR: Go Modules are just sometimes hard to be properly used for some projects. This is why bigger projects like
Kubernetes
,Prometheus
orThanos
has to usereplace
statements (plus others likeexclude
orretract
). To make thisgo get
work we would need to manually craftreplace
statements in our own gomod
file. But what if we don't want to do that or don't know how or simply we want to install pinned version of Thanos locally without having Go project? Just use bingo:bingo get github.com/thanos-io/thanos/cmd/thanos@v0.17.2 ${GOBIN}/thanos-v0.17.2 --help
Advanced Techniques
- Using advanced go build flags and environment variables.
To tell bingo to use certain env vars and tags during build time, just add them as a comment to the go.mod file manually and do bingo get
. Done!
NOTE: Order of comment matters. First bingo expects relative package name (optional), then environment variables, then flags. All space delimited.
Real example from production project that relies on extended Hugo.
module _ // Auto generated by https://github.com/bwplotka/bingo. DO NOT EDIT
go 1.16
require github.com/gohugoio/hugo v0.83.1 // CGO_ENABLED=1 -tags=extended
Run bingo list
to see if build options are parsed correctly. Run bingo get
to install all binaries including the modified one with new build flags.
Production Usage
To see production example see:
Contributing
Any contributions are welcome! Just use GitHub Issues and Pull Requests as usual. We follow Thanos Go coding style guide.
See an extensive and up-to-date description of the bingo
usage below:
Command Help
bingo: 'go get' like, simple CLI that allows automated versioning of Go package level binaries (e.g required as dev tools by your project!)
built on top of Go Modules, allowing reproducible dev environments. 'bingo' allows to easily maintain a separate, nested Go Module for each binary.
For detailed examples and documentation see: https://github.com/bwplotka/bingo
'bingo' supports following commands:
Commands:
get <flags> [<package or binary>[@version1,none,latest,version2,version3...]]
-go string
Path to the go command. (default "go")
-insecure
Use -insecure flag when using 'go get'
-l If enabled, bingo will also create soft link called <tool> that links to the current<tool>-<version> binary. Use Variables.mk and variables.env if you want to be sure that what you are invoking is what is pinned.
-moddir string
Directory where separate modules for each binary will be maintained. Feel free to commit this directory to your VCS to bond binary versions to your project code. If the directory does not exist bingo logs and assumes a fresh project. (default ".bingo")
-n string
The -n flag instructs to get binary and name it with given name instead of default, so the last element of package directory. Allowed characters [A-z0-9._-]. If -n is used and no package/binary is specified, bingo get will return error. If -n is used with existing binary name, copy of this binary will be done. Cannot be used with -r
-r string
The -r flag instructs to get existing binary and rename it with given name. Allowed characters [A-z0-9._-]. If -r is used and no package/binary is specified or non existing binary name is used, bingo will return error. Cannot be used with -n.
-v Print more'
list <flags> [<package or binary>]
List enumerates all or one binary that are/is currently pinned in this project. It will print exact path, Version and immutable output.
-moddir string
Directory where separate modules for each binary is maintained. If does not exists, bingo list will fail. (default ".bingo")
-v Print more'
version
Prints bingo Version.
Initial Author
@bwplotka inspired by Paul's research and with a bit of help from Duco (: