cmd/go: add 'go release'
rsc opened this issue · 31 comments
Collecting things we would want a 'go release' command to do:
- API-level compatibility checks
- warning about dependence on prerelease or pseudo-versions.
- checking that root/go.mod and root/v2/go.mod don't both say module foo/v2.
- run the tests and make sure they pass
- add a tag to the source code repo if possible and appropriate
- detects api changes and bumps version for you when incompatibility happens.
similar to http://elm-lang.org/
elm-package diff NoRedInk/json-elm-schema 1.5.0 3.0.0
Comparing NoRedInk/json-elm-schema 1.5.0 to 3.0.0...
This is a MAJOR change.
------ Changes to module JsonSchema - MAJOR ------
Added:
additionalItems : JsonSchema.Schema -> JsonSchema.TupleSchemaProperty
examples : (a -> Json.Encode.Value) -> List a -> JsonSchema.BaseSchemaProperty extras
maxProperties : Int -> JsonSchema.ObjectSchemaProperty
minProperties : Int -> JsonSchema.ObjectSchemaProperty
tuple : List JsonSchema.TupleSchemaProperty -> JsonSchema.Schema
tupleItems : List JsonSchema.Schema -> JsonSchema.TupleSchemaProperty
Changed:
- boolean : List (JsonSchema.BaseSchemaProperty {
}) -> JsonSchema.Schema
+ boolean : List JsonSchema.BooleanSchemaProperty -> JsonSchema.Schema
- maxItems : Int -> JsonSchema.ArraySchemaProperty
+ maxItems : Int -> { a |
maxItems : Maybe.Maybe Int
} -> { a | maxItems : Maybe.Maybe Int }for building automated release notes (to encourage releases to have notes):
- collect any new mentions of
Deprecated:in the documentation - collect RELNOTES from commit messages
- collect fixed bugs from commit messages
collect RELNOTES from commit messages
Isn't that a bit specific to the Go project itself? I haven't seen it in any other projects. Unless the idea is to start encouraging this practice, which might work out.
- warning about using versions excluded by other dependencies
- warning about using versions replaced by other dependencies (unless the replacements match)
- Consider a flag or mode to run tests ignoring local replace/exclude directives in the current module in order to test in the configuration users will see if they import as a dependency.
Useful if the current module is intended to be imported as a dependency but uses local replace/exclude directives that will be ignored by the ultimate top-level module.
(This is the use case described in #24666, but that was closed with a 'Fixes #24666' comment in a CL that caused the issue to be closed automatically, but the CL did not implement the described behavior. Not sure if that 'Fixes #24666' comment was accidentally or deliberately left in the broader CL).
- Consider automatically running the equivalent of:
go mod verifygo mod tidy
- Consider validating the consistency of the current vendor directory, if any (edit: this was later added as #27348)
I don't know the degree to which the intent is to automatically do things covered by other commands, but wanted to at least record these as options to consider.
- Consider a flag or mode of
go releasethat tests as if you had just donego get -uand have the latest available versions of your dependencies, but where your on-diskgo.modrequirements are not actually updated. In other words, without actually changing the versions used in yourrequiredirectives, simulate what would happen if someone downloaded your module, didgo get -u, and then tested.
The overall design of the system is such that a module author might not ready to move to the latest version of dependencies for the module's go.mod, but a module's consumers might move to a more recent set of dependencies of that module.
Coupled with the fact that the module author would already naturally be testing against versions used in the actually released require directives, the flag or mode suggested here for go release would mean the high end (latest available) and low end (versions listed in the released go.mod) would both naturally be tested by the module author before release (or at least, this would make it more easy to do so).
- warning if a
replacedirective points to a local path in the same repository whose contents have changed since therequired version of that module.
(Related to #27542.)
/cc @bradleyfalzon who created apicompat, a tool to check for the introduction of backwards incompatible changes, which is relevant here.
Recommend which kind of semver change to make, patch bump or minor bump.
Major bumps (incompatible changes) are more involved, as they require a new import path & work to allow the new major version to coexist with previous ones (e.g., by wrapping & adding aliases, or extracting package-level state into a shared internal package).
Detect when creating go.mod in a directory under an existing go.mod (is this an error?)
CC @hyangah
I think @jba is planning to do some preliminary work, but this needs more design. Pushing to 1.13.
I'm working only on the part that recommends which part of the semver to bump.
- Consider a warning or some mode to check if there are different version requirements for a shared v0 dependency.
For example, if module A requires v0.1.0 of dependency D, and module B requires v0.2.0 of dependency D, then the build will use v0.2.0 of D. However, that might subtly or explicitly break A, given v0 is a "compatibility free zone", and hence this scenario might be worth some flavor of detection or warning.
The spec/design doc for API compatibility checks is ready. Implementation is in review.
Consider a flag or mode to run tests ignoring local replace/exclude directives in the current module
See also #24666.
If package:
- has yet to adopt modules and
- is already tagged v2.0.0 or higher before adopting modules
then:
recommend that the major version should be incremented (when first adopting modules) as this seems to be the best practice[1][2]
Run tests for the new version of the API/code using the previous version of the test suite.
Requires support from the go test tool, and only black box tests must be executed.
This should catch API incompatibilities due to a change in the implementation.
- Consider a warning if you have a filesystem-based
replacedirective that reaches outside of the module being released.
In some cases, someone might deliberately want to release with a replace foo => /home/user/foo or similar, but that is probably more often going to be a mistake than purposeful, and hence perhaps worth at least a sanity check or warning by go release. If you want to edit multiple modules at the same time during development, an absolute or relative filesystem-based replace can be a common solution (in some cases via rogpeppe/gohack), but often that is a temporary change to a go.mod file that should not be released. My guess is this would not be caught in all cases by #24666.
• hard warning if the module has a transitive dependency on its own path at a version that falls semantically after the proposed tag (#31491)
-
Suggestion: consider starting
go releaseas a command ingolang.org/x/toolsor asgolang.org/x/goreleaseor similar. This might:- help with the looming 1.13 freeze deadline.
- allow for more iteration as 1.13 matures through beta, rc, 1.13.1, etc., including as community experience grows with modules and as it becomes clearer which theoretical capabilities of
go releasemight be more valuable based on the state of 1.13. - provide something of a safety net if something is slightly wrong, given it is easier to update something in
golang.org/xthan to create a new Go release.
@thepudds That's the plan right now. We're planning to develop this as golang.org/x/tools/cmd/gorelease after the 1.13 freeze starts (too much to do before then). We're hoping to have a prototype available this summer so we can collect feedback. We'll planning to eventually merge it into the Go command as go release in the 1.14 cycle.
Change https://golang.org/cl/190904 mentions this issue: x/tools/cmd/gorelease: detect problems before releasing new module versions
Change https://golang.org/cl/197299 mentions this issue: x/exp/cmd/gorelease: detect problems before releasing new module versions
Experience reports from the cl/197299 patch set 4:
Breaking out submodules is not detected. IIRC, this is supported and a process was written up somewhere (include the new submodule as an indirect dependency of the root/old module to prevent import conflicts). It is also currently being done to the cloud.google.com/go packages.
Calculating the exact change set for a PR/change is nontrivial. Currently
gorelease -base v0.0.0-20191006222123-1165772a78a0is the best option I have found, where the provided pseudo-version points to the old version, but generating psudo-versions is hard. Other attempts to resolve this include making testing tags, but they can be accidentally pushed. I would want a way to pass a vcs ref (e.g. HEAD, HEAD~1, origin/master, etc.) as base or via a distinct flag, say -ref, instead.After migrating to the new semantic import version path, e.g. /v2/, gorelease does not provide any change information from the v0/v1 line. I get why this happens, but it prevents using gorelease to generate changelogs. Sometimes it is possible to jump back in git history to before the commit that did the v1 -> v2 bump, but others there were changes after that commit and it gets tricky.
Change https://golang.org/cl/216078 mentions this issue: cmd/gorelease: infer base version if unspecified
Unassigning myself since I'm leaving, but @jadekler is still looking at this. The prototype in golang.org/x/exp/cmd/gorelease is nearly complete, and we're looking at ways to integrate it into cmd/go.
I am testing the usage of gorelease in https://github.com/open-telemetry/opentelemetry-go.
It looks like it is not handling replace and multi-module repositories well.
I am getting everywhere the following message:
# diagnostics
go.sum: one or more sums are missing. Run 'go mod tidy' to add missing sums.
I also get errors like
cannot find module providing package go.opentelemetry.io/otel/semconv/v1.21.0
which should be handled by the replace statement.
Although I assigned myself on this issue a while ago, that was more of an aspiration :) I won't be able to look into this in the near future. My apologies.