golang/go

cmd/go: go install: don’t fail when go.mod can’t be updated on a read-only system

stapelberg opened this issue · 6 comments

stapelberg commented 
$ go version
go version go1.13beta1 linux/amd64

I talked about this issue with @bcmills over GopherCon 2019 and he recommended I file this issue for tracking.

I am using cmd/go in a build environment where the entire system is read-only, but I have a writable copy of the source tree I’m building (e.g. github.com/junegunn/fzf). The system provides a read-only module cache (at /ro/gopath/pkg/mod) which contains all the dependencies of the source tree.

I’m using replace directives to enforce that my environment’s module versions are used, which can be different from the versions listed in go.mod/go.sum. See #29410 (comment) for details.

When running go install, it fails to lock the module cache because it’s read-only:

go: failed to lock file at /ro/gopath/pkg/mod/cache/lock

So far so good—it’s reasonable to not continuing when the side lock can’t be acquired. But, when specifying -mod=readonly, I’m getting:

go: updates to go.mod needed, disabled by -mod=readonly

I don’t know that go install should error out here. Certainly it could just continue with whatever changes needed just being retained in memory? As a user, I’m interested in the binary installation, not the side effect updates.

Thanks for considering,

gopherbot commented

Change https://golang.org/cl/204521 mentions this issue: cmd/go: default to mod=readonly when the go.mod file is read-only

gopherbot commented

Change https://golang.org/cl/204878 mentions this issue: cmd/go: make commands other than 'tidy' prune go.mod less agressively

bcmills commented

it fails to lock the module cache because it’s read-only:

That should be fixed by https://golang.org/cl/205637, which is now in.

@stapelberg, provided that the go.sum file is complete (or complete enough) and the go.mod file is consistent, this should now be fixed at head (and in Go 1.14 when that is released). Please give it a spin and let us know if there are any gaps remaining.

gopherbot commented

Change https://golang.org/cl/210341 mentions this issue: cmd/go: include cfg.BuildModReason in 'import lookup disabled' errors

stapelberg commented

Thanks for the changes! Just got around to giving this a spin with the now-available Go 1.14 beta 1 release.

Unfortunately, I still get the same error message when running go install:

2019/12/25 20:38:56 build step 4 of 5: [/bin/sh -c GOSUMDB=off GOCACHE=/tmp/throwaway GOPATH=/ro/gopath GOPROXY=off  GOBIN=/dest/tmp//ro/fzf-amd64-v0.0.0-20190719042446-65773882505b-4/out/bin go install -mod=readonly -v ]
go: updates to go.mod needed, disabled by -mod=readonly

Running go mod tidy before gets the go install command to work, but requires a writable GOPATH with a populated module cache (what I’m trying to get away from) because it does module lookup.

I compared the go.mod file before and after a go mod tidy run to see which changes the go tool thinks it needs:

--- before	2019-12-25 20:37:30.061450310 +0100
+++ after	2019-12-25 20:37:35.951455273 +0100
@@ -1,19 +1,11 @@
 module github.com/junegunn/fzf
 
 require (
-	github.com/gdamore/encoding v0.0.0-20151215212835-b23993cbb635 // indirect
 	github.com/gdamore/tcell v0.0.0-20170915061752-0a0db94084df
-	github.com/gopherjs/gopherjs v0.0.0-20181103185306-d547d1d9531e // indirect
-	github.com/jtolds/gls v4.2.1+incompatible // indirect
-	github.com/lucasb-eyer/go-colorful v0.0.0-20170223221042-c900de9dbbc7 // indirect
 	github.com/mattn/go-isatty v0.0.0-20160806122752-66b8e73f3f5c
-	github.com/mattn/go-runewidth v0.0.0-20170201023540-14207d285c6c
+	github.com/mattn/go-runewidth v0.0.4
 	github.com/mattn/go-shellwords v1.0.3
-	github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d // indirect
-	github.com/smartystreets/goconvey v0.0.0-20181108003508-044398e4856c // indirect
-	golang.org/x/crypto v0.0.0-20170728183002-558b6879de74
-	golang.org/x/sys v0.0.0-20190509141414-a5b02f93d862 // indirect
-	golang.org/x/text v0.0.0-20170530162606-4ee4af566555 // indirect
+	golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
 )
 
 replace golang.org/x/tools => golang.org/x/tools v0.0.0-20190311212946-11955173bddd
@@ -49,3 +41,5 @@
 replace github.com/gdamore/encoding => github.com/gdamore/encoding v1.0.0
 
 replace github.com/DATA-DOG/go-sqlmock => github.com/DATA-DOG/go-sqlmock v1.3.3
+
+go 1.13

Whole go.mod files:

before 
module github.com/junegunn/fzf

require (
	github.com/gdamore/encoding v0.0.0-20151215212835-b23993cbb635 // indirect
	github.com/gdamore/tcell v0.0.0-20170915061752-0a0db94084df
	github.com/gopherjs/gopherjs v0.0.0-20181103185306-d547d1d9531e // indirect
	github.com/jtolds/gls v4.2.1+incompatible // indirect
	github.com/lucasb-eyer/go-colorful v0.0.0-20170223221042-c900de9dbbc7 // indirect
	github.com/mattn/go-isatty v0.0.0-20160806122752-66b8e73f3f5c
	github.com/mattn/go-runewidth v0.0.0-20170201023540-14207d285c6c
	github.com/mattn/go-shellwords v1.0.3
	github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d // indirect
	github.com/smartystreets/goconvey v0.0.0-20181108003508-044398e4856c // indirect
	golang.org/x/crypto v0.0.0-20170728183002-558b6879de74
	golang.org/x/sys v0.0.0-20190509141414-a5b02f93d862 // indirect
	golang.org/x/text v0.0.0-20170530162606-4ee4af566555 // indirect
)

replace golang.org/x/tools => golang.org/x/tools v0.0.0-20190311212946-11955173bddd

replace golang.org/x/text => golang.org/x/text v0.3.2

replace golang.org/x/sys => golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b

replace golang.org/x/sync => golang.org/x/sync v0.0.0-20190423024810-112230192c58

replace golang.org/x/net => golang.org/x/net v0.0.0-20190724013045-ca1201d0de80

replace golang.org/x/crypto => golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2

replace github.com/smartystreets/goconvey => github.com/smartystreets/goconvey v0.0.0-20190710185942-9d28bd7c0945

replace github.com/smartystreets/assertions => github.com/smartystreets/assertions v1.0.1

replace github.com/mattn/go-shellwords => github.com/mattn/go-shellwords v1.0.5

replace github.com/mattn/go-runewidth => github.com/mattn/go-runewidth v0.0.4

replace github.com/mattn/go-isatty => github.com/mattn/go-isatty v0.0.8

replace github.com/lucasb-eyer/go-colorful => github.com/lucasb-eyer/go-colorful v1.0.2

replace github.com/jtolds/gls => github.com/jtolds/gls v4.20.0+incompatible

replace github.com/gopherjs/gopherjs => github.com/gopherjs/gopherjs v0.0.0-20190430165422-3e4dfb77656c

replace github.com/gdamore/tcell => github.com/gdamore/tcell v1.1.4

replace github.com/gdamore/encoding => github.com/gdamore/encoding v1.0.0

replace github.com/DATA-DOG/go-sqlmock => github.com/DATA-DOG/go-sqlmock v1.3.3
after 
module github.com/junegunn/fzf

require (
	github.com/gdamore/tcell v0.0.0-20170915061752-0a0db94084df
	github.com/mattn/go-isatty v0.0.0-20160806122752-66b8e73f3f5c
	github.com/mattn/go-runewidth v0.0.4
	github.com/mattn/go-shellwords v1.0.3
	golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
)

replace golang.org/x/tools => golang.org/x/tools v0.0.0-20190311212946-11955173bddd

replace golang.org/x/text => golang.org/x/text v0.3.2

replace golang.org/x/sys => golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b

replace golang.org/x/sync => golang.org/x/sync v0.0.0-20190423024810-112230192c58

replace golang.org/x/net => golang.org/x/net v0.0.0-20190724013045-ca1201d0de80

replace golang.org/x/crypto => golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2

replace github.com/smartystreets/goconvey => github.com/smartystreets/goconvey v0.0.0-20190710185942-9d28bd7c0945

replace github.com/smartystreets/assertions => github.com/smartystreets/assertions v1.0.1

replace github.com/mattn/go-shellwords => github.com/mattn/go-shellwords v1.0.5

replace github.com/mattn/go-runewidth => github.com/mattn/go-runewidth v0.0.4

replace github.com/mattn/go-isatty => github.com/mattn/go-isatty v0.0.8

replace github.com/lucasb-eyer/go-colorful => github.com/lucasb-eyer/go-colorful v1.0.2

replace github.com/jtolds/gls => github.com/jtolds/gls v4.20.0+incompatible

replace github.com/gopherjs/gopherjs => github.com/gopherjs/gopherjs v0.0.0-20190430165422-3e4dfb77656c

replace github.com/gdamore/tcell => github.com/gdamore/tcell v1.1.4

replace github.com/gdamore/encoding => github.com/gdamore/encoding v1.0.0

replace github.com/DATA-DOG/go-sqlmock => github.com/DATA-DOG/go-sqlmock v1.3.3

go 1.13

I’m not sure why the Go tool thinks it needs to update anything—I thought fully specifying the versions using replace directives should have been enough.

Let me know if I should file a separate issue for this, but the intention and steps to reproduce are still the same :)

gopherbot commented

Change https://golang.org/cl/219237 mentions this issue: cmd/go: eliminate empty '()' when passing -mod=readonly explicitly to 'go list'