ethereum/hive

Unable to use `go mod replace` since workspaces without causing havoc

marioevz opened this issue · 5 comments

marioevz commented

Since the upgrade to use workspaces, it seems like any version replacements in a go.mod file within a simulator is affecting and replacing the version on other module.

For example in branch https://github.com/marioevz/hive/tree/eth2-withdrawals-interop i'm using lightclient's version of geth that introduces withdrawals to all types and apis.

However with this replacement, upper modules such as cmd/hivechain now fail to compile because the version has been replaced for its go.mod file too.

I'm currently unable to merge the withdrawals tests or eth2/common even because they all require the withdrawals constructs, and I've been debugging for days and trying many workarounds to only partial success (breaking something else in the process).

I'm opening this issue for discussion and suggestions since I rely a lot on replace to write tests for new features, and I believe this will cause us issues for 4844 testing in the near future too.

cc @fjl @protolambda

marioevz commented

Possible solutions I've thought so far:

  • Dropping dependency of go-ethereum altogether and write all required features within a shared module in this repo (requires implementing types.Block, types.Header, engine api constructs, etc.).

  • Another option is committing to the replace on a workspace level and simply fixing all other simulators too when replace is used, but I think this is a bit hacky and does not address the problem (we might hit incompatible replacements in the future).

Any suggestions will be appreciated.

fjl commented

This feels like a downside of the project-wide workspace to me. As you noted, there is no easy fix right now. Another part of the problem is us making so many incompatible changes in go-ethereum that end up breaking downstream projects like hive.

protolambda commented

If a sub-module uses a different incompatible version of a library than the rest of the project, it shouldn't be put in the global workspace (go.work), or temporarily removed until the changes become canonical & adopted across the hive project.

marioevz commented

The replace I used was in a single go.mod in a simulator, but it somehow ends up replacing go-ethereum on other upper modules too.

I tried using a replace that is constrained to a specific version, latest commit at go-ethereum/master (which for some reason shows up as v1.10.24-0.20230115082358-450d771bee9c in go instead of v1.10.26...), and unless I specifically use exclude also in the go.mod file, the module always falls back to using the released v1.10.26.
I think this is because one of the modules required by go-ethereum, github.com/status-im/keycard-go, requires v1.10.26, but I have no proof to back this up.

So in summary, my approaches were:

  • require github.com/ethereum/go-ethereum v1.10.26, replace replace github.com/ethereum/go-ethereum v1.10.26 => ..., all simulators/hive modules do the replace.
  • require github.com/ethereum/go-ethereum v1.10.24-0.20230115082358-450d771bee9c, replace github.com/ethereum/go-ethereum v1.10.24-0.20230115082358-450d771bee9c => ..., the simulator where the replace is used fallbacks to v1.10.26, and nothing gets replaced anywhere.
  • require github.com/ethereum/go-ethereum v1.10.24-0.20230115082358-450d771bee9c, replace github.com/ethereum/go-ethereum v1.10.24-0.20230115082358-450d771bee9c => ..., exclude github.com/ethereum/go-ethereum v1.10.26, all simulators/hive modules do the replace.

I think this could even be a bug in go, but I'm not too confident about it.

fjl commented

This was fixed in #681