New submodules: upgrade before 2024-03-01 to avoid ambiguous import errors with google.golang.org/genproto
quartzmo opened this issue · 4 comments
TL;DR
Upgrade to a version of google.golang.org/genproto at or above version v0.0.0-20230526161137-0005af68ea54 with the following command:
go get google.golang.org/genproto@latest
go mod tidy
Doing this before the 9-month deadline of 2024-03-01 will avoid "ambiguous import" errors.
Background
Many of the packages in the google.golang.org/genproto module represent API surfaces belonging to Google Cloud. Since approximately the v0.0.0-20220921223823-23cae91e6737 release of google.golang.org/genproto, a change in #901 introduced dependencies on virtually every submodule in cloud.google.com/go. This change moved type definitions for Google Cloud client libraries to cloud.google.com/go, greatly improving usability of the latter. To maintain compatibility with existing code, aliases were introduced into google.golang.org/genproto that pointed to the new sources of truth in cloud.google.com/go. This added the dependencies on cloud modules.
Even though most packages in google.golang.org/genproto are Google Cloud, a few packages provide simple “common” types used in projects unrelated to Google Cloud. For users of the simpler packages, this commingled existence resulted in a large number of extra cloud modules appearing in the dependency graph. This can trigger alarms in automatic dependency checks, false positives in vulnerability checks, and adds constraints to minimum version selection when resolving acceptable versions of other dependencies.
New submodules in this repo
To improve usability for consumers that do not depend on Google Cloud modules, packages providing widely used "common" types will be published as submodules. The following packages do not depend on cloud types and are now isolated in new non-cloud submodules:
google.golang.org/genproto/googleapis/api(Note: this does not include 4 subpackages referencing cloud types, listed below)google.golang.org/genproto/googleapis/bytestreamgoogle.golang.org/genproto/googleapis/rpc
The following subpackages of google.golang.org/genproto/googleapis/api reference cloud types and are now isolated from the api package as new cloud submodules:
google.golang.org/genproto/googleapis/api/apikeysgoogle.golang.org/genproto/googleapis/api/servicecontrolgoogle.golang.org/genproto/googleapis/api/servicemanagementgoogle.golang.org/genproto/googleapis/api/serviceusage
To ease migration, these submodules initially have a “back-ref” dependency on the google.golang.org/genproto parent. After a 9-month period, this "back-ref" dependency will be removed, making it possible to once again depend on the simpler packages of this project without adding cloud modules to the dependency graph.
Avoiding ambiguous imports for submodules
The root package docs for google.golang.org/genproto state:
IMPORTANT This repository is currently experimental. The structure of the contained packages is subject to change. Please see the original source repositories (listed below) to find out the status of each protocol buffer's associated service.
In addition to this warning, the pseudo version numbering of the module:
Signals that the module is still in development and unstable. This release carries no backward compatibility or stability guarantees.
However, despite these warnings, we know that critical infrastructure depends on many of the packages in google.golang.org/genproto. To avoid "ambiguous import" errors for consumers, we are adding a “back-ref” dependency on the google.golang.org/genproto parent to each new submodule for 9 months to allow a seamless transition.
After the 9-month deadline of 2024-03-01 has passed, the “back-ref” dependency on the google.golang.org/genproto parent will be removed from each new submodule. Client code still using a version of the google.golang.org/genproto parent earlier than v0.0.0-20230525234044-86246a831fb4 may then encounter "ambiguous import" errors when importing packages that have been moved to submodules. Those errors can be resolved by updating to a version at or above version v0.0.0-20230526161137-0005af68ea54 with the following command:
go get google.golang.org/genproto@latest
go mod tidy
References
We're going to migrate but curious regarding the behavior after 2024-03-01
Client code still using a version of the google.golang.org/genproto parent earlier than v0.0.0-20230525234044-86246a831fb4 may then encounter "ambiguous import" errors when importing packages that have been moved to submodules
If we were using offical Go modules proxy which caches the modules indefinitely (from my understanding), will this still error out (or will it be removed from the cache as well)?
It isn't a matter of the old module versions being removed or becoming unavailable, they would still be available via module proxy or git sha.
The "ambiguous import" issue happens when some dependencies in your dependency graph refer to a version of genproto prior to the carveout, some things refer to a version after the carveout along with some of the submodules, and dependency resolution can't figure out which module is supposed to be providing the package.
Making sure you require a version of genproto after the carveout makes it unambiguous which module provides that package.
Closing. as the major projects targeted by this effort have been upgraded to the new submodules.
If you're here because your builds are now breaking within your Google framework dependencies, this is a good example of what happens when a Go project doesn't practice standard Go versioning.
EDIT: In the end, my issue was a dependency on the old api/idtoken that, in some circumstances, created dependency resolution ambiguity. Moving to a combination of cloud.google.com/go/auth/credentials/idtoken and cloud.google.com/go/auth.
While this repository is marked as experimental (Alpha? Beta?), there are versioned, production Google modules that reference this repository via un-versioned tags. IMHO, there's cognitive dissonance here, where either the other modules are actually tainted to Alpha, or this repository really isn't.