Viper doesn't work with etcd
Closed this issue · 21 comments
import (
"github.com/spf13/viper"
_ "github.com/spf13/viper/remote"
)
func main() {
if err := viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001","/config"); err != nil {
panic(err)
}
viper.SetConfigType("yaml")
if err := viper.ReadRemoteConfig(); err != nil {
panic(err)
}
}
I have error:
panic: codecgen version mismatch: current: 8, need 10. Re-generate file: ...my-app/pkg/mod/github.com/coreos/etcd@v3.3.10+incompatible/client/keys.generated.go
I tried
go get github.com/ugorji/go v1.1.1
and had error:
../pkg/mod/github.com/coreos/etcd@v3.3.10+incompatible/client/keys.generated.go:15:2: unknown import path "github.com/ugorji/go/codec": ambiguous import: found github.com/ugorji/go/codec in multiple modules:
github.com/ugorji/go v1.1.1 (...my-app/pkg/mod/github.com/ugorji/go@v1.1.1/codec)
github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8 (...my-app/pkg/mod/github.com/ugorji/go/codec@v0.0.0-20181204163529-d75b2dcb6bc8)
go.sum
github.com/DataDog/zstd v1.3.5 h1:DtpNbljikUepEPD16hD4LvIcmhnhdLTiW/5pHgbmp14=
github.com/DataDog/zstd v1.3.5/go.mod h1:1jcaCB/ufaK+sKp1NBhlGmpz41jOoPQ35bpF36t7BBo=
github.com/Shopify/sarama v1.21.0 h1:0GKs+e8mn1RRUzfg9oUXv3v7ZieQLmOZF/bfnmmGhM8=
github.com/Shopify/sarama v1.21.0/go.mod h1:yuqtN/pe8cXRWG5zPaO7hCfNJp5MwmkoJEoLjkm5tCQ=
github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI=
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6 h1:G1bPvciwNyF7IUmKXNt9Ak3m6u9DE1rF+RmtIkBpVdA=
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
github.com/coreos/etcd v3.3.10+incompatible h1:jFneRYjIvLMLhDLCzuTuU4rSJUjRplcJQ7pD7MnhC04=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
github.com/coreos/go-semver v0.2.0 h1:3Jm3tLmsgAYcjC+4Up7hJrFBPr+n7rAqYeSw/SZazuY=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/eapache/go-resiliency v1.1.0 h1:1NtRmCAqadE2FN4ZcN6g90TP3uk8cg9rn9eNK2197aU=
github.com/eapache/go-resiliency v1.1.0/go.mod h1:kFI+JgMyC7bLPUVY133qvEBtVayf5mFgVsvEsIPBvNs=
github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21 h1:YEetp8/yCZMuEPMUDHG0CW/brkkEp8mzqk2+ODEitlw=
github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21/go.mod h1:+020luEh2TKB4/GOp8oxxtq0Daoen/Cii55CzbTV6DU=
github.com/eapache/queue v1.1.0 h1:YOEu7KNc61ntiQlcEeUIoDTJ2o8mQznoNvUhiigpIqc=
github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=
github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db h1:woRePGFeVFfLKN/pOkfl+p/TAqKOfFu+7KPlMVpok/w=
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
github.com/klauspost/compress v1.4.0/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.4.1 h1:8VMb5+0wMgdBykOV96DwNwKFQ+WTI4pzYURP99CcB9E=
github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/cpuid v0.0.0-20180405133222-e7e905edc00e/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/klauspost/cpuid v1.2.0 h1:NMpwD2G9JSFOE1/TJjGSo5zG7Yb2bTe7eq1jH+irmeE=
github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/magiconair/properties v1.8.0 h1:LLgXmsheXeRoUOBOjtwPQCWIYqM/LU1ayDtDePerRcY=
github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/mitchellh/mapstructure v1.1.2 h1:fmNYVwqnSfB9mZU6OS2O6GsXM+wcskZDuKQzvN1EDeE=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/pelletier/go-toml v1.2.0 h1:T5zMGML61Wp+FlcbWjRDT7yAxhJNAiPPLOFECq181zc=
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/pierrec/lz4 v2.0.5+incompatible h1:2xWsjqPFWcplujydGg4WmhC/6fZqK42wMM8aXeqhl0I=
github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a h1:9ZKAASQSHhDYGoxY8uLVpewe1GDZ2vu2Tr/vTdVAkFQ=
github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
github.com/spf13/afero v1.1.2 h1:m8/z1t7/fwjysjQRYbP0RD+bUIF/8tJwPdEZsI83ACI=
github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
github.com/spf13/cast v1.3.0 h1:oget//CVOEoFewqQxwr0Ej5yjygnqGkvggSE/gB35Q8=
github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/jwalterweatherman v1.0.0 h1:XHEdyB+EcvlqZamSM4ZOMGlc93t6AcsBEu9Gc1vn7yk=
github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/viper v1.3.1 h1:5+8j8FTpnFV4nEImW/ofkzEt8VoOiLXxdYIDsB73T38=
github.com/spf13/viper v1.3.1/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/ugorji/go v1.1.1 h1:gmervu+jDMvXTbcHQ0pd2wee85nEoE0BsVyEuzkfK8w=
github.com/ugorji/go v1.1.1/go.mod h1:hnLbHMwcvSihnDhEfx2/BzKp2xb0Y+ErdfYcrs9tkJQ=
github.com/ugorji/go v1.1.2 h1:JON3E2/GPW2iDNGoSAusl1KDf5TRQ8k8q7Tp097pZGs=
github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8 h1:3SVOIvH7Ae1KRYyQWRjXWJEA9sS/c/pjvH++55Gr648=
github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasthttp v1.2.0 h1:dzZJf2IuMiclVjdw0kkT+f9u4YdrapbNyGAN47E/qnk=
github.com/valyala/fasthttp v1.2.0/go.mod h1:4vX61m6KN+xDduDNwXrhIAVZaZaZiQ1luJk8LWSxF3s=
github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a/go.mod h1:v3UYOV9WzVtRmSR+PDvWpU/qWl4Wa5LApYYX4ZtKbio=
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77 h1:ESFSdwYZvkeru3RtdrYueztKhOBCSAAzS4Gf+k0tEow=
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
go.uber.org/atomic v1.3.2 h1:2Oa65PReHzfn29GpvgsYwloV9AVFHPDk8tYxt2c2tr4=
go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/multierr v1.1.0 h1:HoEmRHQPVSqub6w2z2d2EOVs2fjyFRGyofhKuyDq0QI=
go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
go.uber.org/zap v1.9.1 h1:XCJQEf3W6eZaVwhRBof6ImoYGJSITeKWsyeh3HFu/5o=
go.uber.org/zap v1.9.1/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9 h1:mKdxBk7AujPs8kU4m80U72y/zjbZ3UcXC7dClwKbUI0=
golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/net v0.0.0-20180911220305-26e67e76b6c3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a h1:1n5lsVfiQW3yfsRGu98756EH1YthsFqr/5mxHduZW2A=
golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
etcd-io/etcd#10325 It is same problem
I still have this problem,using the latest version of viper(v1.3.2) and go mod.
I did rm github.com/coreos/etcd/client/keys.generated.go and solved the problem.
The file should be generated auto,so I wonder how it works under go mod.It is said that go-etcd is "DEPRECATED" and we should "use the official Go client",but xordataexchange/crypt which viper use to contect etcd seems have not been updated for a long time.I think it's the matter.
@amarox I had the same problem. For now, It should be OK. I hope ugorji/go#43 can help you.
same problem
panic: codecgen version mismatch: current: 8, need 10. Re-generate file: .../vendor/github.com/coreos/etcd/client/keys.generated.go
goroutine 1 [running]:
github.com/coreos/etcd/client.init.0()
.../vendor/github.com/coreos/etcd/client/keys.generated.go:45 +0x135
exit status 2
The solution in gin-gonic/gin#1897 may help you.
I think the ultimate fix for this is to remove viper's indirect dependency on github.com/ugorji/go/codec (or to switch it to github.com/ugorji/go).
This is a major pain point for us trying to move forward to go modules based dependency management as many shared libraries depend on github.com/ugorji/go. Could we get a new release w/o the github.com/ugorji/go/codec dependency, please?
A related thread: https://groups.google.com/forum/#!topic/golang-nuts/0mJh8SkaomA
One workaround you can try is:
go get github.com/ugorji/go/codec@none
From the modules doc:
The version suffix @none indicates that the dependency should be removed entirely, downgrading or removing modules depending on it as needed.
Setting aside any particular workaround, though, it seems at least somewhat likely that a new release of viper is warranted, with an updated go.mod.
Would anyone here be interested in trying a PR for viper?
@bep @spf13 @sagikazarmark any brief thoughts on a way to try to move this forward?
The issue title currently says "etcd", but this is not specific just to ectd.
Here is a playground example: https://play.golang.org/p/QvLwENa4pTp
which fails with:
build tempmod:
cannot load github.com/ugorji/go/codec: ambiguous import:
found github.com/ugorji/go/codec in multiple modules:
github.com/ugorji/go v1.1.4 (/tmp/gopath878249240/pkg/mod/github.com/ugorji/go@v1.1.4/codec)
github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8
(/tmp/gopath878249240/pkg/mod/github.com/ugorji/go/codec@v0.0.0-20181204163529-d75b2dcb6bc8)
found github.com/ugorji/go/codec in multiple modules:
Hm... I have seen that error in several projects lately.
It's very pervasive. It seems like lots of things find their way back to ugorji eventually.
I would guess viper is a large part of it because it is such a popular package, but it seems like everybody depended on ugorji at some point, and is now having to deal with the fallout of some poor use of modules, or just abandon ugorji entirely. (See kubernetes/kubernetes#48287, etcd-io/etcd#10667.)
Unless there is some change to how module resolution works with ambiguous packages, I'm not sure there's any better way to fix this than continuing to update project dependencies so that they no longer depend on an old version of ugorji/go/codec (either by updating to newer versions or just using replace). But the fact that this continues to bubble up to user projects is pretty frustrating and feels like a problem to solve with modules themselves.
(Maybe the problem is just that go modules so persistently pull in pseudo-versions like github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8 // indirect - a commit back from before ugorji supported modules at all!)
By the way, here is how viper gets to an old version of codec:
$ go mod why github.com/ugorji/go/codec
# github.com/ugorji/go/codec
github.com/spf13/viper/remote
github.com/xordataexchange/crypt/config
github.com/xordataexchange/crypt/backend/etcd
github.com/coreos/etcd/client
github.com/ugorji/go/codec
(etcd has since been updated to not even use ugorji!)
Actually, it's interesting that all these issues are caused by a pseudo-version from before there was even a go.mod file in the repo. Since it didn't have a go.mod file, ugorji/go never actually specified whether it ought to be accessed via github.com/ugorji/go or github.com/ugorji/go/codec. I suspect that whoever first added it as a module dependency just happened to end up with github.com/ugorji/go/codec in their go.mod because that was the import path in the project. Just think how much confusion we could have avoided if it had chosen github.com/ugorji/go instead!
Maybe in situations like this, the go module solver could attempt to resolve the ambiguity by trying with a less specific path instead (by removing /codec from the path, in this case). Maybe someone like @bcmills can weigh in on whether that's a plausible solution, or if that would have problems I don't foresee.
Maybe in situations like this, the go module solver could attempt to resolve the ambiguity by trying with a less specific path instead
I don't think specificity helps: the problem is that there are too many providers of the package, not too few.
That said, see golang/go#27899 for some possible ways the go command could try to upgrade (or downgrade) past ambiguous imports.
It is probably worth trying just deleting the older module path github.com/ugorji/go/codec from viper's go.mod, and then run go mod tidy, and see how much that helps.
In a quick test:
git clone https://github.com/spf13/viper && cd viper
sed -i '/ugorji/d' go.mod
go mod tidy
go mod why -m github.com/ugorji/go/codec
# github.com/ugorji/go/codec
(main module does not need module github.com/ugorji/go/codec)
go mod why -m github.com/ugorji/go
# github.com/ugorji/go
github.com/spf13/viper/remote
github.com/xordataexchange/crypt/config
github.com/xordataexchange/crypt/backend/etcd
github.com/coreos/etcd/client
github.com/ugorji/go/codec
In other words, the problematic old ugorji/go/codec module path no longer appears in go mod why -m output, whereas the new "good" module path ugorji/go does appear. (Note the -m argument to go mod why is important here to specify the module path; on top of that, go mod why is a bit of an unreliable narrator in general).
And then go get -u seems to work as well.
By the way, here is how viper gets to an old version of codec:
$ go mod why github.com/ugorji/go/codec # github.com/ugorji/go/codec github.com/spf13/viper/remote ...
FWIW, because there is no -m there, that is showing how to get to the package github.com/ugorji/go/codec, which is less interesting question at this point, I think.
The package github.com/ugorji/go/codec is fine. It's the old module path github.com/ugorji/go/codec that is problematic, at least as far as I am following here.
And it goes without saying that this is confusing... ;-)
Good to know...seems odd that a command called mod why requires an extra flag to acknowledge the existence of modules though!
Hi folks,
Please see ugorji/go#299 and add your thoughts/ideas/etc. Thanks.
@bep thanks! And @jadekler thanks for the PR!
FWIW, my failing example from several comments back in #658 (comment) now no longer fails complaining about ambiguous import for github.com/ugorji/go/codec vs. github.com/ugorji/go/codec if I update that example to use the new commit b5bf975e5823 for viper in the go.mod:
https://play.golang.org/p/i6msmOVvznW
@ugorji that is progress...
(That playground example now fails with a different error compaining about fsnotify, but that is apparently because fsnotify doesn't support nacl/amd64p32, which means it fails on the playground... but that is actually a good sign for this particular issue, because it got past the ugorji/go/codec errors).
FYI: I just released a go-codec production release - version 1.1.7 (finally)
First, it resolves the go.mod impasse where we had different import paths (github.com/ugorji/go and github.com/ugorji/go/codec) causing the ambiguous import error.
This is now fixed by leveraging import cycles to ensure that either one works well and resolves to the same bits.
Please let me know if seeing any issues. If all is well over the next few days, I will close this github issue.