obsidiandynamics/goharvest

Update confluent kafka go dep

Closed this issue · 5 comments

B3rs commented

Hi all,
First of all I want to say good job in creating this package.
I tried to use it as a package inside my application but I got stuck in this confluentinc/confluent-kafka-go#450 confluent kafka go issue that has been fixed in v1.4.2.
TL; DR; if you try to vendor confluent kafka go v1.4.0 you end up without a confluent folder and the package won't compile...
Can we please update at least to v1.4.2? (i can provide a pr for that)
thanks

Hi @B3rs, I had the same issue. Not sure how far you got with this, but I solved it by switching from go's native vendoring to a tool called vend (https://github.com/nomad-software/vend). It appears that go's native vendoring does not copy over non-go files (e.g. c headers), and so go reports that files are missing. vend copies over the remaining files, fixing the compilation process. Hope it helps.

B3rs commented

hi @milan-mnf actually we have been using counfluent-kafka-go > 1.4.2 in production for a while and it is working fine without having to resort to third party vendor scripts...
I just think that this library is unmaintained so maybe the best option can be forking it

Let's get this in. (1.4.2)

Looking at the release notes for 1.4.2, they fixed the bundling issue as per @B3rs original comment:

The bundled librdkafka directory (kafka/librdkafka) is no longer pruned by Go mod vendor import.

This was raised with Magnus in early 2020. Glad to see that was finally addressed.
There's also release 1.5.2. We might as well go with the latest if we can.

B3rs commented

that whould be super useful thanks... i can provide a pr for it!
do you have any contributing guidelines?