rockdb and go get
Opened this issue · 6 comments
your using this github.com/tecbot/gorocksdb
There is i believe a better way. Coockroachdb also need rocks bd, and have written a fully go gettable implementation of rockdb. As i understand it the cockroach team do not need all the the API of rocksdb, so your milage may vary.
This is it:
https://github.com/cockroachdb/c-rocksdb
Thanks for this. I was made aware of this a few weeks ago by @vmx.
I agree, having 'go gettable' version is nice, but at the moment we use a rather large surface area of the API and the Cockroachdb version was very up-front about not supporting everything. So that caused me to back off a bit.
That said, @steveyen is currently doing some performance optimizations that side-step the tecbot gorocksdb wrapper as well, so its entirely possible that we might end up moving off tecbot and onto something more direct.
I'll leave this bug open as we evaluate different options.
Wow nice one.
Is this it ?
Steve and I both work cbgt for Couchbase, and that project is using Bleve. However, the performance improvements I was referring to are in this repo. One of the more significant ones involved the putv/mergev/deletev methods on a batch. It lets us build and execute a batch in RocksDB with a single cgo call, instead of what would have previously been thousands of individual cgo calls.
I suspect we'll either move to the cockroachdb version in the future, or dispense with a wrapper entirely.
Ok thanks. It helps me understand the fundamentals. I was concerned about devs and CI being a pain with the CGI aspects.
So, sounds like its just a matter of sucking it up and seeing how painful it becomes. I guess you are doing CI, so I can have a look at the Bleve scriots... If any other thoughts let me know, otherwise please feel free to close this
I just put together a PR which switches us over to the cockroachdb wrapper. Tests pass, but further review is welcome.