oasysai/oasysdb

FEAT: python integration as optional feature

AdrianEddy opened this issue · 6 comments

AdrianEddy commented

Use case

I'm trying to integrate OasysDB as part of a rust staticlib, which gets linked to a C++ based project. I'm not using Python integration at all, and yet all of pyo3 dependencies are compiled and the final binary needs to be linked with python library.

Proposed solution

Please make pyo3 an optional feature.

edwinkys commented

Hi 👋

I did have a plan to configure features for the crate but it got overshadowed by other features. Thank you for bringing this to my attention 😁

  • 👍1
AdrianEddy commented

Thanks! For what it's worth, sled could be optional too, we could then just use bincode::serialize(collection) and store this wherever.

edwinkys commented

Hey, that's interesting to hear. Could you elaborate more about the optional sled?

AdrianEddy commented

Sure, since the database::Database is basically the only place where sled is used, making it optional (but enabled by default) would be pretty simple but potentially useful for some people

My use case is a bit custom, but I actually store the bincode::serialize(collection) in MySQL, which is passed from Rust to C++ and then saved and loaded in MySQL. I'm in the process of rewriting a large C++ project to Rust and we're replacing bits/modules one by one, so there's a fair bit of mixing different approaches until we're fully Rust.
In this one case we just wanted to store these vectors in distributed MySQL instead of local filesystem.
I realize there are distributed vector databases, but OasysDB is nice and compact and works well for this use case

edwinkys commented

I see. Thank you for the detailed explanations. While we are at it, I think I could make this happen as well.

edwinkys commented

Hi Adrian, we just released v0.4.1 that hides PyO3 dependencies and implementations behind the feature py. The APIs should be backward compatible with v0.4.0.

I didn't include the "optional database behind feature" functionality as that would introduce breaking change. This functionality is likely going to be included with the next version of 0.5.0.

Please let me know if everything works well for you. I'd be happy to assist if anything comes up 😁