Wrong onnxruntime.dll Picked Up When Running Integration Tests

[mcmah309]

Error

ort 1.16 is not compatible with the ONNX Runtime binary found at `onnxruntime.dll`; expected GetVersionString to return '1.16.x', but got '1.10.0'

How To Replicate

For windows, fastembed does not include onnxruntime.dll in target/debug/deps (which is used by integration tests). Therefore if onnxruntime.dll exists in C:/Windows/System32/ then that version is used. Causing the issue above.

To replicate, ensure a version of onnxruntime.dll exists in C:/Windows/System32/ that is not the same version as fastembed (currently 1.16.x). Adding a fastembed model to a library, like

lazy_static! {
    pub static ref EMBEDDING_MODEL: FlagEmbedding = FlagEmbedding::try_new(InitOptions {
            model_name: EmbeddingModel::BGEBaseEN,
            show_download_message: true,
            ..Default::default()
        }).unwrap();
}

Then importing that module into an integration and attempting to use it that error will cause the error.

How To fix by hand

Add onnxruntime.dll to target/debug/deps

Expected Behavior

If fast fastembed runs correctly in a binary or library crate, it should also run correctly in their integration tests. (Note: I have not tested release mode or regular unit tests with fastembed yet.)

Possible solution

ort has a feature that could be used, but I have not tested. https://github.com/pykeio/ort/blob/4ab57859caa9490473bac3dfcd043dbb1b89d9a5/Cargo.toml#L44

[Anush008]

You should be able to mitigate this by setting the ORT_DYLIB_PATH=/path/to/libonnxruntime.so|dlib env var.

[mcmah309]

Looks like a fair temporary work around. Also looks the the copy-dylibs feature may work https://github.com/pykeio/ort#shared-library-hell . It's probably worth exposing ort features as features of this crate. e.g.

[dependencies]
ort = { version = "1"}

[features]
default = ["ort-load-dynamic"]
ort-copy-libs = ["ort/copy-libs"]
ort-load-dynamic = ["ort/load-dynamic"]
ort-compile-static = ["ort/compile-static"]
...

I believe the above should work.

[yaman]

Looks like a fair temporary work around. Also looks the the copy-dylibs feature may work https://github.com/pykeio/ort#shared-library-hell . It's probably worth exposing ort features as features of this crate. e.g.

[dependencies]
ort = { version = "1"}

[features]
default = ["ort-load-dynamic"]
ort-copy-libs = ["ort/copy-libs"]
ort-load-dynamic = ["ort/load-dynamic"]
ort-compile-static = ["ort/compile-static"]
...

I believe the above should work.

from documentation:

load-dynamic: Loads the ONNX Runtime binaries at runtime via dlopen() without a link dependency on them. 
The path to the binary can be controlled with the environment variable ORT_DYLIB_PATH=/path/to/libonnxruntime.so. 
**This is heavily recommended**, as it mitigates the [shared library hell](https://github.com/pykeio/ort/tree/1.16#shared-library-hell).

load dynamic is heavily recommended

[mcmah309]

Sure, that's why in my example I left "load-dynamic" as the default. But so some of the other options fix the issue as well. In our case, we need a single static binary. Ort was actually just update yesterday, funny enough, to give more control to downstream over ort-sys. https://github.com/pykeio/ort/blob/main/Cargo.toml . The alpha version of 2 was also released yesterday https://github.com/pykeio/ort/releases/tag/v2.0.0-alpha.1 . When version 2 is finally released, this repo should probably expose what ort exposes. I can create a PR for that when v2 releases.

[Anush008]

Resolved in #12.