Registering an ExecutionProvider

[ragyabraham]

Im trying to get FastEmbed to utilise GPU for acceleration using the below code.

let coreml = CoreMLExecutionProvider::default();
if !coreml.is_available().unwrap() {
    eprintln!("Please compile ONNX Runtime with CoreML!");
    std::process::exit(1);
}
let model: FlagEmbedding = FlagEmbedding::try_new(InitOptions {
    model_name: translation,
    show_download_message: true,
    execution_providers: vec![coreml.with_subgraphs().build()],
    ..Default::default()
})?;
let embeddings = model.passage_embed(text, None)?;
Ok(embeddings)

it seems to be making its way to the correct struct

[2024-02-13T05:40:13Z INFO  ort::session] drop; self=SharedSessionInner { session_ptr: 0x10e81f200, allocator: Allocator { ptr: 0x108db2900, is_default: true }, _environment: Environment { execution_providers: [CoreML(CoreMLExecutionProvider { use_cpu_only: false, enable_on_subgraph: true, only_enable_device_with_ane: false })], env_ptr: 0x600001419260 } }

however it still gives me the warning

ort::execution_providers] No execution providers registered successfully. Falling back to CPU.

Not sure what I'm doing wrong here.

[Anush008]

I don't have GPU access. So I can't reproduce.

Maybe you can refer to https://github.com/prattcmp/fastembed-rs which tries to work with GPU and FastEmbed-rs.

[ragyabraham]

it seems you need to install ort with the correct feature flags to get the correct GPU acceleration running. See this issue pykeio/ort#109. Can you please add to your Cargo.toml

[Anush008]

I won't be able to verify this without a GPU. Anyone who could, please try.