intel/openvino-rs

Syntax examples

David-OConnor opened this issue · 7 comments

David-OConnor commented

Hi! I think an examples folder with one or more syntax example would help. The rust docs indicate that most functionality is based on the Core struct. It links to the Core C++ docs. When I try to run methods from that page, like GetAvailableDevices(), or get_available_devices() from the Python API, the method is unrecognized. How should you, for example, list available devices in the Rust bindings? Thank you!

abrown commented

I agree. I might even prefer them as doc tests; I've done this before (e.g., here) but more examples would be helpful. Are you interested in helping out with this? (Or, alternately: what parts were confusing when you tried to use the crate?).

Re: your GetAvailableDevices() question, I think that is a slightly different matter: the Rust bindings are incomplete in the sense that they don't implement every possible function from OpenVINO's C API. I only initially added the functions I needed, but I am open to PRs adding more. Do you want to take a stab at it? It looks like we would need to call ie_core_get_available_devices and then read the returned ie_available_devices struct into a Vec<String>.

  • 👍1
David-OConnor commented

I have a fork and am ready to get cranking. Example to start: I'd like to implement these:

use openvino::runtime::{ie_api::Model, AsyncInferQueue, Core, PartialShape};

Would I need to edit openvino-sys directly, or just openvino? I'm guessing both, since I can't find these other than Core in the openveino-sys docs.

Core methods in question would be, say, read_model, and compile_model.

abrown commented

Cool! I think there are a couple of things to note here:

  • the openvino-sys crate is generated from OpenVINO's upstream C API; you can see how this is done in the codegen.rs task. All of the functions you need should be in the generated folder.
  • if the functions you are looking for are not there, it's probably because OpenVINO is in the process of switching to a new API, "API 2.0," which we haven't yet moved to in these bindings but you may be using in C/C++/Python. In other words, we need to shift these Rust bindings from ie_* to ov_*; you can see more of the differences between the current API and API 2.0 here.

Can you give me an idea of whether the APIs you are looking to add are available now or only in API 2.0? If they're available now, you should be good to go to add new Rust functionality; if they're only available in API 2.0, it might be a bit of a heavy lift for you to refactor everything to use the API 2.0 and then add the functionality you want. If it's that latter case, I do plan to get to the "API 2.0 refactor" but it might not happen immediately; I'm available on Zulip if you want to chat about all this.

  • 👍1
David-OConnor commented

Great! I'm using the OpenVINO face recognition demo.

Well, a heavily modified version of it, but the OpenVINO calls are the same. So, this should be the latest API, since I'm running OpenVINO 2023.1.0.

I'm a bit of a learner-by-doer, so I used that Python example as the core of how to use OpenVINO, but don't understand anything beyond it.

I'll put this on hold for now, and use OpenVINO + Python and/or tensorflow-rs, until this has been updated for API=2.0. Ty for all the info!

I wonder if it would be possible to use bindgen to wrap the C OpenVINO directly, without having to manually implement things. I've done that before, eg with CMSIS-DSP. But I don't know how well it would work for stuff that h as an allocator and/or C++ features. You can then wrap the results in a more rust-friendly way, like replacing pointers with refs, or just leave it.

abrown commented

I wonder if it would be possible to use bindgen to wrap the C OpenVINO directly, without having to manually implement things. I've done that before, eg with CMSIS-DSP. But I don't know how well it would work for stuff that h as an allocator and/or C++ features. You can then wrap the results in a more rust-friendly way, like replacing pointers with refs, or just leave it.

That's essentially how these crates work: openvino-sys contains the bindgen-generated C API and then openvino wraps up those unsafe calls in Rust-friendly syntax. I'll revive this issue once I get to refactoring these crates to use "API 2.0."

  • 👍1
abrown commented

@David-OConnor, it's been a while since this issue was opened; should I close it or are you still considering adding some examples?

David-OConnor commented

You can close it; I've moved on; thx!