Support Android and IOS
thewh1teagle opened this issue · 28 comments
Currently, sherpa-rs supports Windows, Linux, and macOS.
It would be beneficial to add support for Android and iOS as well. At first glance, it seems feasible to link to prebuilt sherpa-onnx libraries. However, it's unclear whether we can easily cross-compile from any platform to any other, and whether Android should use jniLibs or if the approach is more similar to how Linux builds are handled.
@thewh1teagle
I've used flutter_rust_bridge and it's easy to call rust lib
https://cjycode.com/flutter_rust_bridge/
As long as you can successfully run iOS and Android after calling sherpa-rs under this framework, then you are basically successful.
Thanks! I'm wondering how we actually provide a way first to call this library from java / flutter / swift / kotlin.
I mean, I guess we need first the sherpa-onnx android/ios pre compiled libs. should we link them then just like with windows/linux/macos? how it's different?
For Android we might need to support:
armv7-linux-androideabi (32 bit arm)
aarch64-linux-android (64 bit arm)
Plus the NDK
For IOS:
aarch64-apple-ios
By the way, sherpa-onnx also supports HarmonyOS from Huawei
By the way, sherpa-onnx also supports HarmonyOS from Huawei
How can we use sherpa C API in Android / IOS? does sherpa-onnx have prebuilt .so files for Android or .dylib for IOS? (if that's valid)
yes, we use jni
By the way, sherpa-onnx also supports HarmonyOS from Huawei
How can we use sherpa C API in Android / IOS? does sherpa-onnx have prebuilt
.sofiles for Android or.dylibfor IOS? (if that's valid)
sherpa-onnx has always been there, I've run through both android and ios
Some research I did, since I haven't worked with mobile, reveals that Android and iOS cannot directly use native libraries. Instead, Android requires a JNI wrapper library (which is native but wrapped), and iOS also requires a wrapper, though XCode can generate it automatically from a .dylib or .framework for Swift/ObjC.
This means we need to expose a C API from sherpa-rs. However, this seems unnecessary since sherpa-onnx already does this. Instead, I think we should add an option to compile sherpa-rs with the precompiled or source code of sherpa-onnx for Android/iOS. The user can then write their own native wrapper for the specific use case.
https://gist.github.com/thewh1teagle/794add8801e7c0923042ff877b179ee5
Some research I did, since I haven't worked with mobile, reveals that Android and iOS cannot directly use native libraries. Instead, Android requires a JNI wrapper library (which is native but wrapped), and iOS also requires a wrapper, though XCode can generate it automatically from a .dylib or .framework for Swift/ObjC.
This means we need to expose a C API from sherpa-rs. However, this seems unnecessary since sherpa-onnx already does this. Instead, I think we should add an option to compile sherpa-rs with the precompiled or source code of sherpa-onnx for Android/iOS. The user can then write their own native wrapper for the specific use case.
There's no need at all, there's no need to use java ffi and swift ffi to call the C APIs exposed by rust.
And are you sure someone needs to call the C api that is exposed after rust wraps the C++ library?
There's no need at all, there's no need to use java ffi and swift ffi to call the C APIs exposed by rust.
Do you have/know some project that expose Rust to swift/java? Also we don't have currently C API exposed by Rust
And are you sure someone needs to call the C api that is exposed after rust wraps the C++ library?
There's no way to call Rust directly it needs to be exposed as C API (or maybe with some library that makes it easier)
There's no need at all, there's no need to use java ffi and swift ffi to call the C APIs exposed by rust.
See in sherpa-onnx repo, there's complete folder just for JNI exposing
https://github.com/k2-fsa/sherpa-onnx/tree/f3f896146268bf748cf58ad717e82d89d1a913d1/sherpa-onnx/jni
jni is java's ffi 😅😅
I think the essence of making a rust wrapper is to output the ub behavior brought by C++, and provide safe rust code, rather than simply taking the c api and using it directly, so the sub-essence is still unsafe, so the necessity of exposing a c api is questionable. And the security package I wrote hasn't avoided some UB yet, I also need to modify it, add some lifecycle and encapsulation, and put an end to the UB behavior brought by the C API.