Problem with loading tflite model on Android in release version

[mario688]

Hi, there is an issues with the library on Android in the release version.

First:
Function loadTensorflowModel in TensorflowLite.ts use

const asset = Image.resolveAssetSource(source)
uri = asset.uri

In Dev mode, assets are served from the server, so the URI will be: "http://localhost:8081/assets/src/assets/mymodel.flite?platform=android&hash=...." so it works,
but in release mode on Android, asset.uri will be: src_assets_mymodel

Second:
The function fetchByteDataFromUrl in lib/android in TfliteModule.java uses the HTTP request method, which throws an error because it expects the URL scheme to be "http" or "https."

 
  public static byte[] fetchByteDataFromUrl(String url) {
    OkHttpClient client = new OkHttpClient();

    Request request = new Request.Builder().url(url).build();

    try (Response response = client.newCall(request).execute()) {
      if (response.isSuccessful() && response.body() != null) {
        return response.body().bytes();
      } else {
        throw new RuntimeException("Response was not successful!");
      }
    } catch (Exception e) {
      Log.e(NAME, "Failed to fetch URL " + url + "!", e);
      return null;
    }
  }

On iOS, everything works fine

[mrousavy]

Hey - ah, interesting. Well I guess this requires some fixing on the android side then to also make it work in release - if you find a fix let me know!

[mbpictures]

Hi! I stumbled into this issue as well and I fixed it by loading the model file from the Android assets folder when the URL doesn't start with http or https.

--- a/node_modules/react-native-fast-tflite/android/src/main/java/com/tflite/TfliteModule.java
+++ b/node_modules/react-native-fast-tflite/android/src/main/java/com/tflite/TfliteModule.java
@@ -12,6 +12,10 @@ import com.facebook.react.bridge.ReactContextBaseJavaModule;
 import com.facebook.react.bridge.ReactMethod;
 import com.facebook.react.module.annotations.ReactModule;
 import com.facebook.react.turbomodule.core.CallInvokerHolderImpl;
+import java.io.ByteArrayOutputStream;
+import java.io.InputStream;
+import java.io.IOException;
+import android.content.Context;
 
 import okhttp3.OkHttpClient;
 import okhttp3.Request;
@@ -20,10 +24,12 @@ import okhttp3.Response;
 /** @noinspection JavaJniMissingFunction*/
 @ReactModule(name = TfliteModule.NAME)
 public class TfliteModule extends ReactContextBaseJavaModule {
+  static Context context;
   public static final String NAME = "Tflite";
 
   public TfliteModule(ReactApplicationContext reactContext) {
     super(reactContext);
+    this.context = reactContext.getApplicationContext();
   }
 
   @Override
@@ -34,6 +40,22 @@ public class TfliteModule extends ReactContextBaseJavaModule {
 
   @DoNotStrip
   public static byte[] fetchByteDataFromUrl(String url) {
+    if (!url.startsWith("http") && !url.startsWith("https")) {
+      String modelName = url.replace("src_assets_", "");
+      try {
+        InputStream inputStream = TfliteModule.context.getAssets().open("custom/" + modelName + ".tflite");
+        byte[] buffer = new byte[8192];
+        int bytesRead;
+        ByteArrayOutputStream output = new ByteArrayOutputStream();
+        while ((bytesRead = inputStream.read(buffer)) != -1) {
+          output.write(buffer, 0, bytesRead);
+        }
+        return output.toByteArray();
+      } catch (IOException e) {
+        Log.e(NAME, "Failed to load asset modal: " + "custom/" + modelName + ".tflite", e);
+        return null;
+      }
+    }
     OkHttpClient client = new OkHttpClient();
 
     Request request = new Request.Builder().url(url).build();

However, this fix has some drawbacks: As it reads the model from the Android assets folder, you have to use react-native-asset to copy your tflite model to the Android asset directory AND the naming of the model should be something like: "src/assets/MODEL_NAME.tflite". The resolved URL will then be "src_assets_MODEL_NAME.tflite", then the model name can be extracted correctly by the provided patch. I hope this helps anyone!

I also guess this will be an issue on iOS as well, as the passed URL is not valid. So maybe some can rewrite the data loading function of the Objective C code as well, as I have no experience in ObjectiveC.

[mrousavy]

Thanks for the patch @mbpictures - could you maybe create a PR for that?

Not sure if this can be made generic, but it also works for Images in RN so I'm not sure why it wouldnt work for such custom types.

[mrousavy]

See this PR: #30

[AlexHird]

Still facing the same issue in latest version (1.2.0).

I initially attempted to use a require statement, when that was unsuccessful, I tried loading the asset async with expo-assets and pass the file:///... string as the key value to the url parameter. In dev mode, both attempts crashes the Android version but works fine on iOS. When built and running in release mode it does not crash but I get the same error as previously mentioned:

Error: java.lang.IllegalArgumentException: Expected URL scheme 'http' or 'https' but was 'file'

Path during runtime (logged using datadog) was indeed valid: file:///data/user/0/<package-name>/files/.expo-internal/8c27d0eef1a3447e68f5704859bdc418.tflite

Let me know if you need additional info!

[joonshakya]

I was having the same issue of models not working in release builds.
I dug through files removing everything and for me I found out that expo-updates was causing some issues
I removed the expo.updates property from app.json and the model loaded fine in Android release build using the following code:

  const model = useTensorflowModel(
    require("../assets/ai-models/ze-most-amazing-model.tflite"),
  );