My app keeps crashing when I initials the model with initLlama

[hamdoudhakem]

So I'm using an emulator that uses Android 12 (API 31, named : S), and whenever I call initLlama inside my app it just crashes. I'm using developement builds btw.

here is my code :

  import * as DocumentPicker from 'react-native-document-picker'
  import {initLlama, LlamaContext} from 'llama.rn'

  ...

  const result = await DocumentPicker.pickSingle({
      type: [DocumentPicker.types.allFiles],  
    })

    console.log('result.copyError',result.copyError)

    if(!result.copyError){

      try{
        const mod = await initLlama({
          model: result.uri,        
        })

        console.log('I\'m Ready!')
        setModel(mod)
      }catch(err){
        console.log('Error while initializing the model',err)
      } 

      console.log('the model was selected succesfully.\nThe file path :',result.uri)
    }else{
      console.log('an Error Occured while selecting the file')
    }

My code goes into the if(!result.copyError) statement and my app crashes at the initLlama function call (I don't get any Alert or error message my app just shuts down).

At first I thought that maybe that has something to do with the model I'm using because I started out with a 3.62GB model, but at the end I used a 322MB model and it still crashed so the model size really isn't a problem, here are the models that I used in this order :

1. bloom_3b.gguf, 3.62GB
2. Phi-3.1-mini-4k-instruct-IQ2_M.gguf, 1.22 GB
3. qwen2-0_5b-instruct-q2_k.gguf, 322MB

I also ended up using the pro-guard rules even thought I didn't even have them unabled to start with, but I did it while staying on the expo managed workflow using This method, maybe I need to do it using native code instead? but it still is something optional so I don't see this fixing the problem.

I'm pretty lost here and I seem to have tested out pretty much every possible solution, So if anyone has an idea of how to fix this or if this is just a bug maybe tied to something like my android version I would appreciate someone telling me what to do.

[Vali-98]

From my experience using llama.rn, doing it this way always seems to fail. This has two steps:

1. Copying the model from storage to app cache.
2. Loading the model.

For whatever reason, llama.rn crashes on the copying part. To solve this on my project, ChatterUI: https://github.com/Vali-98/ChatterUI, I prompt the user to first 'import' the GGUF model which copies the model to a /models dir in the app directory. Then, load it after it is contained within the app directory.

As a side note, Proguard rules shouldn't matter in dev builds (I think) as it doesn't minify any code. Also, proguard rules should be added to your app.json file using expo-build-properties: https://docs.expo.dev/versions/latest/sdk/build-properties/ under expo-build-properties.android.extraProguardRules

[hamdoudhakem]

For whatever reason, llama.rn crashes on the copying part. To solve this on my project, ChatterUI: https://github.com/Vali-98/ChatterUI, I prompt the user to first 'import' the GGUF model which copies the model to a /models dir in the app directory. Then, load it after it is contained within the app directory.

First, Thanks a Lot man! thanks to you I was finally able to initials the model and get it to respond.
I also left a ⭐ on your ChatterUI project 😉.