mybigday/llama.rn

[Android] Seed value does not create deterministic outputs.

Vali-98 opened this issue · 0 comments

As mentioned in the title, setting a seed value does not make an output deterministic on Android.

  • llama.rn version: 0.3.0-rc.13

  • Model used: phi-2.Q3_K_M.gguf

  • Android Devices Tested on: Emulated Pixel 3a - Android 14

Params used:

{
  "frequency_penalty": 0, 
  "grammar": "", 
  "min_p": 0.07, 
  "mirostat": 0, 
  "mirostat_eta": 0.1, 
  "mirostat_tau": 5, 
  "n_predict": 288, 
  "n_threads": 5, 
  "presence_penalty": 0, 
  "prompt": "", 
  "repeat_penalty": 1, 
  "seed": 2, 
  "stop": ["User:", "### Response: "], 
  "temperature": 1, 
  "tfs_z": 1, 
  "top_k": 0, 
  "top_p": 1, 
  "typical_p": 1
}