Expose more llama.cpp options
technicalpickles opened this issue · 2 comments
technicalpickles commented
I've been testing https://huggingface.co/TheBloke/wizardLM-7B-GGML , and the example code is:
./main -t 10 -ngl 32 -m wizardLM-7B.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction: Write a story about llamas\n### Response:"
A few of these options (and others) would be useful to expose:
- temp
- repeat_penalty
- top-k
- top-n
yoshoku commented
@technicalpickles
Thank you for your interesting proposal. In version 0.3.2, I made the sampling options the arguments of the generate
module function in LLaMACpp.
https://yoshoku.github.io/llama_cpp.rb/doc/LLaMACpp.html#generate-class_method
https://github.com/yoshoku/llama_cpp.rb/blob/main/CHANGELOG.md#032---2023-07-08
technicalpickles commented
Thank you!