Different and sometimes wrong answers with ggml-vic13b-q5_1.bin + ggml-vic13b-uncensored-q5_1.bin
breisig opened this issue · 2 comments
After running the latest instructions today to install it, I noticed when trying to run and use ggml-vic13b-q5_1.bin [ ./main -m models/ggml-vic13b-q5_1.bin -f 'prompts/chat-with-vicuna-v1.txt' -r 'User:' --temp 0.36 ] or ggml-vic13b-uncensored-q5_1.bin.
However, every time I stop the app and run it again and ask it the same question, I can get different + wrong answers. For example
User:What is the closest planet to earth?
Vicuna: The closest planet to Earth is Venus, which is about 0.38 AU (5.1 million km or 3.2 million miles) away from Earth on average.
is fine but if I close + start running the app again about 5-8 times, I'll get a different/wrong answer.
User:What is the closest planet to earth?
Vicuna: The closest planet to Earth is the Moon.
Is it normal for ...
- Get different answers asking the same question after a restart?
- It gives wrong answers?
Hi,
Yes its normal if u get different answers.
They are also might be wrong but that's not a issue of the installation guide, as vicuna isn't my language model.
This is due to the temperature parameter set to 0.36. The temperature parameter is used to control the randomness of the model's output. The value is between 0.0 and 1.0. A higher temperature value, such as 1.0, increases the randomness, while a lower value, such as 0.0, reduces it.
When the temperature is set to 0.0, the model tends to produce deterministic responses. In other words, it will consistently generate (almost) the same output for a given input.