the mod take so much RAM
Closed this issue · 8 comments
Is this normal? Everytime i run the chatbot it always take all of my memory. It's so laggy.
My Ram; 12GB,
Intel i31005G1
Intel UHD
If you don't have a GPU and a very low end CPU, it is too be expected that RAM usage will be high for chatbot causing a lot of latency. If you can provide information about which model you are currently using, it might explain it.
I'm using Acer Aspire 5 5 a514-53g.
I was asking about which chatbot model you were using.
ohh, it's the one on your tutorial. Is it actually because of the model?
Bigger model also means higher computation ressources usage, but small models are also really bad in terms of quality.
Do you have recommendation in mind for low end though?
Maybe you can try GGUF models with quantized versions that should be lighter with reasonable performance. However, no GPU and an i3 CPU will clearly restrict your usage with high latency even with a smaller model.
I need to upgrade my gear, really. Anyway Thanks!