ParisNeo/lollms-webui

ggml-vicuna-13b-4bit-rev1 problem

andzejsp opened this issue · 6 comments

andzejsp commented

Something wrong with this model?
image

andzejsp commented

i did conver the model just now and tired to do it again and now it just cuts out mid prompt
image

ParisNeo commented

hi and thanks for trying to test the ui.
I think it may have something to do with the end character. In GPT4All model, it is /f I don't know how vicuna does. I hon't have a good enough connection these days to download the model as I am on vacation. But I'll try to think of a solution.

Bare in mind that there are new updates to the python bindings that will roll out soon and would make the model way more eazy to use. We're working on it.
I also see that you are using an outdated version of the UI. Please upgrade it and try again.

Best regards

andzejsp commented

thanks, keep working on this :D its awsome. maybe you and your team could look into creating matrix server for these types of discussions for people to share what works, and tips and tricks? right now im like a blind man at an orgy.. im feeling things out.

EDIT: found your discord :)

ParisNeo commented

I was planning on doing this :)

macronomicus commented

Im testing this same model now myself, seems to work on the latest build, although im hitting the token limit again which was fixed recently for the standard model we use here.

I guess its because its more verbose, since I provided less prompts but its responses were longer, some even included what appear to be comments from other users, I guess from the training data? Also it claimed to be gpt5 😅

ParisNeo commented

looool. Well for now the token limit is set manually using 5 messages limit. I'll make it adjustable to the model as different models have different token size. Also the limit should be in tokens not in messages.