DualCLIPLoader (GGUF) needs hunyuan type
ABDe3N opened this issue · 1 comments
ABDe3N commented
DualCLIPLoader (GGUF) needs hunyuan type to wrk with the new gguf text encoder
https://huggingface.co/IbnAbdeen/llava-llama-3-8b-text-encoder-tokenizer-Q8_0-GGUF/tree/main
city96 commented
Done: 8d1b56c
This was slightly more elaborate than just adding the mapping since I had to add the reverse of the permute logic llama.cpp uses here but it should work now. I'll put some imatrix quants up in a bit, which should in theory work better than the simple default ones you get from gguf-my-repo.
There's also some differences between whether you convert from the -hf
or the -transformers
repo for LLaVA 8B with the tokenizer missing ~64 extra tokens off the end for some reason.