city96/ComfyUI-GGUF

DualCLIPLoader (GGUF) needs hunyuan type

ABDe3N opened this issue · 1 comments

DualCLIPLoader (GGUF) needs hunyuan type to wrk with the new gguf text encoder

https://huggingface.co/IbnAbdeen/llava-llama-3-8b-text-encoder-tokenizer-Q8_0-GGUF/tree/main

Done: 8d1b56c

This was slightly more elaborate than just adding the mapping since I had to add the reverse of the permute logic llama.cpp uses here but it should work now. I'll put some imatrix quants up in a bit, which should in theory work better than the simple default ones you get from gguf-my-repo.

There's also some differences between whether you convert from the -hf or the -transformers repo for LLaVA 8B with the tokenizer missing ~64 extra tokens off the end for some reason.