F5 F5 F5 F5 F5 F5 F5 F5 F5
GGUF come one :D xD
gogogogogogogogogogogogogogogogogogogogo !!!!
couldn't wait, downloading this for testing, until unsloth come out
https://huggingface.co/Jackrong/Qwen3.6-27B-GGUF/blob/main/Qwen3.6-27B-Q5_K_M.gguf
mmprojs first, teasing is at its maximum :D
finally !!
I should be sleeping right now… but no!! I have to stay up for the hype of a new LLM model. 🥲
Sorry haha we had to fix the chat template for Codex / OpenCode support + tool calling fixes hence the delay
Is something wrong with my system or IQ4_XS quant is broken? Some other quants work, but IQ4_XS generates mostly gibberish.
Update: llama.cpp vulkan is the problem, cpu version works.
Is something wrong with my system or IQ4_XS quant is broken? Some other quants work, but IQ4_XS generates mostly gibberish.
I had this issue with IQ3. For a while I thought it was a broken model, until I saw other people using it normally.
On Linux, I switched from driver 590 (CUDA 13) to 570 (CUDA 12.8) and everything went back to normal.
On Linux, I switched from driver 590 (CUDA 13) to 570 (CUDA 12.8) and everything went back to normal.
i'm on windows, llama.cpp vulkan, NO cuda, amd. I use IQ4_xs quants all the time, no problems (usually)
I also have Qwen3.6-35B-A3B-UD-IQ4_XS.gguf. And it works.
Update: llama.cpp vulkan is the problem, cpu version works.