UD-IQ3_XXS or Q3_K_S?
#8
by Garpez - opened
Is the Unsloth IQ3 quantization better than the standard Q3 despite the slightly smaller size?
Good question, would like to know too
I'm still waiting for some info about IQ4_XS not working properly. Always use this quant without problems, but this doesn't work, generates almost gibberish.
It's a Vulkan llama.cpp bug, because cpu version works.
mine also generates gibberish on vulkan llama.cpp but with Q4_K_M
mine also generates gibberish on vulkan llama.cpp but with Q4_K_M
yeah, but for no reason, when switching to lm studio, it suddenly works... (also using vulkan)