Update README.md
Browse files
README.md
CHANGED
|
@@ -12,8 +12,9 @@ tags:
|
|
| 12 |
- nvfp4
|
| 13 |
---
|
| 14 |
|
| 15 |
-
|
| 16 |
-
I
|
|
|
|
| 17 |
|
| 18 |
# KLD
|
| 19 |
|
|
@@ -27,8 +28,8 @@ Sadly I am getting weird outputs (nan floats from llama-perplexity) from some kl
|
|
| 27 |
|KS |Q8 | |7.0266 +/- 0.05210 |baseline |baseline |
|
| 28 |
|KS |IQ4XS-Q5K |135.5 | | |90.720 ± 0.077 % |
|
| 29 |
|KS |IQ4XS |123.8 |7.153799 ± 0.053213 |0.086127 ± 0.001029 |89.425 ± 0.082 % |
|
|
|
|
| 30 |
|KS |NVFP4 |130.8 |7.177182 ± 0.053324 |0.105053 ± 0.001034 |88.154 ± 0.086 % |
|
| 31 |
|unsloth |UD-Q4_K_XL |141 | | |86.990 ± 0.090 % |
|
| 32 |
-
|KS |IQ4XS-Q4K |126.1 | | | |
|
| 33 |
|KS |Q3K-IQ4XS |108.6 |7.297092 ± 0.054489 |0.140361 ± 0.001216 |86.387 ± 0.091 % |
|
| 34 |
|
|
|
|
| 12 |
- nvfp4
|
| 13 |
---
|
| 14 |
|
| 15 |
+
These are some quants I use depending on the memory availability.
|
| 16 |
+
I also added nvfp4 in the hope for custom kernels emerging in the future.
|
| 17 |
+
I recommend the Q3K-IQ4XS and IQ4XS-Q5K quants.
|
| 18 |
|
| 19 |
# KLD
|
| 20 |
|
|
|
|
| 28 |
|KS |Q8 | |7.0266 +/- 0.05210 |baseline |baseline |
|
| 29 |
|KS |IQ4XS-Q5K |135.5 | | |90.720 ± 0.077 % |
|
| 30 |
|KS |IQ4XS |123.8 |7.153799 ± 0.053213 |0.086127 ± 0.001029 |89.425 ± 0.082 % |
|
| 31 |
+
|KS |IQ4XS-Q4K |126.1 | | |89.205 ± 0.083 % |
|
| 32 |
|KS |NVFP4 |130.8 |7.177182 ± 0.053324 |0.105053 ± 0.001034 |88.154 ± 0.086 % |
|
| 33 |
|unsloth |UD-Q4_K_XL |141 | | |86.990 ± 0.090 % |
|
|
|
|
| 34 |
|KS |Q3K-IQ4XS |108.6 |7.297092 ± 0.054489 |0.140361 ± 0.001216 |86.387 ± 0.091 % |
|
| 35 |
|