krampenschiesser commited on
Commit
382e6ac
·
verified ·
1 Parent(s): 2c9e724

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -12,8 +12,9 @@ tags:
12
  - nvfp4
13
  ---
14
 
15
- Some quants I use depending on the memory availability at nvfp4 in the hope for custom kernels.
16
- I recommend the Q3-IQ4XS and IQ4XS-Q5K quants. I currently use IQ4XS-Q4K.
 
17
 
18
  # KLD
19
 
@@ -27,8 +28,8 @@ Sadly I am getting weird outputs (nan floats from llama-perplexity) from some kl
27
  |KS |Q8 | |7.0266 +/- 0.05210 |baseline |baseline |
28
  |KS |IQ4XS-Q5K |135.5 | | |90.720 ± 0.077 % |
29
  |KS |IQ4XS |123.8 |7.153799 ± 0.053213 |0.086127 ± 0.001029 |89.425 ± 0.082 % |
 
30
  |KS |NVFP4 |130.8 |7.177182 ± 0.053324 |0.105053 ± 0.001034 |88.154 ± 0.086 % |
31
  |unsloth |UD-Q4_K_XL |141 | | |86.990 ± 0.090 % |
32
- |KS |IQ4XS-Q4K |126.1 | | | |
33
  |KS |Q3K-IQ4XS |108.6 |7.297092 ± 0.054489 |0.140361 ± 0.001216 |86.387 ± 0.091 % |
34
 
 
12
  - nvfp4
13
  ---
14
 
15
+ These are some quants I use depending on the memory availability.
16
+ I also added nvfp4 in the hope for custom kernels emerging in the future.
17
+ I recommend the Q3K-IQ4XS and IQ4XS-Q5K quants.
18
 
19
  # KLD
20
 
 
28
  |KS |Q8 | |7.0266 +/- 0.05210 |baseline |baseline |
29
  |KS |IQ4XS-Q5K |135.5 | | |90.720 ± 0.077 % |
30
  |KS |IQ4XS |123.8 |7.153799 ± 0.053213 |0.086127 ± 0.001029 |89.425 ± 0.082 % |
31
+ |KS |IQ4XS-Q4K |126.1 | | |89.205 ± 0.083 % |
32
  |KS |NVFP4 |130.8 |7.177182 ± 0.053324 |0.105053 ± 0.001034 |88.154 ± 0.086 % |
33
  |unsloth |UD-Q4_K_XL |141 | | |86.990 ± 0.090 % |
 
34
  |KS |Q3K-IQ4XS |108.6 |7.297092 ± 0.054489 |0.140361 ± 0.001216 |86.387 ± 0.091 % |
35