Hugging Face's logo

Guilherme34
/
Medra4b-abliterated-i1-VISION-GGUF

Summarization
Transformers
GGUF
English
Romanian
text-generation
medical-ai
diagnostic-reasoning
gemma-3b
fine-tuned
imatrix
conversational
Model card Files
xet
 Community
Guilherme34 mradermacher commited on
Commit
44fd04f
·
verified ·
0 Parent(s):

Duplicate from mradermacher/Medra4b-abliterated-i1-GGUF

Browse files

Co-authored-by: team mradermacher <mradermacher@users.noreply.huggingface.co>

Files changed (27) hide show
  1. .gitattributes +60 -0
  2. Medra4b-abliterated.i1-IQ1_M.gguf +3 -0
  3. Medra4b-abliterated.i1-IQ1_S.gguf +3 -0
  4. Medra4b-abliterated.i1-IQ2_M.gguf +3 -0
  5. Medra4b-abliterated.i1-IQ2_S.gguf +3 -0
  6. Medra4b-abliterated.i1-IQ2_XS.gguf +3 -0
  7. Medra4b-abliterated.i1-IQ2_XXS.gguf +3 -0
  8. Medra4b-abliterated.i1-IQ3_M.gguf +3 -0
  9. Medra4b-abliterated.i1-IQ3_S.gguf +3 -0
  10. Medra4b-abliterated.i1-IQ3_XS.gguf +3 -0
  11. Medra4b-abliterated.i1-IQ3_XXS.gguf +3 -0
  12. Medra4b-abliterated.i1-IQ4_NL.gguf +3 -0
  13. Medra4b-abliterated.i1-IQ4_XS.gguf +3 -0
  14. Medra4b-abliterated.i1-Q2_K.gguf +3 -0
  15. Medra4b-abliterated.i1-Q2_K_S.gguf +3 -0
  16. Medra4b-abliterated.i1-Q3_K_L.gguf +3 -0
  17. Medra4b-abliterated.i1-Q3_K_M.gguf +3 -0
  18. Medra4b-abliterated.i1-Q3_K_S.gguf +3 -0
  19. Medra4b-abliterated.i1-Q4_0.gguf +3 -0
  20. Medra4b-abliterated.i1-Q4_1.gguf +3 -0
  21. Medra4b-abliterated.i1-Q4_K_M.gguf +3 -0
  22. Medra4b-abliterated.i1-Q4_K_S.gguf +3 -0
  23. Medra4b-abliterated.i1-Q5_K_M.gguf +3 -0
  24. Medra4b-abliterated.i1-Q5_K_S.gguf +3 -0
  25. Medra4b-abliterated.i1-Q6_K.gguf +3 -0
  26. README.md +93 -0
  27. imatrix.dat +3 -0
.gitattributes ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ imatrix.dat filter=lfs diff=lfs merge=lfs -text
37
+ Medra4b-abliterated.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Medra4b-abliterated.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Medra4b-abliterated.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Medra4b-abliterated.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Medra4b-abliterated.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Medra4b-abliterated.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Medra4b-abliterated.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Medra4b-abliterated.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Medra4b-abliterated.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Medra4b-abliterated.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Medra4b-abliterated.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Medra4b-abliterated.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Medra4b-abliterated.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Medra4b-abliterated.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Medra4b-abliterated.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
52
+ Medra4b-abliterated.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
53
+ Medra4b-abliterated.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
54
+ Medra4b-abliterated.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
55
+ Medra4b-abliterated.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
56
+ Medra4b-abliterated.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
57
+ Medra4b-abliterated.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
58
+ Medra4b-abliterated.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
59
+ Medra4b-abliterated.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
60
+ Medra4b-abliterated.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Medra4b-abliterated.i1-IQ1_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e2600d109e072c50bff7a5559db47fea322e7ef0ea0abd7641bbefe8e0e76995
3
+ size 1419825856
Medra4b-abliterated.i1-IQ1_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1381ed1f07d543654f815ef2f69dd1878298d5cf90a8faf35a4a289e7f75513d
3
+ size 1353347776
Medra4b-abliterated.i1-IQ2_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:372784524b40eb1987195927adbb9aab27436f086b2a39090b7474731372f8a8
3
+ size 1826411456
Medra4b-abliterated.i1-IQ2_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:24f8c068dac8995c2e2a07234fbc991e47daf05565006a189909e5ec8818df13
3
+ size 1737774016
Medra4b-abliterated.i1-IQ2_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a3636ecbb181c7551f89596cf6f7bf746d4e3542685a460cf6816f0d98883f2d
3
+ size 1624830656
Medra4b-abliterated.i1-IQ2_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dce032189af6e8179f2a9745b21902d9bc59345dad3576b073d69096c54e1832
3
+ size 1530622656
Medra4b-abliterated.i1-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:45ef73c91efd57c567733b088767e7b30c69d308a1c6f60f960655540658fc08
3
+ size 2275231936
Medra4b-abliterated.i1-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4d2f816f839b9c1ad789a4c7fc6b73081962ba67b4648f582826cfd4a7c256a
3
+ size 2225793216
Medra4b-abliterated.i1-IQ3_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3c0dc694b9953bbc61fadbbde05f67c64bf6caba99021aec3589a90bcfbea0b
3
+ size 2151819456
Medra4b-abliterated.i1-IQ3_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4796a44ab30fbac355e693001406ab59b4667b9f8f46032dd10885eb23e5e231
3
+ size 1977881536
Medra4b-abliterated.i1-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f3f4bafc3e0e7861e80ce482c24c92302c727bdb55c2585bd028f470e83c051
3
+ size 2741091776
Medra4b-abliterated.i1-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71cf5c7044e22721bca1413c91e201b46a22cef792d4c9b0f3fa67d0e3d4f8e4
3
+ size 2619845056
Medra4b-abliterated.i1-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c4809431757450409ab88b54bbef91100ddc65aac162440b4f2bdf553a9b98a6
3
+ size 1949419456
Medra4b-abliterated.i1-Q2_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f72a3e610a52f14e9022d4ba47fe76668ba793d2fc29f438666b676195870766
3
+ size 1856317376
Medra4b-abliterated.i1-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:157b6f5161dd9d62b8e764eea96c9d952148070a1d1d7528b78dd6509ed444b0
3
+ size 2524514496
Medra4b-abliterated.i1-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8647ce9645a1564300c224d4de70811eb90cd8664d0a4eee0c0b83e13acfea3a
3
+ size 2386888896
Medra4b-abliterated.i1-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae9dbd0efd20eb61a56bf4d7954ebf44d3725aab94158ae20c0001224cf69e20
3
+ size 2225793216
Medra4b-abliterated.i1-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:792aa5ad16cdfcfe23f189497c51884981a9f160c36545ab468aaef1f77dad42
3
+ size 2747645376
Medra4b-abliterated.i1-Q4_1.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0eea3bb1b0a16e57bd4b5b026cacb2541a2663526b409e5e70f3239a2abb076b
3
+ size 2983585216
Medra4b-abliterated.i1-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6db6066ae7e03c788d53f9e71f08e64d10b3d9d3252419436677daf3f952029
3
+ size 2867473856
Medra4b-abliterated.i1-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a63c8e243ad56277d4ec57b40de3d603eb21710002f21215d8258fccefe44bf2
3
+ size 2755509696
Medra4b-abliterated.i1-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6038dd51ca51513481273ce9115afd10e0fcc286fe7fdb170871989e359d79c2
3
+ size 3291184576
Medra4b-abliterated.i1-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:222a11bb80983a3beda2b274fe16307cdbaa9c328e9674699343a1e0d7767fd1
3
+ size 3226078656
Medra4b-abliterated.i1-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4d1b7f0dac7cb4c5edb62529902eb257b0b74fc6a19e3cc151aa19e6e454c36
3
+ size 3741377216
README.md ADDED
@@ -0,0 +1,93 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: drwlf/Medra4b-abliterated
3
+ datasets:
4
+ - drwlf/medra-thinking-768
5
+ language:
6
+ - en
7
+ - ro
8
+ library_name: transformers
9
+ license: apache-2.0
10
+ mradermacher:
11
+ readme_rev: 1
12
+ quantized_by: mradermacher
13
+ tags:
14
+ - text-generation
15
+ - medical-ai
16
+ - summarization
17
+ - diagnostic-reasoning
18
+ - gemma-3
19
+ - fine-tuned
20
+ ---
21
+ ## About
22
+
23
+ <!-- ### quantize_version: 2 -->
24
+ <!-- ### output_tensor_quantised: 1 -->
25
+ <!-- ### convert_type: hf -->
26
+ <!-- ### vocab_type: -->
27
+ <!-- ### tags: nicoboss -->
28
+ weighted/imatrix quants of https://huggingface.co/drwlf/Medra4b-abliterated
29
+
30
+ <!-- provided-files -->
31
+
32
+ ***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Medra4b-abliterated-i1-GGUF).***
33
+
34
+ static quants are available at https://huggingface.co/mradermacher/Medra4b-abliterated-GGUF
35
+
36
+ **This is a vision model - mmproj files (if any) will be in the [static repository](https://huggingface.co/mradermacher/Medra4b-abliterated-GGUF).**
37
+ ## Usage
38
+
39
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
40
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
41
+ more details, including on how to concatenate multi-part files.
42
+
43
+ ## Provided Quants
44
+
45
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
46
+
47
+ | Link | Type | Size/GB | Notes |
48
+ |:-----|:-----|--------:|:------|
49
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ1_S.gguf) | i1-IQ1_S | 1.5 | for the desperate |
50
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ1_M.gguf) | i1-IQ1_M | 1.5 | mostly desperate |
51
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.6 | |
52
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.7 | |
53
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ2_S.gguf) | i1-IQ2_S | 1.8 | |
54
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ2_M.gguf) | i1-IQ2_M | 1.9 | |
55
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q2_K_S.gguf) | i1-Q2_K_S | 2.0 | very low quality |
56
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q2_K.gguf) | i1-Q2_K | 2.0 | IQ3_XXS probably better |
57
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 2.1 | lower quality |
58
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ3_XS.gguf) | i1-IQ3_XS | 2.3 | |
59
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ3_S.gguf) | i1-IQ3_S | 2.3 | beats Q3_K* |
60
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q3_K_S.gguf) | i1-Q3_K_S | 2.3 | IQ3_XS probably better |
61
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ3_M.gguf) | i1-IQ3_M | 2.4 | |
62
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.5 | IQ3_S probably better |
63
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.6 | IQ3_M probably better |
64
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.7 | |
65
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.8 | prefer IQ4_XS |
66
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q4_0.gguf) | i1-Q4_0 | 2.8 | fast, low quality |
67
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.9 | optimal size/speed/quality |
68
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q4_K_M.gguf) | i1-Q4_K_M | 3.0 | fast, recommended |
69
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q4_1.gguf) | i1-Q4_1 | 3.1 | |
70
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q5_K_S.gguf) | i1-Q5_K_S | 3.3 | |
71
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q5_K_M.gguf) | i1-Q5_K_M | 3.4 | |
72
+ | [GGUF](https://huggingface.co/mradermacher/Medra4b-abliterated-i1-GGUF/resolve/main/Medra4b-abliterated.i1-Q6_K.gguf) | i1-Q6_K | 3.8 | practically like static Q6_K |
73
+
74
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
75
+ types (lower is better):
76
+
77
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
78
+
79
+ And here are Artefact2's thoughts on the matter:
80
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
81
+
82
+ ## FAQ / Model Request
83
+
84
+ See https://huggingface.co/mradermacher/model_requests for some answers to
85
+ questions you might have and/or if you want some other model quantized.
86
+
87
+ ## Thanks
88
+
89
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
90
+ me use its servers and providing upgrades to my workstation to enable
91
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
92
+
93
+ <!-- end -->
imatrix.dat ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3e8946dae9ba9a661051173dfb126d9fe7ff9b7b91f22ae3f6b329d5988dbea
3
+ size 3419855