Add GGUF quantized Sunflower models

.gitattributes

@@ -33,3 +33,10 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+sunflower-imatrix.dat filter=lfs diff=lfs merge=lfs -text
+sunflower-merged-f16.gguf filter=lfs diff=lfs merge=lfs -text
+sunflower-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
+sunflower-q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text
+sunflower-q5_k_s.gguf filter=lfs diff=lfs merge=lfs -text
+sunflower-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
+sunflower-q8_0.gguf filter=lfs diff=lfs merge=lfs -text

Modelfile

@@ -0,0 +1,22 @@
+FROM sunflower-q4_k_m.gguf
+
+# System message
+SYSTEM """You are a linguist and translator specialising in Ugandan languages, made by Sunbird AI."""
+
+TEMPLATE """<|im_start|>system
+{{ .System }}<|im_end|>
+<|im_start|>user
+{{ .Prompt }}<|im_end|>
+<|im_start|>assistant
+{{ .Response }}<|im_end|>"""
+
+# Stop tokens
+PARAMETER stop "<|im_start|>"
+PARAMETER stop "<|im_end|>"
+
+# Quality parameters
+PARAMETER temperature 0.3
+PARAMETER top_p 0.95
+PARAMETER top_k 40
+PARAMETER repeat_penalty 1.1
+PARAMETER num_ctx 4096

README.md

@@ -0,0 +1,91 @@
+---
+license: apache-2.0
+base_model: Sunbird/qwen3-14b-sunflower-merged
+tags:
+- quantized
+- gguf
+- llama.cpp
+- ollama
+- ugandan-languages
+- translation
+- qwen
+library_name: transformers
+pipeline_tag: text-generation
+language:
+- en
+- lg
+---
+
+# Sunflower 14B - GGUF
+
+GGUF quantized versions of the Sunflower model for Ugandan language translation tasks.
+
+## Model Details
+
+- **Base Model**: [Sunbird/qwen3-14b-sunflower-merged](https://huggingface.co/Sunbird/qwen3-14b-sunflower-merged)
+- **Model Size**: 14B parameters
+- **Architecture**: Qwen2.5
+- **Quantization**: K-means quantization with importance matrix
+- **Languages**: English, Luganda, and other Ugandan languages
+
+## Available Files
+
+| Filename | Quant type | File Size | Description |
+| -------- | ---------- | --------- | ----------- |
+| sunflower-merged-f16.gguf | F16 | 28GB | Original precision |
+| sunflower-q8_0.gguf | Q8_0 | 15GB | Highest quality quantized |
+| sunflower-q6_k.gguf | Q6_K | 12GB | High quality |
+| sunflower-q5_k_m.gguf | Q5_K_M | 9.8GB | Balanced quality/size |
+| sunflower-q5_k_s.gguf | Q5_K_S | 9.6GB | Smaller Q5 variant |
+| sunflower-q4_k_m.gguf | Q4_K_M | 8.4GB | Recommended for most users |
+| sunflower-imatrix.dat | - | 7.4MB | Importance matrix data |
+
+## Usage
+
+### llama.cpp
+
+```bash
+# Download model
+huggingface-cli download Sunbird/Sunflower-14B-GGUF sunflower-q4_k_m.gguf --local-dir .
+
+# Run inference
+./llama-cli -m sunflower-q4_k_m.gguf -p "Translate to Luganda: Hello, how are you today?"
+```
+
+### Ollama
+
+```bash
+# Create Modelfile
+echo 'FROM ./sunflower-q4_k_m.gguf' > Modelfile
+
+# Create model
+ollama create sunflower-14b -f Modelfile
+
+# Use model
+ollama run sunflower-14b "Translate to Luganda: Good morning"
+```
+
+### Python (llama-cpp-python)
+
+```python
+from llama_cpp import Llama
+
+llm = Llama(model_path="sunflower-q4_k_m.gguf")
+result = llm("Translate to Luganda: How are you?")
+print(result['choices'][0]['text'])
+```
+
+## Performance Notes
+
+- **Q4_K_M**: Recommended for most use cases
+- **Q5_K_M**: Better quality with moderate size increase
+- **Q6_K**: High quality for production use
+- **Q8_0**: Near-lossless quality
+
+## Technical Details
+
+Quantized using llama.cpp with importance matrix calibration for optimal quality preservation.
+
+## License
+
+Apache 2.0

sunflower-imatrix.dat

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9717ce1e0643a19b3686e86162ce04db8bd01635954eb85284826a3834ebc0f8
+size 7743552

sunflower-merged-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eded257f1acad597e1ddda3f6e69cd387b58bfe8f6ef45dbcae724b68f0b1ba5
+size 29543423264

sunflower-q4_k_m.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d548245e1b7d4d52e9dad363e3f0076dbf8ee8a4e9eaf66332c8b856cfd9bbc
+size 9001753088

sunflower-q5_k_m.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:23c419c7560a205cecc991ca95bd0f7d55b641d1d250efe92172b58c39fb3ee6
+size 10514569728

sunflower-q5_k_s.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d1aa7f358e345bf0bad2ce21f82fb7e830e9650ab5ce6ad2d54dd6362bd9cb6
+size 10263894528

sunflower-q6_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b2074f4414b47b05c7d468cebc9c9ed2a21443cdd8e58cb489d20be4693b942
+size 12121937408

sunflower-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e50d50e7cfd15af8ad6ae0accdf13a8fe740d7daca95d38a36c78034ee8d4f9b
+size 15698533888