File size: 6,399 Bytes
69ffb92 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 | ---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- reasoning
- qwen3.5
- conversational
- unsloth
- self-correction
- chain-of-thought
base_model: unsloth/Qwen3.5-27B
pipeline_tag: text-generation
---
# Harmonic-27B
The flagship of the Harmonic series. A reasoning-focused fine-tune of [Qwen 3.5 27B](https://huggingface.co/unsloth/Qwen3.5-27B) trained on the same structurally validated data as [Harmonic-9B](https://huggingface.co/DJLougen/Harmonic-9B) and [Harmonic-2B](https://huggingface.co/DJLougen/Harmonic-2B). Every row passes automated quality gates. No junk, no filler, no shallow traces.
The name comes from harmonic analysis of reasoning patterns — the structural signal that separates genuine thinking from surface-level chain-of-thought.
## Support This Work
I'm a PhD student in visual neuroscience at the University of Toronto who also happens to spend way too much time fine-tuning, merging, and quantizing open-weight models on rented H100s and a local DGX Spark. All training compute is self-funded — balancing GPU costs against a student budget. If my uploads have been useful to you, consider buying a PhD student a coffee. It goes a long way toward keeping these experiments running.
**[Support on Ko-fi](https://ko-fi.com/djlougen)**
---
## Training Approach
Same pipeline as Harmonic-9B. **799 curated rows** — a small, precisely curated dataset instead of tens of thousands of unfiltered examples. The base model already has the knowledge from pretraining — the fine-tune teaches it a reasoning behavior pattern.
Every training row contains explicit self-correction ("wait, that's not right"), verification ("let me check by plugging back in"), and multi-path exploration ("alternatively, I could try..."). The data was generated from multiple frontier models and filtered through a custom structural quality pipeline that enforces reasoning depth, coherence, and flow patterns. 100% of rows pass all quality gates simultaneously.
## Training Data Quality
The same reasoning data as Harmonic-9B and Harmonic-2B, curated using a custom structural process supervision pipeline:
| Metric | Value |
|---|---|
| Signal quality score | 78.7 mean (61.5 min, 90.0 max) |
| Thinking trace depth | 1,667 words average |
| Self-correction | 100% of rows (17.2 per row avg) |
| Verification | 100% of rows (10.3 per row avg) |
| Exploration | 100% of rows (6.3 per row avg) |
| Quality gate pass rate | 100% |
## How It Compares
We ran our structural quality analysis against every major public reasoning dataset used for Opus/Qwen distillation. The results:
| Dataset | Rows | Think Words | Self-Correction | Verification | Exploration | Signal Score | Gate Pass |
|---|---|---|---|---|---|---|---|
| **Harmonic (ours)** | **799** | **1,667** | **100%** | **100%** | **100%** | **78.7** | **100%** |
| Crownelius/Opus-3300x | 2,160 | 188 | 5.9% | 22.6% | 5.2% | 28.0 | 0.1% |
| nohurry/Opus-Filtered | 2,326 | 191 | 6.7% | 24.1% | 5.3% | 28.5 | 0.1% |
| TeichAI/Opus-250x | 250 | 323 | 17.2% | 26.8% | 6.8% | 24.6 | 0.4% |
| Jackrong/Qwen-700x | 633 | 6,653 | 97.5% | 97.6% | 69.8% | 75.6 | 22.7% |
| Bespoke-Stratos-17k | 16,710 | 1,322 | 88.2% | 72.7% | 59.7% | 71.7 | 49.0% |
| glaiveai/reasoning-20m | 22M+ | 799 | 64.1% | 41.4% | 37.3% | 46.2 | 12.8% |
| KingNish/reasoning-20k | 19,944 | 132 | 0.7% | 4.2% | 4.3% | 27.4 | 0.0% |
## Speculative Decoding
Harmonic-27B pairs with [Harmonic-2B](https://huggingface.co/DJLougen/Harmonic-2B) for speculative decoding. Both models share the same training data, reasoning format, and architecture family (Qwen 3.5), which keeps draft token acceptance rates high.
```python
from transformers import AutoModelForCausalLM
target = AutoModelForCausalLM.from_pretrained("DJLougen/Harmonic-27B")
draft = AutoModelForCausalLM.from_pretrained("DJLougen/Harmonic-2B")
outputs = target.generate(
**inputs,
assistant_model=draft,
max_new_tokens=512,
)
```
## Training Configuration
```
base_model: unsloth/Qwen3.5-27B
dataset: 799 curated reasoning rows
epochs: 1
learning_rate: 1e-4
lr_scheduler: cosine
warmup_ratio: 0.1
max_seq_length: 8192
lora_rank: 32
lora_alpha: 32
dropout: 0.05
micro_batch_size: 1
gradient_accumulation_steps: 4
weight_decay: 0.01
```
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("DJLougen/Harmonic-27B")
tokenizer = AutoTokenizer.from_pretrained("DJLougen/Harmonic-27B")
```
### Reasoning format
The model uses think blocks for reasoning:
```
<|thinking|>
The user is asking about X. Let me consider two approaches...
Approach 1: ...
Approach 2: ...
I will go with Approach 1 because...
Wait, I need to be careful here - this assumes Y, which may not hold.
Let me verify by checking a special case...
Yes, that confirms the result.
<|/thinking|>
[Final answer here]
```
## Intended Use
- Reasoning tasks requiring genuine multi-step thinking
- Mathematical problem-solving with self-correction
- Code analysis and generation with structured verification
- General conversation (conversational ability preserved through training design)
- Target model for speculative decoding with Harmonic-2B
- Base model for Stage 2 agentic fine-tuning
## Limitations
- Reasoning traces can be verbose for simple questions
- Not optimized for tool calling — see [Harmonic-Hermes-9B](https://huggingface.co/DJLougen/Harmonic-Hermes-9B) for agentic use
- Benchmark evaluation is ongoing
## Architecture
- **Base**: Qwen 3.5 27B (27.36B parameters)
- **Training**: LoRA fine-tuning, merged into base weights
- **Precision**: BF16
- **Context**: 8192 tokens
## License
Apache 2.0 — same as the base model. All training data is from Apache 2.0 or MIT licensed sources. Fully commercial use permitted.
## Links
- 9B variant: [DJLougen/Harmonic-9B](https://huggingface.co/DJLougen/Harmonic-9B)
- 9B GGUF: [DJLougen/Harmonic-9B-GGUF](https://huggingface.co/DJLougen/Harmonic-9B-GGUF)
- 2B draft model: [DJLougen/Harmonic-2B](https://huggingface.co/DJLougen/Harmonic-2B)
- Agentic variant: [DJLougen/Harmonic-Hermes-9B](https://huggingface.co/DJLougen/Harmonic-Hermes-9B)
|