v36: full-FT GRPO with substantive-deletion-aware reward — filler-free 96.9%, sub-del-15-long 0.64%

README.md

@@ -17,39 +17,38 @@ datasets:
 - juanquivilla/sotto-transcript-cleanup
 ---
 
-# SottoASR Transcript Cleanup — LFM2.5-350M MLX 5-bit (v23 + Paragraphs)
+# SottoASR Transcript Cleanup — LFM2.5-350M MLX 5-bit (v36 + Preservation)
 
 [sottoasr.app](https://sottoasr.app) · [Full precision (bf16)](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m) · [MLX 4-bit (smaller)](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m-mlx-4bit) · [Training Dataset](https://huggingface.co/datasets/juanquivilla/sotto-transcript-cleanup)
 
 ## Overview
 
-**MLX 5-bit affine quantization** of [juanquivilla/sotto-cleanup-lfm25-350m](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m) for on-device deployment on Apple Silicon. **This is the recommended variant** for SottoASR's on-device transcript cleanup — minimal quality loss vs full precision, fits in 237 MB, runs at ~85 ms per typical transcript on M-series chips.
+**MLX 5-bit affine quantization** of [juanquivilla/sotto-cleanup-lfm25-350m](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m). The recommended variant for most Apple Silicon users — best size/quality trade-off.
 
-This model powers on-device transcript cleanup in [SottoASR](https://sottoasr.app) — a local, privacy-first speech-to-text application for macOS. It removes filler words, corrects grammar, formats punctuation, handles false starts and self-corrections, and **— new in v23 — restructures long dictations into paragraph-formatted prose**, all locally with zero cloud dependency.
+This model powers on-device transcript cleanup in [SottoASR](https://sottoasr.app) — a local, privacy-first speech-to-text application for macOS. It removes filler words, corrects grammar, formats punctuation, handles false starts and self-corrections, restructures long dictations into paragraph-formatted prose, and **— new in v36 — preserves substantive content reliably even on long inputs**, all locally with zero cloud dependency.
 
-## What's new in v23
+## What's new in v36
 
-v23 adds **paragraph emission** for long-form dictation. The previous v18/v22 production model produced output as a single run-on paragraph regardless of input length. v23 was retrained on a dataset augmented with **4,012 new `paragraph_formatting` samples**, teaching the model to insert `\n\n` paragraph breaks at natural topic / time-reference / discourse-marker boundaries.
+v36 fixes the **aggressive-edits failure mode** that earlier checkpoints occasionally exhibited: on long inputs the model would sometimes delete substantive content along with the fillers. v36 is a GRPO **full fine-tune** (all 354M params trainable, no LoRA) with a substantive-deletion-aware reward. Result: high-substantive-deletion incidence on long inputs drops from **3.85% → 0.64%** while filler-free rate climbs from **50.9% → 96.9%**.
 
-| Capability | v22 (previous prod) | **v23 R6 (this model)** |
+| Capability | v23 baseline | **v36 (this model)** |
 |---|---|---|
-| Paragraph emission rate on long inputs | **0.0 %** | **91.5 %** |
-| ROUGE-L on paragraph-formatted inputs | 0.9521 | **0.9784** |
-| ROUGE-L on standard val set | 0.9539 | 0.9537 |
-| **Filler-Free rate on standard val set** | 90.3 % | **91.0 %** ⭐ |
+| Filler-Free rate | 50.9 % | **96.9 %** ⭐ |
+| Substantive-deletion >15% on long inputs | 3.85 % | **0.64 %** ⭐ |
+| ROUGE-L F1 on long inputs (>100 words) | 0.9242 | **0.9425** |
 
 ## Key Specs
 
 | Property | Value |
 |----------|-------|
-| **Size** | **237 MB** |
+| **Size** | **~237 MB** |
 | **Quantization** | 5-bit affine, group_size=64 |
 | **Effective bits/weight** | 5.502 |
-| **ROUGE-L (val set)** | ~0.9505 (≈ bf16) |
-| **Paragraph rate (long inputs)** | ~89.5 % |
 | **Architecture** | Hybrid: 10 conv + 6 GQA attention (354M params) |
 | **Latency** | ~85 ms average per transcript (M-series) |
 
+Quality at this quantization tracks the bf16 model closely. See the [bf16 model card](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m) for full benchmark numbers.
+
 ## Quantization Recipe
 
 ```bash
@@ -93,21 +92,17 @@ For long dictation that may need paragraph formatting, raise `max_tokens` to 102
 | okay so the thing is basically we're running out of disk space | We're running out of disk space. |
 | uh yes | Yes. |
 
-### NEW in v23: Paragraph emission on long dictations
-
-Multi-topic input is now restructured into paragraphed prose with `\n\n` breaks at natural topic boundaries. See the [bf16 model card](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m) for a full example.
-
-## Benchmark Results
+### Paragraph emission on long dictations (inherited from v23)
 
-Identical to the [bf16 model](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m) within MLX 5-bit quantization noise.
+Multi-topic input is restructured into paragraphed prose with `\n\n` breaks at natural topic boundaries. See the [bf16 model card](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m) for a full example.
 
 ## All Variants
 
 | Variant | Size | Use Case |
 |---------|------|----------|
 | [Full precision (bf16)](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m) | 676 MB | Training, GPU inference |
-| **[MLX 5-bit (this)](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m-mlx-5bit)** | **237 MB** | **Recommended for Apple Silicon** |
-| [MLX 4-bit](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m-mlx-4bit) | 195 MB | Smallest, slight quality trade-off |
+| **[MLX 5-bit (this)](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m-mlx-5bit)** | ~237 MB | **Recommended for Apple Silicon** |
+| [MLX 4-bit](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m-mlx-4bit) | ~195 MB | Smallest, slight quality trade-off |
 
 ## License
 

config.json

@@ -21,6 +21,7 @@
     "eos_token_id": [
         7
     ],
+    "full_attn_idxs": null,
     "hidden_size": 1024,
     "initializer_range": 0.02,
     "intermediate_size": 6656,
@@ -64,9 +65,9 @@
         "rope_theta": 1000000.0,
         "rope_type": "default"
     },
-    "tie_embedding": true,
+    "rope_theta": 1000000.0,
     "tie_word_embeddings": true,
-    "transformers_version": "5.3.0",
+    "transformers_version": "5.6.2",
     "use_cache": false,
     "use_pos_enc": true,
     "vocab_size": 65536

generation_config.json

@@ -5,5 +5,5 @@
     7
   ],
   "pad_token_id": 0,
-  "transformers_version": "5.3.0"
+  "transformers_version": "5.6.2"
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bb28992c3112af29230a97abc161d8cc4fc2e4b3b7736695696d5f9842b4ea01
-size 243830226
+oid sha256:8fdff7017e6929cb5d3bb90e3136da4be6bfa0897109286532486dc92d57ab33
+size 243830312

tokenizer_config.json

@@ -6,6 +6,7 @@
   "extra_special_tokens": [],
   "is_local": true,
   "legacy": false,
+  "local_files_only": false,
   "model_input_names": [
     "input_ids",
     "attention_mask"