Create README.md

README.md

@@ -0,0 +1,62 @@
+---
+language:
+- en
+license: lgpl-3.0
+base_model: Qwen/Qwen3.5-27B
+tags:
+- unsloth
+- qwen
+- qwen3.5
+- reasoning
+- chain-of-thought
+- lora
+- moe
+pipeline_tag: text-generation
+---
+
+# 🌟 Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled
+
+## 💡 Model Introduction
+**Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled** is a highly capable reasoning model fine-tuned on top of the powerful Qwen3.5 MoE architecture. The model's core directive is to leverage state-of-the-art Chain-of-Thought (CoT) distillation primarily sourced from Claude-4.6 Opus interactions. 
+
+Through Supervised Fine-Tuning (SFT) focusing specifically on structured reasoning logic, this model excels in breaking down complex user problems, planning step-by-step methodologies within strictly formatted `<think>` tags, and ultimately delivering precise, nuanced solutions. 
+
+## 🗺️ Training Pipeline Overview
+
+```text
+Base Model (Qwen3.5-27B)
+ │
+ ▼
+Supervised Fine-Tuning (SFT) + LoRA
+(Response-Only Training masked on "<|im_start|>assistant\n<think>")
+ │
+ ▼
+Final Model (Claude-4.6-Opus-Reasoning-Distilled)
+```
+
+## 📋 Stage Details
+
+### 🔹 Supervised Fine-Tuning (SFT)
+- **Objective:** To inject high-density reasoning logic and establish a strict format for problem-solving involving an internal thinking state prior to outputting the final response.
+- **Methodology:** We utilized **Unsloth** for highly efficient memory and compute optimization (LoRA Rank = 32). A critical component of this stage is the `train_on_responses_only` strategy, masking instructions so the loss is purely calculated over the generation of the `<think>` sequences and the subsequent solutions. 
+- **Format Enforcement:** All training samples were systematically normalized so the model strictly abides by the structure `<think> {internal reasoning} </think>\n {final answer}`.
+
+### 📚 All Datasets Used
+The dataset consists of high-quality, filtered reasoning distillation data:
+
+| Dataset Name | Description / Purpose |
+|--------------|-----------------------|
+| [nohurry/Opus-4.6-Reasoning-3000x-filtered](https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning-3000x-filtered) | Provides comprehensive Claude 4.6 Opus reasoning trajectories. Sampled ~3,900 rows. |
+| [TeichAI/claude-4.5-opus-high-reasoning-250x](https://huggingface.co/datasets/TeichAI/claude-4.5-opus-high-reasoning-250x) | Injecting high-intensity, structured reasoning instances. Sampled ~350 rows. |
+
+## 🌟 Core Skills & Capabilities
+1. **Modular & Structured Thinking:** Inheriting traits from Opus-level reasoning, the model demonstrates confident parsing of the prompt, establishing an outlined plan in its `<think>` block sequentially rather than exploratory "trial-and-error" self-doubt.
+2. **Extended Context Support:** Fine-tuned smoothly with an 8192 context window allowing complex multi-step reasoning traces to exist gracefully within memory limits.
+3. **MoE Efficiency:** Specifically tuned targeting MoE routing and expert projection layers (`gate_proj`, `up_proj`, `down_proj`, `gate_up_proj`) preserving the inherent inference efficiency of the Qwen3.5 routing layout.
+
+## ⚠️ Limitations & Intended Use
+- **Hallucination Risk:** While reasoning is strong, the model remains an autoregressive LLM; external facts provided during the thinking sequence may occasionally contain hallucinations if verifying real-world events.
+- **Intended Scenario:** Best suited for offline analytical tasks, coding, math, and heavy logic-dependent prompting where the user needs to transparently follow the AI's internal logic.
+
+## 🙏 Acknowledgements
+Significant thanks to the [Unsloth AI](https://unsloth.ai/) team for making rapid fine-tuning of MoE and large LLM models accessible. Additionally, we acknowledge Qwen internally, and the open-source community developers producing exceptional distilled datasets (`nohurry` and `TeichAI`).