Create README.md

README.md

@@ -0,0 +1,43 @@
+---
+license: apache-2.0
+base_model: google/gemma-3-270m
+tags:
+- alignment
+- dpo
+- triton
+- weight-analysis
+---
+
+# Gemma-3-270m-IT-DPO (Weight Delta Analysis)
+
+This model is a fine-tuned version of [Gemma-3-270m](https://huggingface.co/google/gemma-3-270m) using **Direct Preference Optimization (DPO)**. Beyond standard alignment, this repository explores structural weight analysis through mask generation from weight deltas.
+
+## 🛠 Technical Features
+
+### 1. Weight Delta Masking
+The repository includes tools to build binary masks from weight delta logs to identify the most significant parameter changes during DPO.
+* **Methods Supported:** `Magnitude`, `Momentum`, and `Fisher`.
+* **Comparison Logic:** Includes a Jaccard/IoU (Intersection over Union) method to compare generated masks against the default magnitude mask. 
+    * **Score 0:** No similarity.
+    * **Score 1:** Perfect similarity.
+* **Storage:** All generated masks are saved as `.pt` files in the `/masks` directory.
+
+### 2. Optimized Training Kernels
+To ensure maximum efficiency on high-end compute, the training environment utilizes:
+* **BSR-AdamW Kernel:** A specialized Triton-based optimizer kernel for DPO.
+* **Hardware Compatibility:** Verified for **NVIDIA H100/H200** GPUs.
+* **Triton Validation:** Environment readiness can be tested with a short run (50-100 steps), typically taking only a few minutes on H-series hardware.
+
+## 📂 Repository Structure
+
+* `/final_checkpoint`: The weights of the DPO-tuned model.
+* `/masks`: Contains `.pt` mask files generated using the methods mentioned above.
+
+## 🚀 Reproduction & Debugging
+
+If you are running the mask generation or training scripts:
+* **Jaccard Flag:** If the Jaccard/IoU comparison breaks during execution, it is recommended to disable the `--debug/jaccard` flag temporarily. A fix is scheduled for the upcoming weekend.
+* **Environment Check:** Ensure `triton` is properly installed to handle the BSR-AdamW kernel.
+
+## 📝 Usage Note
+This model is part of an ongoing research project into how DPO shifts model weights. Results from the Jaccard similarity analysis can be used to interpret which parameters are most "critical" for preference alignment.