v2: fix merge ratio 85/15 — preserve instruction following

README.md

@@ -1,76 +1,39 @@
 ---
-license: apache-2.0
-base_model:
-  - Qwen/Qwen2.5-7B-Instruct
-  - Qwen/Qwen2.5-Coder-7B-Instruct
+base_model: []
+library_name: transformers
 tags:
-  - merge
-  - linear
-  - qwen2.5
-  - coding
-  - chat
-  - mergekit
-language:
-  - en
-pipeline_tag: text-generation
----
-
-# Qwen2.5-7B-ChatCoder
+- mergekit
+- merge
 
-A linearly merged model combining **Qwen2.5-7B-Instruct** and **Qwen2.5-Coder-7B-Instruct** (60% chat / 40% coder).
-
-## Usage
+---
+# merged-model-v2
 
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-import torch
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
 
-model_id = "ragunath-ravi/Qwen2.5-7B-ChatCoder"
-tokenizer = AutoTokenizer.from_pretrained(model_id)
-model = AutoModelForCausalLM.from_pretrained(
-    model_id,
-    dtype=torch.bfloat16,
-    device_map="auto",
-)
-model.eval()
+## Merge Details
+### Merge Method
 
-messages = [
-    {"role": "system", "content": "You are a helpful coding assistant."},
-    {"role": "user",   "content": "Write a binary search in Python."},
-]
-text = tokenizer.apply_chat_template(
-    messages, tokenize=False, add_generation_prompt=True
-)
-inputs = tokenizer(text, return_tensors="pt").to(model.device)
+This model was merged using the [Linear](https://arxiv.org/abs/2203.05482) merge method.
 
-with torch.no_grad():
-    out = model.generate(
-        **inputs,
-        max_new_tokens=512,
-        do_sample=False,
-        temperature=None,
-        top_p=None,
-        repetition_penalty=1.1,
-        eos_token_id=[151645, 151643],  # <|im_end|> and <|endoftext|>
-        pad_token_id=151645,
-    )
-print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
-```
+### Models Merged
 
-## Merge Details
+The following models were included in the merge:
+* /kaggle/input/models/qwen-lm/qwen2.5/transformers/7b-instruct/1
+* /kaggle/input/models/qwen-lm/qwen2.5-coder/transformers/7b/1
 
-| Property | Value |
-|---|---|
-| Method | Linear (weighted average) |
-| Chat instruct weight | 0.6 |
-| Coder weight | 0.4 |
-| dtype | bfloat16 |
+### Configuration
 
-## Hardware
+The following YAML configuration was used to produce this model:
 
-| Precision | VRAM |
-|---|---|
-| bfloat16 | ~16 GB |
-| 4-bit (bnb) | ~5 GB |
+```yaml
+models:
+  - model: /kaggle/input/models/qwen-lm/qwen2.5/transformers/7b-instruct/1
+    parameters:
+      weight: 0.85
+  - model: /kaggle/input/models/qwen-lm/qwen2.5-coder/transformers/7b/1
+    parameters:
+      weight: 0.15
 
-Created by [ragunath-ravi](https://huggingface.co/ragunath-ravi) using [mergekit](https://github.com/arcee-ai/mergekit).
+merge_method: linear
+dtype: bfloat16
+```

mergekit_config.yml

@@ -1,10 +1,10 @@
 models:
   - model: /kaggle/input/models/qwen-lm/qwen2.5/transformers/7b-instruct/1
     parameters:
-      weight: 0.6
+      weight: 0.85
   - model: /kaggle/input/models/qwen-lm/qwen2.5-coder/transformers/7b/1
     parameters:
-      weight: 0.4
+      weight: 0.15
 
 merge_method: linear
 dtype: bfloat16

model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0c334a6fb2b7f9ff76222392c4e0f889f6b4bb04728829d828eb1953b4612dc6
+oid sha256:71bbe7642888659d56f9aa27450fb82d72041d077b869d09d1d6a0f1e6afc282
 size 4976698776

model-00002-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:84d2a06ed8a18c5259fe2415b5150359a4a8c0f3e4208d602e313b25ae1602a1
+oid sha256:f3f2a5df171f8505fb611a2a2b11da62dada95f0fef07bba5b6b01e080257dbf
 size 4932751032

model-00003-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:40bd46ed157269fa8fa817b2beb7876d6c6e810a03108f38d780f0e951d917b5
+oid sha256:5bd2f7ecee402267f702f2be3fbec5765a47a503cd77a4100537ce987d976f78
 size 4991495808

model-00004-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:14636fe9efd734a900818b1ce848377327f7ba652e3b7f936fc6866f9e37791f
+oid sha256:e44bdabe1c577f1a3e689c7b8486c8f0a5a258e83d3815848852d2cca23d71f6
 size 330326240