Resolving Interference When Merging Models
Paper • 2306.01708 • Published • 18
This is a merge of pre-trained language models created using mergekit.
This model was merged using the TIES merge method using Sakalti/Saka-14B as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
name: NQLSG-Qwen2.5-14B-MegaFusion-TIES-1
models:
- model: YOYO-AI/Qwen2.5-14B-YOYO-1005
- model: Sakalti/ultiima-14B-v0.2
- model: jpacifico/Chocolatine-2-14B-Instruct-v2.0b2
- model: jpacifico/Chocolatine-2-14B-Instruct-v2.0b3
- model: Sakalti/Saka-14B
base_model: Sakalti/Saka-14B
chat_template: auto
dtype: bfloat16
merge_method: ties
parameters:
weight: 1.0
density: 1.0
normalize: true
int8_mask: true
tokenizer:
source: union