| ---
|
| language:
|
| - en
|
| license: llama3.2
|
| library_name: transformers
|
| tags:
|
| - facebook
|
| - meta
|
| - pytorch
|
| - llama
|
| - llama-3
|
| - mergekit
|
| - merge
|
| - chat
|
| - moonride
|
| base_model:
|
| - meta-llama/Llama-3.2-3B
|
| - bunnycore/Llama-3.2-3B-Mix-Skill
|
| - bunnycore/Llama-3.2-3B-Sci-Think
|
| - FuseAI/FuseChat-Llama-3.2-3B-Instruct
|
| - theprint/ReWiz-Llama-3.2-3B
|
| pipeline_tag: text-generation
|
| model-index:
|
| - name: Llama-3.2-3B-Khelavaster
|
| results:
|
| - task:
|
| type: text-generation
|
| name: Text Generation
|
| dataset:
|
| name: IFEval (0-Shot)
|
| type: HuggingFaceH4/ifeval
|
| args:
|
| num_few_shot: 0
|
| metrics:
|
| - type: inst_level_strict_acc and prompt_level_strict_acc
|
| value: 49.25
|
| name: strict accuracy
|
| source:
|
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MoonRide/Llama-3.2-3B-Khelavaster
|
| name: Open LLM Leaderboard
|
| - task:
|
| type: text-generation
|
| name: Text Generation
|
| dataset:
|
| name: BBH (3-Shot)
|
| type: BBH
|
| args:
|
| num_few_shot: 3
|
| metrics:
|
| - type: acc_norm
|
| value: 22.69
|
| name: normalized accuracy
|
| source:
|
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MoonRide/Llama-3.2-3B-Khelavaster
|
| name: Open LLM Leaderboard
|
| - task:
|
| type: text-generation
|
| name: Text Generation
|
| dataset:
|
| name: MATH Lvl 5 (4-Shot)
|
| type: hendrycks/competition_math
|
| args:
|
| num_few_shot: 4
|
| metrics:
|
| - type: exact_match
|
| value: 16.16
|
| name: exact match
|
| source:
|
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MoonRide/Llama-3.2-3B-Khelavaster
|
| name: Open LLM Leaderboard
|
| - task:
|
| type: text-generation
|
| name: Text Generation
|
| dataset:
|
| name: GPQA (0-shot)
|
| type: Idavidrein/gpqa
|
| args:
|
| num_few_shot: 0
|
| metrics:
|
| - type: acc_norm
|
| value: 3.69
|
| name: acc_norm
|
| source:
|
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MoonRide/Llama-3.2-3B-Khelavaster
|
| name: Open LLM Leaderboard
|
| - task:
|
| type: text-generation
|
| name: Text Generation
|
| dataset:
|
| name: MuSR (0-shot)
|
| type: TAUR-Lab/MuSR
|
| args:
|
| num_few_shot: 0
|
| metrics:
|
| - type: acc_norm
|
| value: 5.5
|
| name: acc_norm
|
| source:
|
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MoonRide/Llama-3.2-3B-Khelavaster
|
| name: Open LLM Leaderboard
|
| - task:
|
| type: text-generation
|
| name: Text Generation
|
| dataset:
|
| name: MMLU-PRO (5-shot)
|
| type: TIGER-Lab/MMLU-Pro
|
| config: main
|
| split: test
|
| args:
|
| num_few_shot: 5
|
| metrics:
|
| - type: acc
|
| value: 23.57
|
| name: accuracy
|
| source:
|
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MoonRide/Llama-3.2-3B-Khelavaster
|
| name: Open LLM Leaderboard
|
| ---
|
| <img src="https://huggingface.co/MoonRide/Llama-3.2-3B-Khelavaster/resolve/main/Khelavaster.jpg">
|
|
|
| # Intro
|
|
|
| Experimental merge of multiple Llama 3.2 3B models, guided by [MoonRide-Index-v7](https://huggingface.co/datasets/MoonRide/MoonRide-LLM-Index-v7). Created with [mergekit](https://github.com/cg123/mergekit).
|
|
|
| ## Merge Details
|
| ### Merge Method
|
|
|
| This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using [meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) as a base.
|
|
|
| ### Models Merged
|
|
|
| The following models were included in the merge:
|
| * [bunnycore/Llama-3.2-3B-Sci-Think](https://huggingface.co/bunnycore/Llama-3.2-3B-Sci-Think)
|
| * [FuseAI/FuseChat-Llama-3.2-3B-Instruct](https://huggingface.co/FuseAI/FuseChat-Llama-3.2-3B-Instruct)
|
| * [theprint/ReWiz-Llama-3.2-3B](https://huggingface.co/theprint/ReWiz-Llama-3.2-3B)
|
| * [bunnycore/Llama-3.2-3B-Mix-Skill](https://huggingface.co/bunnycore/Llama-3.2-3B-Mix-Skill)
|
|
|
| ### Configuration
|
|
|
| The following YAML configuration was used to produce this model:
|
|
|
| ```yaml
|
| models:
|
| - model: bunnycore/Llama-3.2-3B-Mix-Skill
|
| - model: bunnycore/Llama-3.2-3B-Sci-Think
|
| - model: FuseAI/FuseChat-Llama-3.2-3B-Instruct
|
| - model: theprint/ReWiz-Llama-3.2-3B
|
| base_model: meta-llama/Llama-3.2-3B
|
| tokenizer:
|
| source: meta-llama/Llama-3.2-3B-Instruct
|
| merge_method: sce
|
| parameters:
|
| normalize: true
|
| dtype: float32
|
| out_dtype: float16
|
|
|
| ```
|
|
|
| # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) |
| Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/MoonRide__Llama-3.2-3B-Khelavaster-details) |
|
|
| | Metric |Value| |
| |-------------------|----:| |
| |Avg. |20.14| |
| |IFEval (0-Shot) |49.25| |
| |BBH (3-Shot) |22.69| |
| |MATH Lvl 5 (4-Shot)|16.16| |
| |GPQA (0-shot) | 3.69| |
| |MuSR (0-shot) | 5.50| |
| |MMLU-PRO (5-shot) |23.57| |
|
|
|
|
